/ 
IPC-Shareable-0.99_01
River stage two • 8 direct dependents • 11 total dependents
/ IPC::Shareable

NAME

IPC::Shareable - Share variables across separate processes

SYNOPSIS

DESCRIPTION

IPC::Shareable allows you to tie a variable to shared memory making it easy to share the contents of that variable with other Perl processes.

Scalars, arrays, and hashes can be tied. The variable being tied may contain arbitrarily complex data structures - including references to arrays, hashes of hashes, etc.

The association between variables in distinct processes is provided by GLUE. This is an integer number or 4 character string[1] that serves as a common identifier for data across process space. Hence the statement

tie $scalar, 'IPC::Shareable', 'data';

in program one and the statement

tie $variable, 'IPC::Shareable', 'data';

in program two will bind $scalar in program one and $variable in program two.

There is no pre-set limit to the number of processes that can bind to data; nor is there a pre-set limit to the complexity of the underlying data of the tied variables[2]. The amount of data that can be shared within a single bound variable is limited by the system's maximum size for a shared memory segment (the exact value is system-dependent).

The bound data structures are all linearized (using Raphael Manfredi's Storable module) before being slurped into shared memory. Upon retrieval, the original format of the data structure is recovered. Semaphore flags can be used for locking data between competing processes.

OPTIONS

Options are specified by passing a reference to a hash as the fourth argument to the tie() function that enchants a variable. Alternatively you can pass a reference to a hash as the third argument; IPC::Shareable will then look at the field named key in this hash for the value of GLUE. So,

tie $variable, 'IPC::Shareable', 'data', \%options;

is equivalent to

tie $variable, 'IPC::Shareable', { key => 'data', ... };

The following fields are recognized in the options hash.

key

The key field is used to determine the GLUE when using the three-argument form of the call to tie(). This argument is then, in turn, used as the KEY argument in subsequent calls to shmget() and semget().

The default value is IPC_PRIVATE, meaning that your variables cannot be shared with other processes.

Default: IPC_PRIVATE

create

create is used to control whether calls to tie() create new shared memory segments or not. If create is set to a true value, IPC::Shareable will create a new binding associated with GLUE as needed. If create is false, IPC::Shareable will not attempt to create a new shared memory segment associated with GLUE. In this case, a shared memory segment associated with GLUE must already exist or the call to tie() will fail and return undef.

Default: false

exclusive

If exclusive field is set to a true value, calls to tie() will fail (returning undef) if a data binding associated with GLUE already exists. If set to a false value, calls to tie() will succeed even if a shared memory segment associated with GLUE already exists.

Default: false

persist

This option if set to a true value will allow you to retain access to the shared memory segments underlying the data, even after all processes finish.

Meaning that you can regain access to the shared data at any later time. This feature also allows you to share data between completely separate processes (ie: two separate scripts running in separate windows), not only between forked processes.

NOTE: Only hashes can be used in persist mode, and all values of the top-level hash must be either scalar or hash reference values. If you try to save any other structure underneath of your hash, we croak().

Default: false

mode

The mode argument is an octal number specifying the access permissions when a new data binding is being created. These access permission are the same as file access permissions in that 0666 is world readable, 0600 is readable only by the effective UID of the process creating the shared variable, etc.

Default: 0666 (world read and writeable)

destroy

If set to a true value, the shared memory segment underlying the data binding will be removed when the process calling tie() exits (gracefully)[3].

Only those memory segments that were created by the current process will be removed.

Use this option with care. In particular you should not use this option in a program that will fork after binding the data. On the other hand, shared memory is a finite resource and should be released if it is not needed.

NOTE: If the persist option is set to a true value, the memory segments used to house the shared data structure will *NOT* be removed, even if destroy is set to a true value.

Default: false

size

This field may be used to specify the size of the shared memory segment allocated.

Default: IPC::Shareable::SHM_BUFSIZ() (ie. 65536)

Default Option Values

Default values for options are:

key       => IPC_PRIVATE,
create    => 0,
exclusive => 0,
persist   => 0,
destroy   => 0,
mode      => 0,
size      => IPC::Shareable::SHM_BUFSIZ(),

METHODS

erase

Class/tied object call.

Example:

IPC::Shareable->erase;

# or

tied($var)->erase;

# or

$knot->erase;

This method will absolutely and completely remove all semaphores and shared memory segments related to the shared data, regardless of which process created them, or whether the destroy parameter is set.

It's primary use is for when using the persist option, and you know for fact that all other work in all other processes is complete.

persist_ids

Class/tied object call. Example:

my @ids = IPC::Shareable->persist_ids;

# or

my @ids = tied($var)->persist_ids;

# or

my @ids = $knot->persist_ids;

Returns an array of IDs that are currently registered for the persistent parts of the data structure, when the persist option has been used.

shlock($type)

Obtains a lock on the shared memory. $type specifies the type of lock to acquire. If $type is not specified, an exclusive read/write lock is obtained. Acceptable values for $type are the same as for the flock() system call.

Returns true on success, and undef on error. For non-blocking calls (see below), the method returns 0 if it would have blocked.

Obtain an exclusive lock like this:

tied($var)->lock(LOCK_EX); # same as default

Only one process can hold an exclusive lock on the shared memory at a given time.

Obtain a shared lock this this:

tied($var)->lock(LOCK_SH);

Multiple processes can hold a shared lock at a given time. If a process attempts to obtain an exclusive lock while one or more processes hold shared locks, it will be blocked until they have all finished.

Either of the locks may be specified as non-blocking:

tied($var)->lock( LOCK_EX|LOCK_NB );
tied($var)->lock( LOCK_SH|LOCK_NB );

A non-blocking lock request will return 0 if it would have had to wait to obtain the lock.

Note that these locks are advisory (just like flock), meaning that all cooperating processes must coordinate their accesses to shared memory using these calls in order for locking to work. See the flock() call for details.

Locks are inherited through forks, which means that two processes actually can possess an exclusive lock at the same time. Don't do that.

The constants LOCK_EX, LOCK_SH, LOCK_NB, and LOCK_UN are available for import using any of the following export tags:

use IPC::Shareable qw(:lock);
use IPC::Shareable qw(:flock);
use IPC::Shareable qw(:all);

Or, just use the flock constants available in the Fcntl module.

See "LOCKING" for further details.

shunlock()

Removes a lock. Takes no parameters, returns true on success.

This is equivalent of calling shlock(LOCK_UN).

See "LOCKING" for further details.

LOCKING

IPC::Shareable provides methods to implement application-level advisory locking of the shared data structures. These methods are called shlock() and shunlock(). To use them you must first get the object underlying the tied variable, either by saving the return value of the original call to tie() or by using the built-in tied() function.

To lock and subsequently unlock a variable, do this:

my $knot = tie my $scalar, 'IPC::Shareable', $glue, { %options };

$knot->shlock;
$scalar = 'foo';
$knot->shunlock;

or equivalently, if you've decided to throw away the return of tie():

tie my $scalar, 'IPC::Shareable', $glue, { %options };

tied($sv)->shlock;
$scalar = 'foo';
tied($sv)->shunlock;

This will place an exclusive lock on the data of $scalar. You can also get shared locks or attempt to get a lock without blocking.

IPC::Shareable makes the constants LOCK_EX, LOCK_SH, LOCK_UN, and LOCK_NB exportable to your address space with the export tags :lock, :flock, or :all. The values should be the same as the standard flock option arguments.

if (tied($scalar)->shlock(LOCK_SH|LOCK_NB)){
    print "The value is $scalar\n";
    tied($scalar)->shunlock;
} else {
    print "Another process has an exlusive lock.\n";
}

If no argument is provided to shlock, it defaults to LOCK_EX. To unlock a variable do this:

There are some pitfalls regarding locking and signals about which you should make yourself aware; these are discussed in "NOTES".

If you use the advisory locking, IPC::Shareable assumes that you know what you are doing and attempts some optimizations. When you obtain a lock, either exclusive or shared, a fetch and thaw of the data is performed. No additional fetch/thaw operations are performed until you release the lock and access the bound variable again. During the time that the lock is kept, all accesses are perfomed on the copy in program memory. If other processes do not honor the lock, and update the shared memory region unfairly, the process with the lock will not be in sync. In other words, IPC::Shareable does not enforce the lock for you.

A similar optimization is done if you obtain an exclusive lock. Updates to the shared memory region will be postponed until you release the lock (or downgrade to a shared lock).

Use of locking can significantly improve performance for operations such as iterating over an array, retrieving a list from a slice or doing a slice assignment.

REFERENCES

When a reference to a non-tied scalar, hash, or array is assigned to a tie()d variable, IPC::Shareable will attempt to tie() the thingy being referenced[4]. This allows disparate processes to see changes to not only the top-level variable, but also changes to nested data. This feature is intended to be transparent to the application, but there are some caveats to be aware of.

First of all, IPC::Shareable does not (yet) guarantee that the ids shared memory segments allocated automagically are unique. The more automagical tie()ing that happens, the greater the chance of a collision.

Secondly, since a new shared memory segment is created for each thingy being referenced, the liberal use of references could cause the system to approach its limit for the total number of shared memory segments allowed.

OBJECTS

IPC::Shareable implements tie()ing objects to shared memory too. Since an object is just a reference, the same principles (and caveats) apply to tie()ing objects as other reference types (see "REFERENCES").

DESTRUCTION

perl(1) will destroy the object underlying a tied variable when then tied variable goes out of scope. Unfortunately for IPC::Shareable, this may not be desirable: other processes may still need a handle on the relevant shared memory segment.

IPC::Shareable therefore provides several options to control the timing of removal of shared memory segments.

destroy Option

As described in "OPTIONS", specifying the destroy option when tie()ing a variable coerces IPC::Shareable to remove the underlying shared memory segment when the process calling tie() exits gracefully.

NOTE: The destruction is handled in an END block. Only those memory segments that are tied to the current process will be removed.

NOTE: If the persist option is specified on a data structure, we will remove only the memory segments that aren't required for persistent storage of the variable.

remove()

$knot->remove;

# or

tied($var)->remove;

Calling remove() on the object underlying a tie()d variable removes the associated shared memory segments. The segment is removed irrespective of whether it has the destroy option set or not and irrespective of whether the calling process created the segment.

NOTE: If the persist option is set, we will retain just enough memory segments to keep the shared data available for future runs.

clean_up()

IPC::Shareable->clean_up;

# or

tied($var)->clean_up;

# or

$knot->clean_up;

This is a class method that provokes IPC::Shareable to remove all shared memory segments created by the process. Segments not created by the calling process are not removed. We'll retain the minimum segments for persistent storage if persist is set to a true value.

clean_up_all()

IPC::Shareable->clean_up_all;

# or

tied($var)->clean_up_all;

# or

$knot->clean_up_all

This is a class method that provokes IPC::Shareable to remove all shared memory segments encountered by the process. Segments are removed even if they were not created by the calling process. We'll retain the minimum segments for persistent storage if persist is set to a true value.

RETURN VALUES

Calls to tie() that try to implement IPC::Shareable will return an instance of IPC::Shareable on success, and undef otherwise.

AUTHOR

Benjamin Sugars <bsugars@canoe.ca>

NOTES

Footnotes from the above sections

  1. If GLUE is longer than 4 characters, only the 4 most significant characters are used. These characters are turned into integers by unpack()ing them. If GLUE is less than 4 characters, it is space padded.

  2. IPC::Shareable provides no pre-set limits, but the system does. Namely, there are limits on the number of shared memory segments that can be allocated and the total amount of memory usable by shared memory.

  3. If the process has been smoked by an untrapped signal, the binding will remain in shared memory. If you're cautious, you might try

    $SIG{INT} = \&catch_int;
sub catch_int {
    die;
}
...
tie $variable, IPC::Shareable, 'data', { 'destroy' => 'Yes!' };

    which will at least clean up after your user hits CTRL-C because IPC::Shareable's END method will be called. Or, maybe you'd like to leave the binding in shared memory, so subsequent process can recover the data...

  4. This behaviour is markedly different from previous versions of IPC::Shareable. Older versions would sometimes tie() referenced thingies, and sometimes not. The new approach is more reliable (I think) and predictable (certainly) but uses more shared memory segments.

General Notes

o

When using shlock() to lock a variable, be careful to guard against signals. Under normal circumstances, IPC::Shareable's END method unlocks any locked variables when the process exits. However, if an untrapped signal is received while a process holds an exclusive lock, DESTROY will not be called and the lock may be maintained even though the process has exited. If this scares you, you might be better off implementing your own locking methods.

One advantage of using flock on some known file instead of the locking implemented with semaphores in IPC::Shareable is that when a process dies, it automatically releases any locks. This only happens with IPC::Shareable if the process dies gracefully.

The alternative is to attempt to account for every possible calamitous ending for your process (robust signal handling in Perl is a source of much debate, though it usually works just fine) or to become familiar with your system's tools for removing shared memory and semaphores. This concern should be balanced against the significant performance improvements you can gain for larger data structures by using the locking mechanism implemented in IPC::Shareable.

o

There is a program called ipcs(1/8) (and ipcrm(1/8)) that is available on at least Solaris and Linux that might be useful for cleaning moribund shared memory segments or semaphore sets produced by bugs in either IPC::Shareable or applications using it.

Examples:

# list all semaphores and memory segments in use on the system

ipcs -a

# list all memory segments along with each one's associated process ID

ipcs -ap

# remove *all* semaphores and memory segments

ipcrm -a
o

This version of IPC::Shareable does not understand the format of shared memory segments created by versions prior to 0.60. If you try to tie to such segments, you will get an error. The only work around is to clear the shared memory segments and start with a fresh set.

o

Iterating over a hash causes a special optimization if you have not obtained a lock (it is better to obtain a read (or write) lock before iterating over a hash tied to LShareable>, but we attempt this optimization if you do not).

The fetch/thaw operation is performed when the first key is accessed. Subsequent key and and value accesses are done without accessing shared memory. Doing an assignment to the hash or fetching another value between key accesses causes the hash to be replaced from shared memory. The state of the iterator in this case is not defined by the Perl documentation. Caveat Emptor.

CREDITS

Thanks to all those with comments or bug fixes, especially

Steve Bertrand      <steveb@cpan.org>
Maurice Aubrey      <maurice@hevanet.com>
Stephane Bortzmeyer <bortzmeyer@pasteur.fr>
Doug MacEachern     <dougm@telebusiness.co.nz>
Robert Emmery       <roberte@netscape.com>
Mohammed J. Kabir   <kabir@intevo.com>
Terry Ewing         <terry@intevo.com>
Tim Fries           <timf@dicecorp.com>
Joe Thomas          <jthomas@women.com>
Paul Makepeace      <Paul.Makepeace@realprogrammers.com>
Raphael Manfredi    <Raphael_Manfredi@pobox.com>
Lee Lindley         <Lee.Lindley@bigfoot.com>
Dave Rolsky         <autarch@urth.org>

SEE ALSO

perltie, Storable, shmget, ipcs, ipcrm and other SysV IPC manual pages.