NAME
IPC::ConcurrencyLimit - Lock-based limits on cooperative multi-processing
SYNOPSIS
use IPC::ConcurrencyLimit;
sub run {
my $limit = IPC::ConcurrencyLimit->new(
type => 'Flock', # that's also the default
max_procs => 10,
path => '/var/run/myapp', # an option to the locking strategy
);
my $id = $limit->get_lock;
if (not $id) {
warn "Got none of the worker locks. Exiting.";
exit(0);
}
else {
# Got one of the worker locks (ie. number $id)
do_work();
}
# lock released with $limit going out of scope here
}
run();
exit();DESCRIPTION
This module implements a mechanism to limit the number of concurrent processes in a cooperative multiprocessing environment. This is an alternative to, for example, running several daemons.
Roughly speaking, a typical setup would be the following:
Cron starts a new process every minute.
The process attempts to get a lock as shown in synopsis.
If it obtains a lock, it starts working and exits when the work is done.
If not,
max_procsprocesses are already working, so it exits.
This has several distinct advantages over daemons.
Processes do not run as long. Small memory leaks are less likely to become a problem.
Rolling out new code is trivial. No need to do any daemon restarting and worrying about interrupting a unit of work.
No complicated master/slave setups and process/thread pooling.
The implementation uses some form of locking to limit concurrency: There's simply a limited number of locks to go around. The detailed locking implementation is chosen using the type parameter to the constructor. The base distributions ships with one locking strategy only: IPC::ConcurrencyLimit::Lock::Flock for a file-locking based concurrency limit.
Among the other potential strategies that are not part of this distribution are NFS-based locking using File::SharedNFSLock or using MySQL's GET_LOCK. Both of these schemes would allow limiting concurrency across multiple hosts without a special-purpose daemon.
METHODS
new
Creates a new concurrency limit. Creating the object does not lock anything. This requires a followup call to the get_lock() method!
After calling get_lock(), the lock will be held by the IPC::ConcurrencyLimit object and released when either release_lock() is called or the IPC::ConcurrencyLimit object is freed.
Takes named parameters, one of which is the type parameter, which specifies the type of lock to use and defaults to Flock.
The max_procs option indicates the maximum number of locks that can be held at the same time and thus usually the maximum no. of running processes. It defaults to 1.
All concurrency limits that refer to the same resource/limit must use the same setting for max_procs. If not, the behaviour is undefined.
All other named parameters will be passed as options to the lock implementation. See IPC::ConcurrencyLimit::Lock::Flock or other implementations.
get_lock
Creates the actual lock and if successful, returns its id (starting from 1, not 0).
Returns undef if locking was unsuccessful. Multiple calls do not stack. They will simply return the same lock id as long as a lock is held.
release_lock
Releases the lock. Returns 1 if a lock has been released or undef if there had been no lock to be released.
is_locked
Returns whether we have a lock.
lock_id
Returns the id of the lock or undef if there is none.
AUTHOR
Steffen Mueller, smueller@cpan.org
Yves Orton
ACKNOWLEDGMENT
This module was originally developed for booking.com. With approval from booking.com, this module was generalized and put on CPAN, for which the authors would like to express their gratitude.
COPYRIGHT AND LICENSE
(C) 2011 Steffen Mueller. All rights reserved.
This code is available under the same license as Perl version
5.8.1 or higher.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.