/ 
IPC-ConcurrencyLimit-0.14
River stage one • 5 direct dependents • 5 total dependents
/ IPC::ConcurrencyLimit

NAME

IPC::ConcurrencyLimit - Lock-based limits on cooperative multi-processing

SYNOPSIS

use IPC::ConcurrencyLimit;

sub run {
  my $limit = IPC::ConcurrencyLimit->new(
    type      => 'Flock', # that's also the default
    max_procs => 10,
    path      => '/var/run/myapp', # an option to the locking strategy
  );
  
  # NOTE: when $limit goes out of scope, the lock is released
  my $id = $limit->get_lock;
  if (not $id) {
    warn "Got none of the worker locks. Exiting.";
    exit(0);
  }
  else {
    # Got one of the worker locks (ie. number $id)
    do_work();
  }
  
  # lock released with $limit going out of scope here
}

run();
exit();

DESCRIPTION

This module implements a mechanism to limit the number of concurrent processes in a cooperative multiprocessing environment. This is an alternative to, for example, running several daemons.

Roughly speaking, a typical setup would be the following:

  • Cron starts a new process every minute.

  • The process attempts to get a lock as shown in synopsis.

  • If it obtains a lock, it starts working and exits when the work is done.

  • If not, max_procs processes are already working, so it exits.

This has several distinct advantages over daemons.

  • Processes do not run as long. Small memory leaks are less likely to become a problem.

  • Rolling out new code is trivial. No need to do any daemon restarting and worrying about interrupting a unit of work.

  • No complicated master/slave setups and process/thread pooling.

The implementation uses some form of locking to limit concurrency: There's simply a limited number of locks to go around. The detailed locking implementation is chosen using the type parameter to the constructor. The base distributions ships with one locking strategy only: IPC::ConcurrencyLimit::Lock::Flock for a file-locking based concurrency limit.

Among the other potential strategies that are not part of this distribution are NFS-based locking using File::SharedNFSLock or using MySQL's GET_LOCK. Both of these schemes would allow limiting concurrency across multiple hosts without a special-purpose daemon.

METHODS

new

Creates a new concurrency limit. Creating the object does not lock anything. This requires a followup call to the get_lock() method!

After calling get_lock(), the lock will be held by the IPC::ConcurrencyLimit object and released when either release_lock() is called or the IPC::ConcurrencyLimit object is freed.

Takes named parameters, one of which is the type parameter, which specifies the type of lock to use and defaults to Flock.

The max_procs option indicates the maximum number of locks that can be held at the same time and thus usually the maximum no. of running processes. It defaults to 1.

All concurrency limits that refer to the same resource/limit must use the same setting for max_procs. If not, the behaviour is undefined.

All other named parameters will be passed as options to the lock implementation. See IPC::ConcurrencyLimit::Lock::Flock or other implementations.

get_lock

Creates the actual lock and if successful, returns its id (starting from 1, not 0).

Returns undef if locking was unsuccessful. Multiple calls do not stack. They will simply return the same lock id as long as a lock is held.

WARNING! make sure the variable holding the lock remains in scope at all times, otherwise the lock will be released and locking will become apparently ineffective. This is the most common reason for having several concurrent processes running when only one is expected to be alive.

release_lock

Releases the lock. Returns 1 if a lock has been released or undef if there had been no lock to be released.

is_locked

Returns whether we have a lock.

lock_id

Returns the id of the lock or undef if there is none.

heartbeat

Check whether the lock is still valid. If so, returns true. Otherwise, it releases (destroys) the lock and returns false.

AUTHOR

Steffen Mueller, smueller@cpan.org

Yves Orton

David Morel

Matt Koscica, mattk@cpan.org

Ivan Kruglov

ACKNOWLEDGMENT

This module was originally developed for Booking.com. With approval from Booking.com, this module was generalized and put on CPAN, for which the authors would like to express their gratitude.

COPYRIGHT AND LICENSE

(C) 2011-2015 Steffen Mueller. All rights reserved.

This code is available under the same license as Perl version
5.8.1 or higher.

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.