/ 
Mojolicious-Plugin-Buffer-0.001.0
/ Mojolicious::Plugin::Buffer::Lock::File

NAME

Mojolicious::Plugin::Buffer::Lock::File - flock-based lock backend

VERSION

Version 0.001.0

SYNOPSIS

The usual way: give the plugin a directory and let it build this backend.

    use Mojolicious::Lite;
    use CHI;

    plugin 'Buffer' => {
	cache          => CHI->new(driver => 'File', root_dir => '/var/cache/myapp'),
	cache_lock_dir => '/var/lib/myapp/buffer-locks',
    };

Build the backend yourself, for example to reword its messages, and pass it as cache_lock:

    use Mojolicious::Plugin::Buffer::Lock::File;

    my $lock = Mojolicious::Plugin::Buffer::Lock::File->new(
	dir      => '/var/lib/myapp/buffer-locks',
	messages => {lock_failed => 'lock %{path} is broken: %{error}'},
    );
    plugin 'Buffer' => {cache => $cache, cache_lock => $lock};

Use it on its own, for example so that only one copy of a cron job rebuilds a report at a time:

use Mojolicious::Plugin::Buffer::Lock::File;

my $lock = Mojolicious::Plugin::Buffer::Lock::File->new(dir => '/run/myapp');

# Always take BOTH return values (see COMMON PITFALLS)
my ($guard, $error) = $lock->acquire('nightly-report');
die "$error\n" if defined $error;
exit 0 unless $guard;    # another copy is already running

build_report();
$lock->release($guard);

Spread the lock files over sub-directories by overriding the protected _path method in a subclass:

    package MyApp::Lock;
    use Mojo::Base 'Mojolicious::Plugin::Buffer::Lock::File';
    use File::Path ();
    use File::Spec ();
    use Mojo::Util qw(encode md5_sum);

    sub _path {
	my ($self, $key) = @_;
	my $digest = md5_sum(encode('UTF-8', $key));
	my $sub = File::Spec->catdir($self->dir, substr($digest, 0, 2));
	File::Path::make_path($sub);
	return File::Spec->catfile($sub, "$digest.lock");
    }

DESCRIPTION

This is the built-in lock backend for Mojolicious::Plugin::Buffer. The plugin uses it when you set cache_lock_dir.

What problem does it solve?

A web server often runs several copies of your program at the same time. Each copy is called a worker. When a page is not in the cache yet, every worker that gets a request for it would build the same page at the same time. That wastes work. A lock lets exactly one worker build the page. The other workers wait, then read the finished page from the cache.

How it works, in plain words

1. Each cache key gets its own small, empty file in "dir". The file name is the MD5 digest of the key, so any key makes a safe file name.
2. To take the lock, the backend opens that file and asks the operating system for an exclusive lock on it (flock). It does not wait: if another open file handle already holds the lock, the answer is "busy" at once.
3. The lock lasts as long as the file stays open. The open file lives inside the guard that "acquire" returns.
4. "release" closes the file, which frees the lock. If the process crashes or is killed, the operating system closes the file and frees the lock too. So a dead worker can never block a key for ever.

All workers must use the same directory. This works for workers on one machine. For workers on different machines, use Mojolicious::Plugin::Buffer::Lock::Redis.

Words used in this document

key

A text string that names one cache entry, for example buffer:0123abcd.

guard

The value "acquire" returns when you get the lock. Keep it, and give it back to "release". Do not look inside it.

flock

The operating system's file-locking call. Only one open file handle can hold an exclusive flock on a file at a time.

ATTRIBUTES

Attributes are set with "new". Each one also has an accessor method with the same name, made by Mojo::Base.

dir

The directory that holds the lock files. It is required. "new" creates it (and any missing parent directories) and checks that it is writable.

API SPECIFICATION

Input

# $lock->dir reads the value.  $lock->dir($path) changes it WITHOUT
# creating or checking the directory; see COMMON PITFALLS.
{ dir => { type => 'string', min => 1, optional => 1 } }

Output

# Return::Set schema: the path when reading, the object when writing
{ type => 'string' }

messages

An optional hash of message overrides, keyed by message key. See Mojolicious::Plugin::Buffer::I18N for the keys and the text format. The plugin passes its own messages option here when it builds this backend from cache_lock_dir.

API SPECIFICATION

Input

{ messages => { type => 'hashref', optional => 1 } }

Output

# Return::Set schema: the hash (or undef) when reading, the object
# when writing
{ type => 'hashref', optional => 1 }

METHODS

new

my $lock = Mojolicious::Plugin::Buffer::Lock::File->new(dir => $dir);

Purpose

Create the backend, creating the lock directory if necessary.

Arguments

dir (named, required)

A directory path, as a non-empty string.

messages (named, optional)

A hash of message overrides.

Returns

The new object.

Side effects

Creates dir and any missing parents.

Example

    my $lock = Mojolicious::Plugin::Buffer::Lock::File->new(
	dir => '/var/lib/myapp/buffer-locks',
    );

API SPECIFICATION

Input

    {
	dir      => { type => 'string', min => 1 },
	messages => { type => 'hashref', optional => 1 },
    }

Output

# Return::Set schema
{ type => 'object', isa => 'Mojolicious::Plugin::Buffer::Lock::File' }

MESSAGES

Key                     Message                                         Resolution
----------------------  ----------------------------------------------  ---------------------------------
option_cache_lock_dir   cache_lock_dir must be a directory path         Pass a non-empty string
lock_dir_create_failed  cannot create cache_lock_dir 'DIR': OS ERROR    Fix the parent's permissions, or
                                                                        the path (e.g. a file is in the
                                                                        way)
lock_dir_not_writable   cache_lock_dir 'DIR' is not a writable          Change the owner or mode of DIR
                        directory

All three croak. The OS error is Perl's $! text and follows the process's LC_MESSAGES only inside use locale.

acquire

my ($guard, $error) = $lock->acquire($key);

Purpose

Try, without waiting, to become the only process building $key.

Arguments

$key (positional, required)

The cache key: any defined string, in any language, emoji included. Only its MD5 digest reaches the file system.

Returns

A two-element list, as the lock protocol in "cache_lock" in Mojolicious::Plugin::Buffer requires: ($guard, undef) when the lock was taken, (undef, undef) when another process holds it, and (undef, $error) on failure. The guard is opaque. Call it in list context; see "COMMON PITFALLS".

Side effects

Creates the lock file if it does not exist and keeps it open while the guard lives.

Example

    my ($guard, $error) = $lock->acquire('buffer:0123abcd');
    if ($guard) {
	# ... build the entry ...
	$lock->release($guard);
    } elsif (defined $error) {
	warn $error;
    } else {
	# somebody else is building it
    }

API SPECIFICATION

Input

{ key => { type => 'string', position => 0 } }

Output

    # Return::Set schema (list of two)
    {
	type => 'arrayref',
	schema => [
	    { type => 'hashref', optional => 1 },   # guard
	    { type => 'string',  optional => 1 },   # error
	],
    }

MESSAGES

Key                 Message                        Resolution
------------------  -----------------------------  ---------------------------------
lock_key_undefined  cache key is not defined       Pass a defined key
lock_open_failed    cannot open PATH: OS ERROR     Check the directory still exists
                                                   and is writable
lock_failed         cannot lock PATH: OS ERROR     The filesystem does not support
                                                   flock; use another backend

These are returned, not thrown.

PSEUDOCODE

reject an undefined key
open (creating) DIR/md5(key).lock
on failure return the OS error
try a non-blocking exclusive flock
if it succeeded, return a guard holding the handle
close the handle
if flock said "would block", return (undef, undef)
otherwise return the OS error

release

my ($ok, $error) = $lock->release($guard);

Purpose

Give up a lock taken by "acquire".

Arguments

$guard (positional, required)

A guard returned by "acquire". A false value, or a guard that was already released, is accepted and ignored.

Returns

(1, undef) on success, (0, $error) if unlocking or closing failed. The lock is gone either way, because the handle is closed.

Side effects

Unlocks and closes the lock file handle and removes it from the guard.

Example

my ($ok, $error) = $lock->release($guard);
warn $error unless $ok;

API SPECIFICATION

Input

{ guard => { type => 'hashref', optional => 1, position => 0 } }

Output

    # Return::Set schema (list of two)
    {
	type => 'arrayref',
	schema => [
	    { type => 'boolean' },                  # released
	    { type => 'string', optional => 1 },    # error
	],
    }

MESSAGES

Key            Message                        Resolution
-------------  -----------------------------  -------------------------
unlock_failed  cannot unlock PATH: OS ERROR   Usually harmless: closing
                                              the handle drops the lock

COMMON PITFALLS

  • Always take both return values. "acquire" returns a list of two values. In scalar context Perl gives you only the last one, which is the error slot:

    my $guard = $lock->acquire($key);           # WRONG: $guard is undef
my ($guard, $error) = $lock->acquire($key); # right

    The wrong form is worse than it looks. The real guard is thrown away at once, which closes the file, so the lock you just took is released before the next line runs.

  • Keep the guard while you work. The lock lives inside the guard. If the guard variable goes out of scope, the file is closed and the lock is freed, even though you never called "release".

  • The lock is not re-entrant. If your own process already holds the lock for a key, a second acquire for the same key returns "busy" (undef, undef). This is true even for the same object, because each call opens a new file handle.

  • Forking while you hold a lock shares it. A child process made with fork inherits the open file, so the lock stays held until the parent and the child have closed it. Release locks before you fork.

  • Do not change dir after new. "new" creates and checks the directory. Setting it later with $lock->dir($path) skips both checks, so a typing mistake only shows up as an error from every "acquire".

  • Lock files are never deleted. There is one empty file per key that was ever built. Deleting a file while another process has it open would let two builders run, so only clean the directory while the application is stopped.

  • Network file systems. flock is unreliable on some network file systems, especially older NFS. Use Mojolicious::Plugin::Buffer::Lock::Redis when workers are on different machines.

  • undef values. acquire(undef) does not die; it returns (undef, 'cache key is not defined'). The empty string is a valid key. release(undef) and releasing the same guard twice both return (1, undef) and do nothing. new(dir => undef) croaks. messages => undef means "no overrides".

ENCODING

Keys

Any Perl string is safe, including accented letters, any other non-ASCII text and emoji. The key is encoded as UTF-8 bytes before it is hashed, so the same characters always name the same lock file, however Perl stores them internally. Only the MD5 digest reaches the file system. A subclass that overrides _path must do the same, because md5_sum dies on a character above U+00FF.

dir

A directory name goes to the operating system as bytes. An ASCII path is always safe. For a non-ASCII path, pass a byte string that is already encoded the way your file system expects, usually UTF-8:

    use Encode qw(encode);
    my $lock = Mojolicious::Plugin::Buffer::Lock::File->new(
	dir => encode('UTF-8', "/srv/caf\x{e9}/locks"),
    );
messages

Override texts may hold any characters, including emoji. The messages this module returns are Perl character strings. Print them to a handle with an encoding layer, such as binmode(STDERR, ':encoding(UTF-8)'), or Perl warns "Wide character". Mojo::Log encodes to UTF-8 by itself.

Operating-system errors

The text after OS ERROR in a message is Perl's $!. It is English unless the program runs under use locale.

LIMITATIONS

  • Lock files are never deleted. Removing one while another process has it open would let a third process lock a new file of the same name, and two builders would run. They are empty, so the cost is one inode per key ever built.

  • flock is only advisory and is unreliable on some network filesystems (older NFS in particular). Use the Redis backend there.

  • The protocol returns a two-element list rather than a reference, because that is the documented interface third-party backends implement.

SEE ALSO

Mojolicious::Plugin::Buffer, Mojolicious::Plugin::Buffer::Lock::Redis, Mojolicious::Plugin::Buffer::I18N.

AUTHOR

Nigel Horne, <njh at bandsman.co.uk>

FORMAL SPECIFICATION

This section describes the backend in the Z notation, for readers who want an exact model. You do not need it to use the module. KEY is the set of cache keys, PATH the set of file names and PID the set of processes. held records which process holds the lock for each key.

[KEY, PATH, PID, TEXT, Guard]

file : KEY → PATH
∀ k : KEY • file k = dir ⁀ "/" ⁀ md5(utf8(k)) ⁀ ".lock"

┌─ Locks ─────────────────────────────────────
│ dir : PATH
│ held : KEY ⇸ PID
├─────────────────────────────────────────────
│ dir ≠ ∅
└─────────────────────────────────────────────

┌─ ProcessExit ───────────────────────────────
│ ΔLocks
│ p? : PID
├─────────────────────────────────────────────
│ held′ = held ⩥ {p?}
└─────────────────────────────────────────────

new

┌─ New ───────────────────────────────────────
│ Locks′
│ dir? : PATH ; fs, fs′ : PATH ⇸ Node ; result! : Locks
├─────────────────────────────────────────────
│ dir? ≠ ∅
│ dir? ∈ dom fs′ ∧ isDirectory(fs′ dir?) ∧ writable(fs′ dir?)
│ fs ⊆ fs′
│ dir′ = dir? ∧ held′ = ∅ ∧ result! = θLocks′
└─────────────────────────────────────────────
-- Otherwise: croak, and fs′ may hold the parents created so far

acquire

ok means the lock file could be opened and the file system supports flock. The lock is not re-entrant: when held key? = me the result is "busy", the same as for any other holder.

┌─ Acquire ───────────────────────────────────
│ ΔLocks
│ key? : KEY ; me : PID
│ guard! : Guard ∪ {⊥} ; error! : TEXT ∪ {⊥}
├─────────────────────────────────────────────
│ ok ∧ key? ¬in; dom held ⇒
│     held′ = held ⊕ {key? ↦ me} ∧ guard! ≠ ⊥ ∧ error! = ⊥
│ ok ∧ key? ∈ dom held ⇒
│     held′ = held ∧ guard! = ⊥ ∧ error! = ⊥
│ ¬ok ⇒
│     held′ = held ∧ guard! = ⊥ ∧ error! ≠ ⊥
└─────────────────────────────────────────────

release

┌─ Release ───────────────────────────────────
│ ΔLocks
│ guard? : Guard ∪ {⊥} ; ok! : 𝔹 ; error! : TEXT ∪ {⊥}
├─────────────────────────────────────────────
│ guard? ≠ ⊥ ⇒ held′ = {key(guard?)} ⩤ held
│ guard? = ⊥ ⇒ held′ = held ∧ ok! = true
│ ok! = true ⇔ error! = ⊥
└─────────────────────────────────────────────
-- The lock is gone even when ok! = false, because the handle is closed

STATE DIAGRAM

Each cache key's lock is in one of two states. The backend object itself has no other state: after "new" it is ready, and it stays ready. The numbers on the arrows are explained below the picture.

   +-----------+            (1)              +-----------+
   |   FREE    | --------------------------> |   HELD    |
   |  (key k)  | <-------------------------- |  (key k)  |
   +-----------+       (2), (3) or (4)       +-----------+
      |     ^                                    |     ^
      +-----+ (5), (7)                           +-----+ (6), (8)

(1) acquire(k)  [the file opens and flock is granted]
    / returns (guard, undef); the file stays open inside the guard
(2) release(guard)
    / unlock and close the file; returns (1, undef), or (0, error)
      if unlocking failed - the lock is gone either way
(3) the guard goes out of scope
    / Perl closes the file, which frees the lock
(4) the holding process exits or is killed
    / the operating system closes the file, which frees the lock
(5) acquire(k)  [the file cannot be opened, or flock is not supported]
    / returns (undef, error); nothing changes
(6) acquire(k)  [flock would block], from ANY process, the holder too
    / returns (undef, undef); nothing changes
(7) release(undef), or release of a guard already released
    / returns (1, undef); nothing changes
(8) fork()
    / the child shares the open file, so the key stays HELD until
      both processes have closed it

The plugin's own use of these states is drawn in "STATE DIAGRAM" in Mojolicious::Plugin::Buffer.

LICENSE AND COPYRIGHT

Copyright 2026 Nigel Horne.

This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version.