/ 
Mojolicious-Plugin-Buffer-0.001.0
/ Mojolicious::Plugin::Buffer::Lock::Redis

NAME

Mojolicious::Plugin::Buffer::Lock::Redis - Redis-backed lock backend

VERSION

Version 0.001.0

SYNOPSIS

Workers on several machines, sharing one Redis server and one cache:

    use Mojolicious::Lite;
    use CHI;
    use Redis;
    use Mojolicious::Plugin::Buffer::Lock::Redis;

    plugin 'Buffer' => {
	cache => CHI->new(driver => 'Memcached', servers => ['cache.internal:11211']),
	cache_lock => Mojolicious::Plugin::Buffer::Lock::Redis->new(
	    # A code reference: each worker makes its own connection
	    redis => sub {
		Redis->new(
		    server        => 'redis.internal:6379',
		    cnx_timeout   => 1,
		    read_timeout  => 1,
		    write_timeout => 1,
		);
	    },
	    namespace => 'myapp:buffer:',
	    ttl       => 60,
	),
    };

The same with Redis::Fast, which has the same interface:

    use Redis::Fast;

    my $lock = Mojolicious::Plugin::Buffer::Lock::Redis->new(
	redis => sub { Redis::Fast->new(server => 'redis.internal:6379') },
    );

Use it on its own, for example so that only one machine runs a job:

    my ($guard, $error) = $lock->acquire('nightly-report');
    if ($guard) {
	build_report();
	$lock->release($guard);
    } elsif (defined $error) {
	warn "Redis problem: $error\n";
    }    # else another machine is already doing it

Add a Redis Cluster hash tag by overriding the protected _key method, so every lock lands in one slot:

    package MyApp::Lock;
    use Mojo::Base 'Mojolicious::Plugin::Buffer::Lock::Redis';
    use Mojo::Util qw(encode md5_sum);

    sub _key {
	my ($self, $key) = @_;
	return '{' . $self->namespace . '}' . md5_sum(encode('UTF-8', $key));
    }

DESCRIPTION

This is a lock backend for cache_lock. It lets workers on different machines agree that only one of them builds a missing cache entry. The cache itself must also be shared between those workers.

How it works, in plain words

1. Each cache key is turned into a Redis key: "namespace" followed by the MD5 digest of the cache key.
2. To take the lock, the backend writes a random token into that Redis key, but only if the key does not exist yet (Redis SET ... NX PX). Redis does this in one step, so two workers can never both succeed.
3. The Redis key expires by itself after "ttl" seconds. So if a worker crashes while it holds a lock, the lock disappears later on its own.
4. To release the lock, a small Lua script deletes the Redis key, but only if it still holds our token. If our lock already expired and another worker took it, we do not delete theirs.

The Redis client is called synchronously: the worker waits for Redis to answer before the request goes on. So set short connect, read and write timeouts on the client, and do not use a non-blocking client. Redis 2.6.12 or newer is needed for SET ... NX PX.

Words used in this document

token

A random 32-character string made for each successful "acquire". It proves which worker owns a lock.

TTL

"Time to live": how long Redis keeps the lock before it deletes it by itself.

guard

The value "acquire" returns when you get the lock. It holds the Redis key and the token. Give it back to "release".

ATTRIBUTES

Attributes are set with "new", which checks them. Each one also has an accessor method with the same name, made by Mojo::Base. Changing a value through the accessor after "new" skips those checks.

redis

Required. Either a Redis client object with the same eval($script, $numkeys, @args) method as Redis and Redis::Fast, or a code reference that returns such an object.

Prefer the code reference. It is called the first time this backend is used in each process, so with a pre-forking server such as hypnotoad every worker gets its own connection. A client object made in startup is copied into every worker by fork, and the workers would then share one network connection, which corrupts the replies.

API SPECIFICATION

Input

# An object is accepted when it can('eval') or has an AUTOLOAD, as the
# CPAN Redis client dispatches its commands through AUTOLOAD
{ redis => { type => ['coderef', 'object'], optional => 1 } }

Output

# Return::Set schema: the value when reading, the object when writing
{ type => ['coderef', 'object'] }

namespace

The prefix for every Redis key this backend makes. Defaults to 'buffer:lock:'. Use a different namespace for each application that shares a Redis database.

API SPECIFICATION

Input

{ namespace => { type => 'string', min => 1, optional => 1 } }

Output

# Return::Set schema: the value when reading, the object when writing
{ type => 'string', min => 1 }

ttl

How long a lock lives, in seconds. Defaults to 60. Fractions such as 0.5 are allowed; the value is sent to Redis in whole milliseconds, and never less than 1 millisecond. Choose a value longer than cache_lock_timeout and longer than your slowest cached action.

API SPECIFICATION

Input

{ ttl => { type => 'number', min => 0, optional => 1,
           matches => qr/^\d+(?:\.\d+)?$/ } }   # and > 0

Output

# Return::Set schema: the value when reading, the object when writing
{ type => 'number' }

messages

An optional hash of message overrides. See Mojolicious::Plugin::Buffer::I18N for the keys and the text format.

API SPECIFICATION

Input

{ messages => { type => 'hashref', optional => 1 } }

Output

# Return::Set schema: the hash (or undef) when reading, the object
# when writing
{ type => 'hashref', optional => 1 }

METHODS

new

my $lock = Mojolicious::Plugin::Buffer::Lock::Redis->new(redis => sub { Redis->new });

Purpose

Create the backend and check its configuration. No connection is made.

Arguments

redis (named, required)

A client object or a code reference returning one.

namespace (named, optional)

A non-empty string, default 'buffer:lock:'.

ttl (named, optional)

A positive number of seconds, default 60.

messages (named, optional)

A hash of message overrides.

Returns

The new object.

Side effects

None.

Example

    my $lock = Mojolicious::Plugin::Buffer::Lock::Redis->new(
	redis     => sub { Redis->new(server => 'redis.internal:6379') },
	namespace => 'shop:buffer:',
	ttl       => 30,
    );

API SPECIFICATION

Input

    {
	redis     => { type => ['coderef', 'object'] },   # object: eval() or AUTOLOAD
	namespace => { type => 'string', min => 1, optional => 1 },
	ttl       => { type => 'number', min => 0, optional => 1,
	               matches => qr/^\d+(?:\.\d+)?$/ },   # and > 0
	messages  => { type => 'hashref', optional => 1 },
    }

Output

# Return::Set schema
{ type => 'object', isa => 'Mojolicious::Plugin::Buffer::Lock::Redis' }

MESSAGES

Key                      Message                                   Resolution
-----------------------  ----------------------------------------  -------------------------------
redis_client_invalid     cache_lock redis must be an object with   Pass a Redis/Redis::Fast object
                         an eval() method or a code reference      or, better, a factory sub
                         returning one
redis_namespace_invalid  cache_lock namespace must be a non-empty  Pass e.g. 'myapp:buffer:'
                         string
redis_ttl_invalid        cache_lock ttl must be a positive number  Pass e.g. 60
                         of seconds

All three croak.

acquire

my ($guard, $error) = $lock->acquire($key);

Purpose

Try, without waiting, to become the only worker building $key.

Arguments

$key (positional, required)

The cache key: any defined string, in any language, emoji included. Only its MD5 digest is sent to Redis.

Returns

($guard, undef) when the lock was acquired, (undef, undef) when another worker holds it, and (undef, $error) when Redis is unavailable. The plugin logs an error and continues without a cross-worker lock in the last case. Call it in list context; see "COMMON PITFALLS".

Side effects

Connects to Redis on first use (through the factory) and sets a key that expires after "ttl".

Example

my ($guard, $error) = $lock->acquire('buffer:0123abcd');
$lock->release($guard) if $guard;

API SPECIFICATION

Input

{ key => { type => 'string', position => 0 } }

Output

    # Return::Set schema (list of two)
    {
	type => 'arrayref',
	schema => [
	    { type => 'hashref', optional => 1 },   # guard: {key, token}
	    { type => 'string',  optional => 1 },   # error
	],
    }

MESSAGES

Key                 Message                                   Resolution
------------------  ----------------------------------------  ---------------------------
lock_key_undefined  cache key is not defined                  Pass a defined key
redis_lock_failed   Redis lock failed: ERROR                  Check the server and client
redis_no_client     Redis client factory returned no client   Make the factory return one
redis_no_result     Redis lock command returned no result     Check the client library

These are returned, not thrown.

PSEUDOCODE

reject an undefined key
derive the Redis key and a fresh token
convert the TTL to milliseconds, at least 1
run SET NX PX through the Lua script
if the call failed, return the error
if it returned nothing, return "no result"
if it returned 1, return the guard
otherwise return (undef, undef): someone else holds it

release

my ($ok, $error) = $lock->release($guard);

Purpose

Delete the lock, but only if it still holds this guard's token.

Arguments

$guard (positional, required)

A guard returned by "acquire".

Returns

(1, undef) on success, including when the lock had already expired, and (0, $error) on failure. A failed release is not fatal because the lock expires after "ttl".

Side effects

Deletes the Redis key if it still holds the token.

Example

my ($ok, $error) = $lock->release($guard);
warn $error unless $ok;

API SPECIFICATION

Input

{ guard => { type => 'hashref', position => 0 } }   # {key, token}

Output

    # Return::Set schema (list of two)
    {
	type => 'arrayref',
	schema => [
	    { type => 'boolean' },
	    { type => 'string', optional => 1 },
	],
    }

MESSAGES

Key                  Message                       Resolution
-------------------  ----------------------------  ------------------------------
redis_invalid_guard  invalid lock guard            Pass the guard acquire returned
redis_unlock_failed  Redis unlock failed: ERROR    None needed; the TTL expires it

COMMON PITFALLS

  • Always take both return values. "acquire" returns a list of two values. In scalar context Perl gives you only the last one, the error slot:

    my $guard = $lock->acquire($key);           # WRONG: $guard is undef
my ($guard, $error) = $lock->acquire($key); # right

    Here the mistake is costly. The lock was taken in Redis, but you have lost the token, so you cannot release it. Every worker then sees "busy" for that key until the "ttl" runs out.

  • The TTL is too short. If the builder is still working when the lock expires, a second worker can take the lock and build the same entry. Make "ttl" longer than the slowest action that uses the cache.

  • A client object made before the fork. Pass a code reference as "redis", not an object made in startup. Also, do not call "acquire" in startup: the first call stores the client, and fork would then copy it into every worker.

  • A slow Redis server slows every cached request. The call waits for Redis. Set cnx_timeout, read_timeout and write_timeout on the client.

  • Changing attributes after new. $lock->ttl(0) is not checked; Redis is then sent the smallest TTL, 1 millisecond, and the lock protects nothing. Set attributes with "new" only.

  • The lock is not re-entrant. A second acquire for a key this worker already holds returns "busy" (undef, undef).

  • undef values. acquire(undef) returns (undef, 'cache key is not defined'). release(undef) returns (0, 'invalid lock guard'). A factory that returns undef gives (undef, 'Redis lock failed: Redis client factory returned no client'), and is asked again on the next call. new(ttl => undef) and new(namespace => undef) croak.

ENCODING

Keys

Any Perl string is safe, including accented letters, other non-ASCII text and emoji. The key is encoded as UTF-8 bytes and hashed, so only an ASCII digest is sent to Redis. A subclass that overrides _key must encode before calling md5_sum, which dies on a character above U+00FF.

namespace

Sent to Redis as part of the key. Keep it ASCII. Redis keys are bytes, so a non-ASCII namespace must be encoded to bytes by you first.

messages

Override texts may hold any characters, including emoji. The messages this module returns are Perl character strings; print them through an encoding layer such as :encoding(UTF-8), or Perl warns "Wide character". Mojo::Log encodes to UTF-8 by itself. The text after ERROR comes from the Redis client and is passed on unchanged.

LIMITATIONS

  • The client is called synchronously inside a request. A slow or partitioned Redis server stalls that worker for as long as the client's own timeouts allow; there is no circuit breaker.

  • This is a single-instance lock, not Redlock. With Redis replication, a fail-over can lose a lock that was just set, and two builders may run. For a cache that is acceptable: the cost is one duplicated build, not corrupted data.

  • A lock that outlives "ttl" is silently lost, and a second builder can start. Choose a TTL longer than the slowest cached action.

SEE ALSO

Mojolicious::Plugin::Buffer, Mojolicious::Plugin::Buffer::Lock::File, Mojolicious::Plugin::Buffer::I18N.

AUTHOR

Nigel Horne, <njh at bandsman.co.uk>

FORMAL SPECIFICATION

This section describes the backend in the Z notation, for readers who want an exact model. You do not need it to use the module. store is the part of the Redis database that this backend uses, expiry says when each of its keys disappears, and now is the current time.

[KEY, TEXT, TOKEN, Client]

rkey : KEY → TEXT
∀ k : KEY • rkey k = namespace ⁀ md5(utf8(k))

Guard == TEXT × TOKEN

┌─ Redis ─────────────────────────────────────
│ namespace : seq₁ CHAR
│ ttl : ℝ
│ client : Client ∪ {⊥}
│ store : TEXT ⇸ TOKEN
│ expiry : TEXT ⇸ ℝ
├─────────────────────────────────────────────
│ ttl > 0
│ dom expiry = dom store
└─────────────────────────────────────────────

┌─ Expire ────────────────────────────────────
│ ΔRedis
├─────────────────────────────────────────────
│ store′ = { r : dom store | expiry r ≤ now } ⩤ store
│ expiry′ = { r : dom store | expiry r ≤ now } ⩤ expiry
└─────────────────────────────────────────────

new

┌─ NewRedisLock ──────────────────────────────
│ Redis′
│ redis? : Client ∪ (seq ARG → Client)
│ namespace? : seq₁ CHAR ; ttl? : ℝ
│ result! : Redis
├─────────────────────────────────────────────
│ ttl? > 0
│ namespace′ = namespace? ∧ ttl′ = ttl?
│ client′ = ⊥ ⇐ redis? is a code reference
│ result! = θRedis′
└─────────────────────────────────────────────
-- No connection is made.  Any violated predicate: croak.

acquire

reachable means the client could be made and answered. token is fresh: it differs from every token issued before.

┌─ AcquireRedis ──────────────────────────────
│ ΔRedis
│ key? : KEY ; token : TOKEN
│ guard! : Guard ∪ {⊥} ; error! : TEXT ∪ {⊥}
├─────────────────────────────────────────────
│ let r == rkey key? •
│   reachable ∧ r ¬in; dom store ⇒
│       store′ = store ⊕ {r ↦ token}
│       ∧ expiry′ = expiry ⊕ {r ↦ now + max(ttl, 0.001)}
│       ∧ guard! = (r, token) ∧ error! = ⊥
│   reachable ∧ r ∈ dom store ⇒
│       store′ = store ∧ guard! = ⊥ ∧ error! = ⊥
│   ¬reachable ⇒
│       store′ = store ∧ guard! = ⊥ ∧ error! ≠ ⊥
└─────────────────────────────────────────────

release

┌─ ReleaseRedis ──────────────────────────────
│ ΔRedis
│ guard? : Guard ∪ {⊥} ; ok! : 𝔹 ; error! : TEXT ∪ {⊥}
├─────────────────────────────────────────────
│ guard? = ⊥ ⇒ store′ = store ∧ ok! = false ∧ error! ≠ ⊥
│ guard? = (r, t) ∧ store r = t ⇒
│     store′ = {r} ⩤ store ∧ ok! = true
│ guard? = (r, t) ∧ (r ¬in; dom store ∨ store r ≠ t) ⇒
│     store′ = store ∧ ok! = true
│ ¬reachable ⇒ store′ = store ∧ ok! = false ∧ error! ≠ ⊥
└─────────────────────────────────────────────

STATE DIAGRAM

Each cache key's lock, as Redis sees it. The numbers on the arrows are explained below the picture.

   +-----------+            (1)              +----------------+
   |   FREE    | --------------------------> |  HELD (token)  |
   |  (key k)  | <-------------------------- |                |
   +-----------+          (2) or (3)         +----------------+
      |     ^                                    |     ^
      +-----+ (4)                                +-----+ (5), (6)

(1) acquire(k)  [Redis answers and the key is absent]
    / SET key token NX PX ttl; returns ({key, token}, undef)
(2) release(guard)  [the stored token matches the guard]
    / the Lua script deletes the key; returns (1, undef)
(3) the TTL runs out
    / Redis deletes the key by itself.  If the holder is still
      building, a second worker can now start building as well.
(4) acquire(k)  [Redis is unreachable, or the factory gave no client]
    / returns (undef, error); nothing changes
(5) acquire(k)  [the key is present], from any worker, the holder too
    / returns (undef, undef); nothing changes
(6) release(old guard)  [the token differs: this lock expired and
    another worker took the key]
    / nothing is deleted; returns (1, undef)

The backend object itself has one more state, for its client:

+----------------+   first acquire() or release() in this process
| NO CLIENT YET  |   [redis is a code reference]          +---------+
| (after new)    | -------------------------------------> | CLIENT  |
+----------------+   / call the factory once; keep result | READY   |
       |   ^                                              +---------+
       +---+  the factory returned undef or died
              / the call returns (undef, error); try again next time

When "redis" is an object rather than a code reference, the backend starts in CLIENT READY.

The plugin's own use of these states is drawn in "STATE DIAGRAM" in Mojolicious::Plugin::Buffer.

LICENSE AND COPYRIGHT

Copyright 2026 Nigel Horne.

This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version.