NAME
Data::Buffer::Shared - Type-specialized shared-memory buffers for multiprocess access
SYNOPSIS
use Data::Buffer::Shared::I64;
# Create or open a shared buffer (file-backed mmap)
my $buf = Data::Buffer::Shared::I64->new('/tmp/mybuf.shm', 1024);
# Keyword API (fastest)
buf_i64_set $buf, 0, 42;
my $val = buf_i64_get $buf, 0;
# Method API
$buf->set(0, 42);
my $v = $buf->get(0);
# Lock-free atomic operations (integer types)
buf_i64_incr $buf, 0;
buf_i64_add $buf, 0, 10;
buf_i64_cas $buf, 0, 52, 100;
# Multiprocess
if (fork() == 0) {
my $child = Data::Buffer::Shared::I64->new('/tmp/mybuf.shm', 1024);
buf_i64_incr $child, 0; # atomic, visible to parent
exit;
}
wait;DESCRIPTION
Data::Buffer::Shared provides type-specialized fixed-capacity buffers stored in file-backed shared memory (mmap(MAP_SHARED)), enabling efficient multiprocess data sharing on Linux.
Linux-only. Requires 64-bit Perl.
Features
File-backed mmap for cross-process sharing
Lock-free atomic get/set for numeric types (single-element)
Lock-free atomic counters: incr/decr/add/cas (integer types)
Seqlock-guarded bulk operations (slice, fill)
Write-preferring futex read-write lock with dead-process recovery
eventfd-based cross-process notification (optional)
Keyword API via XS::Parse::Keyword
Presized: fixed capacity, no growing
Variants
Constructors
my $buf = Data::Buffer::Shared::I64->new($path, $capacity); # file-backed
my $buf = Data::Buffer::Shared::I64->new_anon($capacity); # anonymous
my $buf = Data::Buffer::Shared::I64->new_memfd($name, $capacity); # memfd
my $buf = Data::Buffer::Shared::I64->new_from_fd($fd); # reopen memfdThe Str variant takes an additional $max_len argument giving the per-element fixed byte width:
my $buf = Data::Buffer::Shared::Str->new($path, $capacity, $max_len);
my $buf = Data::Buffer::Shared::Str->new_anon($capacity, $max_len);
my $buf = Data::Buffer::Shared::Str->new_memfd($name, $capacity, $max_len);
my $buf = Data::Buffer::Shared::Str->new_from_fd($fd, $max_len);new_from_fd duplicates the caller's fd internally; the caller keeps ownership of the passed fd. The Str variant requires the same $max_len the original was created with: it is recorded in the header as the element size and checked on attach, so passing a different $max_len dies with an "elem_size mismatch" error.
Lifecycle
my $p = $buf->path; # backing file path, or undef for anon/memfd
my $fd = $buf->fd; # memfd fd, or undef for anon/file-backed
my $fd = $buf->memfd; # alias of fd()
$buf->clear; # zero all elements (write-locked)
$buf->sync; # msync(MS_SYNC) mmap to backing store
$buf->unlink; # remove backing file (dies for anonymous buffers)
my $h = $buf->stats; # diagnostic hashrefunlink also works as a class method: Data::Buffer::Shared::I64->unlink($path).
memfd is an alias of fd (present on every variant): both return the backing file descriptor for a memfd-backed buffer (created with new_memfd), or undef for anonymous and file-backed buffers.
API
Replace xx with variant prefix: i8, u8, i16, u16, i32, u32, i64, u64, f32, f64, str.
buf_xx_set $buf, $idx, $value; # set element (lock-free atomic for numeric)
my $v = buf_xx_get $buf, $idx; # get element (lock-free atomic for numeric)
my @v = buf_xx_slice $buf, $from, $count; # bulk read (seqlock)
buf_xx_fill $buf, $value; # fill all elements (write-locked)
buf_xx_clear $buf; # zero all elements (write-locked)set_slice is a method only (its variadic argument list has no keyword form); it writes a run of elements starting at $from and returns true on success:
$buf->set_slice($from, @values); # bulk write (write-locked)Integer variants also have:
my $n = buf_xx_incr $buf, $idx; # atomic increment, returns new value
my $n = buf_xx_decr $buf, $idx; # atomic decrement
my $n = buf_xx_add $buf, $idx, $delta; # atomic add
my $ok = buf_xx_cas $buf, $idx, $old, $new; # compare-and-swap
my $p = buf_xx_cmpxchg $buf, $idx, $old, $new; # CAS, returns prior value
my $n = buf_xx_atomic_and $buf, $idx, $mask; # atomic AND (integer variants)
my $n = buf_xx_atomic_or $buf, $idx, $mask; # atomic OR
my $n = buf_xx_atomic_xor $buf, $idx, $mask; # atomic XORRaw / bulk:
my $raw = buf_xx_get_raw $buf, $from, $count; # bulk bytes, seqlock-guarded
buf_xx_set_raw $buf, $from, $raw; # bulk bytes, write-locked
$buf->add_slice($from, @deltas); # batch atomic add (integer variants; flat list)
my $ptr = buf_xx_ptr $buf; # raw pointer to data, for FFI use
my $ptr = buf_xx_ptr_at $buf, $idx; # pointer to element at indexZero-copy:
my $sv = $buf->as_scalar; # mmap-aliased read-only scalar refThe returned scalar aliases the mapped bytes directly (no copy) and holds a reference to the buffer so the mapping stays alive while it is in use.
Cross-process notification (all variants):
my $efd = $buf->create_eventfd; # create + attach an eventfd, returns the fd
$buf->attach_eventfd($fd); # attach an already-open eventfd
my $efd = $buf->eventfd; # current eventfd, or undef if none
$buf->notify; # signal (eventfd write)
my $n = $buf->wait_notify; # drain the counter, non-blocking (undef if 0)These are a thin wrapper over an eventfd(2) descriptor stored in the handle, letting one process signal another that the buffer changed. The eventfd is created non-blocking, so wait_notify does not block: it reads and clears the counter, returning the accumulated notify count, or undef when the counter is zero (nothing pending) or no eventfd is attached. Nothing else in the API depends on them; watch the descriptor for readability in an event loop rather than expecting a blocking wakeup.
Diagnostics:
my $c = buf_xx_capacity $buf;
my $s = buf_xx_mmap_size $buf;
my $e = buf_xx_elem_size $buf;
my $h = $buf->stats; # hashref: capacity/elem_size/mmap_size/variant_id/recoveriesPersistence:
$buf->sync; # msync(MS_SYNC) mmap to backing storeExplicit locking (for batch operations):
buf_xx_lock_wr $buf; # write lock + seqlock begin
buf_xx_unlock_wr $buf; # seqlock end + write unlock
buf_xx_lock_rd $buf; # read lock
buf_xx_unlock_rd $buf; # read unlockThe explicit locks are non-recursive and non-upgradable: calling lock_wr while holding lock_rd on the same handle, or calling lock_wr twice without an intervening unlock_wr, self-deadlocks. Dropping the last reference to a handle while holding one of its locks leaks that handle's reader slot (and any held lock contribution) until the process exits.
CONCURRENCY AND CRASH SAFETY
Single-element numeric get/set and the atomic counter operations (incr/decr/add/cas/cmpxchg/atomic_and/atomic_or/atomic_xor) are lock-free and safe to call concurrently from any number of processes. Bulk reads (slice, get_raw) are guarded by a seqlock and retry if a writer intervenes. Bulk writes (set_slice, fill, clear, set_raw) and the explicit lock_wr/unlock_wr region take a write lock.
The write/read lock is a write-preferring futex read-write lock with dead-process recovery: if a process crashes while holding the lock, another process detects the dead holder and reclaims its contribution so the mapping does not deadlock. See "Reader-slot exhaustion" for the one narrow case this recovery cannot cover.
SEE ALSO
Data::HashMap::Shared - concurrent hash table
Data::Queue::Shared - FIFO queue
Data::PubSub::Shared - publish-subscribe ring
Data::ReqRep::Shared - request-reply
Data::Sync::Shared - synchronization primitives
Data::Pool::Shared - fixed-size object pool
Data::Stack::Shared - LIFO stack
Data::Deque::Shared - double-ended queue
Data::Log::Shared - append-only log (WAL)
Data::Heap::Shared - priority queue
Data::Graph::Shared - directed weighted graph
Data::RingBuffer::Shared - fixed-size overwriting ring buffer
Data::BitSet::Shared - shared bitset (lock-free per-bit ops)
SECURITY
Backing files are created with mode 0600 (owner-only) by default, so only the creating user can open and attach them. To share a backing file across users, pass an explicit octal file mode such as 0660 as the last argument to new; the mode is applied only when the file is created or initialized (an existing non-empty file keeps its own permissions; a pre-existing empty file owned by your effective uid is initialized as a fresh backing file and the mode is applied to it via fchmod). The file is opened with O_NOFOLLOW, so a symlink planted at the path is refused, and created with O_EXCL; the on-disk header is validated when the file is attached. Any process you grant write access to a shared mapping is trusted not to corrupt its contents while other processes are using it.
Reader-slot exhaustion
Reader-slot exhaustion (slotless readers): dead-process recovery attributes a crashed lock holder's contribution through its reader-slot. The slot table holds 1024 entries (one per concurrent reader process). If more than that many reader processes share one mapping at once, a reader that cannot claim a slot proceeds "slotless" -- it still takes the read lock but leaves no per-process record. If such a slotless reader is then killed while holding the read lock, its share of the lock cannot be attributed to a dead process, so writer recovery cannot reclaim it and writers may block until the mapping is recreated. Reaching this needs more than 1024 concurrent reader processes on one mapping plus a crash in the brief read-lock window; the dead-process slot reclaim keeps the table from filling with stale entries, so in practice it is very unlikely.
AUTHOR
vividsnow
LICENSE
This is free software; you can redistribute it and/or modify it under the same terms as Perl itself.