/ 
Lugh-0.06
/ Lugh::Context

NAME

Lugh::Context - Memory Context for Tensor Allocation

VERSION

Version 0.01

SYNOPSIS

use Lugh;

# Create a context with 16 MB of memory
my $ctx = Lugh::Context->new(mem_size => 16 * 1024 * 1024);

# Create tensors in this context
my $a = Lugh::Tensor->new_f32($ctx, 1024);
my $b = Lugh::Tensor->new_f32($ctx, 1024);

# Tensors are automatically freed when context goes out of scope

DESCRIPTION

Lugh::Context provides memory management for tensor allocation. It wraps ggml's context system, which uses a memory arena for efficient allocation of many tensors without individual malloc/free overhead.

Memory Arena

When you create a Context, a contiguous block of memory is allocated. All tensors created in that context share this memory arena:

Context (16 MB arena)
┌─────────────────────────────────────────────────┐
│ Tensor A (4KB) │ Tensor B (16KB) │ Tensor C ... │
└─────────────────────────────────────────────────┘

Benefits:

  • Fast allocation - No system malloc per tensor

  • Cache friendly - Tensors are contiguous in memory

  • Simple cleanup - Free entire context at once

Context Types

There are several common patterns for context usage:

  • Weight context - Long-lived, holds model weights (read from GGUF)

  • Compute context - Short-lived, holds computation graph tensors

  • Scratch context - Temporary, holds intermediate values

For high-level inference, you typically don't need to manage contexts directly - Lugh::Model and Lugh::Inference handle this internally.

CONSTRUCTOR

new

my $ctx = Lugh::Context->new(
    mem_size => $bytes
);

Creates a new memory context.

Parameters:

  • mem_size (required) - Size of memory arena in bytes

Returns: A Lugh::Context object.

Throws: Dies if memory allocation fails or if maximum contexts exceeded.

Memory Sizing:

The memory size should account for:

  • Tensor metadata (about 256 bytes per tensor)

  • Tensor data (depends on type and dimensions)

  • Alignment padding

Rule of thumb: allocate 2-3× the expected tensor data size.

Example:

# Small context for a few tensors
my $ctx = Lugh::Context->new(mem_size => 1024 * 1024);  # 1 MB

# Large context for model weights
my $ctx = Lugh::Context->new(mem_size => 1024 * 1024 * 1024);  # 1 GB

METHODS

mem_size

my $bytes = $ctx->mem_size;

Returns the total size of the memory arena.

mem_used

my $bytes = $ctx->mem_used;

Returns the amount of memory currently used by tensors.

Example:

my $ctx = Lugh::Context->new(mem_size => 1024 * 1024);
print "Used: ", $ctx->mem_used, " bytes\n";  # Initially small

my $tensor = Lugh::Tensor->new_f32($ctx, 1000);
print "Used: ", $ctx->mem_used, " bytes\n";  # +4000 bytes for data

MEMORY MANAGEMENT

Automatic Cleanup

When a Context object goes out of scope, its destructor automatically frees the entire memory arena:

{
    my $ctx = Lugh::Context->new(mem_size => 1024 * 1024);
    my $tensor = Lugh::Tensor->new_f32($ctx, 1000);
    # ... use tensor ...
}
# $ctx is freed here, including all tensors

Warning: Never use a tensor after its context has been freed!

Maximum Contexts

There is a compile-time limit on the number of simultaneous contexts (default: 4096). This is rarely an issue in practice.

No Individual Tensor Free

You cannot free individual tensors within a context. This is by design - the arena allocator trades flexibility for speed. If you need to free tensors individually, use separate contexts.

TENSOR CREATION

Tensors are created using the static methods on Lugh::Tensor, passing the context as the first argument:

my $f32_1d = Lugh::Tensor->new_f32($ctx, 100);
my $f32_2d = Lugh::Tensor->new_f32($ctx, 100, 200);
my $f32_3d = Lugh::Tensor->new_f32($ctx, 100, 200, 300);
my $f32_4d = Lugh::Tensor->new_f32($ctx, 100, 200, 300, 400);

Memory usage for F32 tensors:

Dimensions      Elements     Memory
(100)          100          400 bytes
(100, 200)     20,000       80 KB
(100, 200, 300) 6,000,000   24 MB

LOW-LEVEL OPERATIONS

For advanced users, contexts can be used with the low-level Lugh::Ops and Lugh::Graph modules:

my $ctx = Lugh::Context->new(mem_size => 10 * 1024 * 1024);

my $a = Lugh::Tensor->new_f32($ctx, 1000);
my $b = Lugh::Tensor->new_f32($ctx, 1000);

$a->set_f32(1.0, 2.0, 3.0, ...);
$b->set_f32(4.0, 5.0, 6.0, ...);

my $c = Lugh::Ops::add($ctx, $a, $b);

my $graph = Lugh::Graph->new($ctx);
$graph->build_forward($c);
$graph->compute($ctx, 4);  # 4 threads

my @result = $c->get_f32();

THREAD SAFETY

Lugh::Context objects are NOT thread-safe. The internal registry uses mutex locks, but individual contexts should not be shared across Perl threads. Each thread must create its own contexts.

For multi-threaded computation, use $graph->compute($ctx, $n_threads) which parallelizes within a single Perl thread using pthreads.

IMPLEMENTATION NOTES

Registry Pattern

Contexts are tracked in a global registry using integer IDs. This allows safe Perl object destruction and thread cloning:

Perl SV → Magic → ID → Registry → LughContext → ggml_context

ggml Context

Internally, this wraps struct ggml_context* from the ggml library. The ggml context provides:

  • Memory arena management

  • Tensor allocation

  • Computation graph building

  • Thread-safe graph execution

SEE ALSO

Lugh, Lugh::Tensor, Lugh::Graph, Lugh::Ops

https://github.com/ggerganov/ggml - ggml library

AUTHOR

lnation <email@lnation.org>

LICENSE

This library is free software; you can redistribute it and/or modify it under the same terms as Perl itself.

1 POD Error

The following errors were encountered while parsing the POD:

Around line 43:

Non-ASCII character seen before =encoding in '┌─────────────────────────────────────────────────┐'. Assuming UTF-8