/ 
Acme-Parataxis-v0.0.9
/ README.md

NAME

Acme::Parataxis - A terrible idea, honestly...

SYNOPSIS

use v5.40;
use Acme::Parataxis;
$|++;

# Basic usage with the integrated scheduler
Acme::Parataxis::run(
    sub {
        say 'Main task started';

        # Spawn background workers
        my $f1 = Acme::Parataxis->spawn(
            sub {
                say '  Task 1: Sleeping in a native thread pool...';
                Acme::Parataxis->await_sleep(1000);
                say '  Task 1: Ah! What a nice nap...';
                return 42;
            }
        );
        my $f2 = Acme::Parataxis->spawn(
            sub {
                say '  Task 2: Performing I/O...';

                # await_read/write for non-blocking socket handling
                return 'I/O Done';
            }
        );

        # Block current fiber until results are ready (without blocking the thread)
        say 'Result 1: ' . $f1->await();
        say 'Result 2: ' . $f2->await();
    }
);

DESCRIPTION

I had this idea while writing cookbook examples for Affix. I wondered if I could implement a hybrid concurrency model for Perl from within FFI. This is that unpublished article made into a module. It's fragile. It's dangerous. It's my attempt at combining cooperative multitasking (green threads or fibers or whatever it's called in the last edit of Wikipedia) with a preemptive native thread pool. It's Acme::Parataxis.

This is in the Acme namespace for a reason. Don't use this. Forget you even saw it. Just reading this has probably made your projects more prone to breaking. Reading the package name out loud might cause brain damage to yourself and those within earshot.

With that out of the way, Acme::Parataxis implements a hybrid concurrency model for Perl. It combines:

WARNING: If the earlier warnings weren't enough, here goes another one... this module is experimental and resides in the Acme:: namespace for a reason. It manually manipulates Perl's internal stacks and C context. It is very dangerous. It's irresponsible, honestly, that I'm even putting this terrible idea into the world. Close the browser and clear your history before this does further harm!

CORE CONCEPTS

The Scheduler

The simplest way to use this module is via Acme::Parataxis::run. This sets up an event loop that manages all fibers. Within this loop, you use spawn to start new tasks.

Fibers vs. Threads

In Parataxis, your Perl code always runs on a single OS thread. However, when you call an await_* function, the current fiber is suspended, and the work is performed on a different OS thread. Once the work is done, your fiber is resumed back on the main thread.

SCHEDULER FUNCTIONS

run( $code )

Starts the event loop and executes $code as the first fiber. The loop runs until all spawned fibers have completed.

Acme::Parataxis::run(sub {
    # Your code here...
});

spawn( $code )

Creates a new fiber and adds it to the scheduler's queue. Returns a Future.

my $future = Acme::Parataxis->spawn(sub {
    return "Hello from fiber #" . Acme::Parataxis->current_fid;
});

yield( @args )

Pauses the current fiber and gives other fibers a chance to run. If @args are provided, they are passed to the context that resumes this fiber. Arguments can be of any Perl data type.

stop( )

Tells the scheduler to exit the loop after the current iteration.

BLOCKING & I/O FUNCTIONS

These functions suspend the current fiber and offload work to the thread pool.

await_sleep( $ms )

Suspends the fiber for $ms milliseconds. Other fibers continue to run during this time.

await_read( $fh, $timeout = 5000 )

Wait for a filehandle (usually a socket) to become ready for reading.

my $status = Acme::Parataxis->await_read($socket);
if ($status > 0) {
    my $data = <$socket>;
}

await_write( $fh, $timeout = 5000 )

Wait for a filehandle to become ready for writing.

await_core_id( )

A utility function that returns the ID of the CPU core the background worker ran on.

MANUAL FIBER MANAGEMENT

For advanced users who want to manage context switching themselves without the integrated scheduler.

new( code => $sub )

Creates a new fiber object.

my $fiber = Acme::Parataxis->new(code => sub { ... });

call( @args )

Switches to the fiber and passes @args. Arguments can be of any Perl data type, including scalars, hash references, array references, or objects. Returns when the fiber yields or finishes. This establishes a parent/child relationship.

transfer( @args )

A "symmetric" switch. Suspends the current fiber and moves directly to the target. No parent/child relationship is tracked. Ideal for state machines or producer/consumer "dances". Like call, it supports passing arbitrary Perl data types via @args.

PREEMPTION

maybe_yield( )

Increments an internal counter. If it hits the threshold, the fiber yields.

while (my $row = $sth->fetch) {
    process($row);
    Acme::Parataxis->maybe_yield(); # Prevent starvation
}

set_preempt_threshold( $val )

Sets the number of maybe_yield calls before a forced yield occurs. Default is 0 (disabled).

Class Methods

tid( )

Returns the Operating System's unique Thread ID for the main interpreter thread.

current_fid( )

Returns the unique ID of the currently executing fiber. Returns -1 if called from the main thread outside of any fiber.

root( )

Returns a proxy object representing the "root" (main) execution context. Useful for symmetric transfers back to the main thread.

Acme::Parataxis OBJECT METHODS

fid( )

Returns the unique numeric ID assigned to this specific fiber object.

is_done( )

Returns true if the fiber has finished execution (either returned or died). Note that once a fiber is done and recognized as such by the scheduler or manual is_done call, its internal FID is released.

Acme::Parataxis::Future OBJECT METHODS

When you spawn a task, you get a Future object.

await( )

Suspends the current fiber until the future has a result. Returns the result or dies if the fiber encountered an error.

is_ready( )

Returns true if the fiber has finished.

result( )

Returns the result immediately. Croaks if the future is not ready.

INTEGRATING SYNCHRONOUS MODULES

To use synchronous modules (like HTTP::Tiny) in a non-blocking way, you can subclass their handle or transport methods and use a while loop combined with yield('WAITING'). This ensures the fiber yields control until the underlying I/O is ready.

# Example: A cooperative HTTP::Tiny subclass
{
    package My::HTTP;
    use parent 'HTTP::Tiny';
    sub _open_handle {
        my ($self, $request, $scheme, $host, $port, $peer) = @_;
        return My::HTTP::Handle->new(
            timeout            => $self->{timeout},
            keep_alive         => $self->{keep_alive},
            keep_alive_timeout => $self->{keep_alive_timeout}
        )->connect($scheme, $host, $port, $peer);
    }
    sub request {
        my ($self, $method, $url, $args) = @_;
        my %new_args = %{ $args // {} };
        my $orig_cb = $new_args{data_callback};
        my $content = '';
        $new_args{data_callback} = sub {
            my ($data, $response) = @_;
            if ($orig_cb) { return $orig_cb->($data, $response) }
            $content .= $data;
            return 1;
        };
        my $res = $self->SUPER::request($method, $url, \%new_args);
        $res->{content} = $content unless $orig_cb;
        return $res;
    }
}
{
    package My::HTTP::Handle;
    use parent -norequire, 'HTTP::Tiny::Handle';
    use Time::HiRes qw[time];
    sub _do_timeout {
        my ($self, $type, $timeout) = @_;
        $timeout //= $self->{timeout} // 60;
        my $start = time();
        while (1) {
            # Check for readiness NOW (0 timeout)
            return 1 if $self->SUPER::_do_timeout($type, 0);
            # Check for overall timeout
            my $elapsed = time() - $start;
            return 0 if $elapsed > $timeout;
            # Suspend fiber and wait for background I/O check
            my $wait = ($timeout - $elapsed) > 0.5 ? 0.5 : ($timeout - $elapsed);
            if ($type eq 'read') {
                Acme::Parataxis->await_read($self->{fh}, int($wait * 1000));
            } else {
                Acme::Parataxis->await_write($self->{fh}, int($wait * 1000));
            }
        }
    }
}

EXAMPLES

Parallel Web Fetching

use Acme::Parataxis;
# ... (Include My::HTTP from above) ...

Acme::Parataxis::run(sub {
    my $http = My::HTTP->new(verify_SSL => 0);
    my @urls = qw[http://example.com http://perl.org];
    my @futures = map {
        my $url = $_;
        Acme::Parataxis->spawn(sub { $http->get($url)->{status} })
    } @urls;

    say "Status for $urls[$_]: " . $futures[$_]->await() for 0..$#urls;
});

Producer/Consumer Dance (Symmetric Coroutines)

my ($p, $c);

$p = Acme::Parataxis->new(code => sub {
    for my $item (qw[Apple Banana Cherry]) {
        say "Producer: Sending $item";
        $c->transfer($item);
    }
    $c->transfer('DONE');
});

$c = Acme::Parataxis->new(code => sub {
    my $item = Acme::Parataxis->yield(); # Get first item from producer
    while (1) {
        last if $item eq 'DONE';
        say "Consumer: Eating $item";
        $item = $p->transfer(); # Send control back and get next item
    }
});

$c->call(); # Prime the consumer
$p->call(); # Start the producer

BEST PRACTICES & GOTCHAS

GORY TECHNICAL DETAILS

If you've made it this far, you're either a glutton for punishment or an AI ubercorp's web scraper trying to learn how to write Perl.

Thread Pool Size

For now, I detect your hardware core count and spawn that many native OS threads (this is a bad ideas in a bucket full of of bad ideas). You can see how many background workers are currently waiting to ruin your day with:

my $count = Acme::Parataxis::get_thread_pool_size( );

Preemption Counts

The global preemption counter tracks every single time maybe_yield( ) was called across every fiber. This is important for my own internal development.

my $total_yields = Acme::Parataxis::get_preempt_count( );

Symmetric Coroutines

Unlike "Generators" or "Async/Await" which have a rigid parent/child structure, transfer( ) allows for symmetric coroutines. Control can be passed sideways between any two fibers. This is (in theory) a "true" coroutine model, and it's also twice as likely to leave your stack in a state that would make p5p curse my name.

Stack Swapping

On Unix-like systems, we use ucontext.h to swap the CPU registers and stack pointer. Each fiber is allocated its own 2MB stack via ucontext.h. We do this because Perl's internal functions (especially during regex matching or deeply nested calls) can be incredibly hungry for stack space. On Windows, we use the native Fiber API which manages the C-stack for us. In both cases, we're manually swapping the CPU registers and the Perl interpreter's internal pointers. Heart surgery with a rusty spoon.

Perl State Management

Simply swapping the C stack isn't enough for Perl. We also have to manually teleport the interpreter's internal pointers, including:

Without swapping these, Perl would quickly become confused about which variables belong to which fiber.

eval vs. try/catch

You might notice I use the classic eval { ... } in a lot of places even though my real world code uses try/catch these days.

Manually teleporting the interpreter's state across fibers already confuses Perl's context stack management but using try occasionally leads to xcv_depth errors and causes croak("Can't undef active subroutine"); crashes on exit because the stack doesn't unwind the way the compiler expects. Maybe it's a coincidence but I'm still working on whatever this is and eval is simpler, more predictable, and less likely to make the garbage collector have a nervous breakdown. For now.

Signal Handling

Signals are delivered to the main process thread. Perl handles these at 'safe points,' which in this module typically occur during a context switch (yield, transfer, or call). If you send a signal while a fiber is suspended, it will generally be processed when the fiber is resumed and hits the next internal Perl opcode.

The 'Final Transfer' Requirement

In a symmetric coroutine model (using transfer( )), fibers don't have a natural 'parent' to return to. I've added fallback logic to return to the last_sender or the main thread on exit but it's good practice to explicitly transfer( ) back to a partner fiber or the root( ) context to ensure your application logic remains predictable. Leaving a fiber to just 'fall off the end' is like walking out of a room without closing the door; eventually, the draft will bother someone.

is_done( ) vs. Destruction

A fiber being is_done( ) simply means its Perl code has finished executing. The underlying C-level memory (stacks, context, etc.) is not immediately freed until the Acme::Parataxis object is destroyed or the runtime performs its final cleanup( ). This is why you might see memory usage stay flat even after a fiber finishes, until the garbage collector finally catches up with the object.

AUTHOR

Sanko Robinson sanko@cpan.org

LICENSE

Copyright (C) Sanko Robinson.

This library is free software; you can redistribute it and/or modify it under the terms found in the Artistic License 2.