NAME

Future::Batch::XS - XS implementation of batch processing for Future operations

VERSION

Version 0.02

SYNOPSIS

use Future::Batch::XS qw(batch);

# Functional interface
my $results = batch(
    items      => \@urls,
    concurrent => 5,
    worker     => sub {
        my ($url, $index) = @_;
        return $http->GET($url);  # returns a Future
    },
)->get;

# Object-oriented interface
my $batch = Future::Batch::XS->new(
    concurrent => 5,
    fail_fast  => 0,
);

my $future = $batch->run(
    items  => \@items,
    worker => sub { ... },
);

my $results = $future->get;

# With an event loop (for true non-blocking with immediate futures)
use IO::Async::Loop;
my $loop = IO::Async::Loop->new;

my $future = batch(
    items      => \@items,
    concurrent => 5,
    loop       => $loop,
    worker     => sub { ... },
);

my $results = $loop->await($future);

# With progress tracking
batch(
    items       => \@items,
    concurrent  => 10,
    on_progress => sub {
        my ($completed, $total) = @_;
        printf "Progress: %d/%d\n", $completed, $total;
    },
    worker => sub { ... },
)->get;

DESCRIPTION

Future::Batch::XS provides a way to process multiple items through a Future returning worker function with controlled concurrency. It ensures that no more than a specified number of operations run simultaneously, while maintaining result order matching the input order.

This is the XS implementation of Future::Batch, providing the same API with improved performance. The entire batch processing loop is implemented in C/XS, including Future callbacks.

This is useful for scenarios like:

• Fetching multiple URLs with limited concurrent connections

• Processing files with bounded parallelism

• Rate-limited API calls

• Any batch operation where you need to balance throughput with resource usage

EXPORTS

batch

my $future = batch(%args);

Functional interface. Takes the same arguments as new() and run() combined. Returns a Future that resolves to an arrayref of results.

METHODS

new

my $batch = Future::Batch::XS->new(%args);

Create a new batch processor.

Arguments

concurrent => $n

Maximum number of concurrent operations. Default: 10.

fail_fast => $bool

If true, abort remaining operations on first failure. Default: false.

on_progress => sub { my ($completed, $total) = @_; ... }

Optional callback invoked after each item completes (success or failure).

loop => $loop

Optional event loop object (e.g., IO::Async::Loop). When provided, the batch processor will use $loop->later() to schedule starting new items, yielding to the event loop between items. This ensures non-blocking behavior even when workers return immediate (already-completed) futures.

Without a loop, immediate futures are processed synchronously, which is fine for truly async workers that return pending futures.

run

my $future = $batch->run(%args);

Execute the batch operation. Returns a Future.

Arguments

items => \@items

Arrayref of items to process.

worker => sub { my ($item, $index) = @_; return $future; }

Coderef that receives each item and its index, and should return a Future. If the worker returns a non-Future value, it will be wrapped in Future->done(). If the worker dies, the error is captured.

concurrent

my $n = $batch->concurrent;

Returns the concurrency limit.

fail_fast

my $bool = $batch->fail_fast;

Returns the fail_fast setting.

on_progress

my $cb = $batch->on_progress;

Returns the progress callback, if set.

loop

my $loop = $batch->loop;

Returns the event loop, if set.

RESULT HANDLING

On success, the returned Future resolves to an arrayref of results in the same order as the input items.

On failure (when any worker fails and fail_fast is false), the Future fails with:

("Batch failed with N error(s)", "batch", \@errors, \@partial_results)

On failure with fail_fast:

("Batch aborted: <error>", "batch", \@errors, \@partial_results)

The @errors arrayref contains hashrefs with:

{ index => $idx, item => $item, failure => \@failure_args }

IMPLEMENTATION NOTES

This module creates XS closures for Future's on_done/on_fail callbacks. Each callback stores its closure data (state reference, item, index) in a C struct attached to the CV via SV magic (PERL_MAGIC_ext). The callback data is automatically freed when the CV is garbage collected.

AUTHOR

LNATION, <email at lnation.org>

BUGS

Please report any bugs or feature requests to bug-future-batch-xs at rt.cpan.org, or through the web interface at https://rt.cpan.org/NoAuth/ReportBug.html?Queue=Future-Batch-XS.

SEE ALSO

Future::Batch, Future, Future::Utils, Future::Queue

LICENSE AND COPYRIGHT

This software is Copyright (c) 2026 by LNATION.

This is free software, licensed under:

The Artistic License 2.0 (GPL Compatible)

cpanm Future::Batch::XS

perl -MCPAN -e shell
install Future::Batch::XS