NAME
Job::Async::Redis - Net::Async::Redis backend for Job::Async
SYNOPSIS
use IO::Async::Loop;
use Job::Async;
my $loop = IO::Async::Loop->new;
$loop->add( my $jobman = Job::Async->new );
my $client = $jobman->client(
redis => { uri => 'redis://127.0.0.1', }
);
my $worker = $jobman->worker(
redis => { uri => 'redis://127.0.0.1', }
);
Future->needs_all(
$client->start,
)->get;
$worker->jobs->each(sub {
$_->done('' . reverse $_->data('some_data'))
});
print Future->needs_all(
$client->start,
$worker->trigger
)->then(sub {
$client->submit(
some_data => 'reverse me please'
)->future
})->get;
DESCRIPTION
The system can be configured to select a performance/reliability tradeoff as follows. Please note that clients and workers must be configured to use the same mode - results are undefined if you try to mix clients and workers using different modes. If it works, don't rely on it.
Operational modes
simple
Jobs are submitted by serialising as JSON and pushing to a Redis list as a queue.
Workers retrieve jobs from queue, and send the results via pubsub.
Multiple queues can be used for priority handling - the client can route based on the job data.
recoverable
As with simple mode, queues are used for communication between the clients and workers. However, these queues contain only the job ID.
Actual job data is stored in a hash key, and once the worker completes the result is also stored here.
Job completion will trigger a "publish" in Net::Redis::Async::Commands notification, allowing clients to listen for completion.
Multiple queues can be used, as with simple
mode.
reliable
Each worker uses "brpoplpush" in Net::Async::Redis::Commands to await job IDs posted to a single queue.
Job details are stored in a hash key, as with the recoverable
approach.
When a worker starts on a job, the ID is atomically moved to an in-process queue, and this is used to track whether workers are still valid.
Only one queue is allowed per worker, due to limitations of the "brpoplpush" in Net::Async::Redis::Commands implementation as described in this issue.
AUTHOR
Tom Molesworth <TEAM@cpan.org>
LICENSE
Copyright Tom Molesworth 2016-2019. Licensed under the same terms as Perl itself.