NAME

Parallel::MapReduce::Worker - MapReduce, local worker

SYNOPSIS

use Parallel::MapReduce::Worker;
my $w = new Parallel::MapReduce::Worker;

my @chunks = chunk_n_store ($memd, $A, $job, 1000);
my $cs = $w->map (\@chunks, 'slice1:', SERVERS, $job);
...
my $ks = $w->reduce (\@cs, SERVERS, $job);

DESCRIPTION

This class implements a local, sequential worker. You will only know about it if you want to subclass it to implement your own worker.

INTERFACE

Constructor

Nothing important to be said.

Destructor

$w->shutdown

While there is no DESTROY method (for technical reasons), there is a shutdown method. It is supposed to terminate any background processes a worker might have started.

For a single-thread, local version nothing will be done, but individual subclasses might have to do substantial work to tear-down network connections, remote servers, etc.

Methods

map

$keys = $w->map ($chunks, $slice, $servers, $job)

The chunks are keys into the memcached servers. They allow to reconstruct the hash slice to be worked on. The slice is a simple id for that hash slice. The list reference to the memcached servers is obviously also necessary and the job id is an identifier for the current MR computation.

reduce

$chunks = $w->reduce ($keys, $servers, $job)

The keys are the keys into the intermediate hash within the memcached servers. The list reference to the memcached servers is obviously also necessary and the job id is an identifier for the current MR computation.

SEE ALSO

Parallel::MapReduce

COPYRIGHT AND LICENSE

Copyright 200[8] by Robert Barta, <drrho@cpan.org>

This library is free software; you can redistribute it and/or modify it under the same terms as Perl itself.