NAME
MCE::Hobo - A fast, pure-Perl threads-like parallelization module
VERSION
This document describes MCE::Hobo version 1.699_009
SYNOPSIS
use
MCE::Hobo;
MCE::Hobo->create(
sub
{
"Hello from hobo\n"
} )->
join
();
sub
parallel {
my
(
$arg1
) =
@_
;
"Hello again, $arg1\n"
;
}
MCE::Hobo->create( \
¶llel
,
$_
)
for
1 .. 3;
my
@hobos
= MCE::Hobo->list();
my
@running
= MCE::Hobo->list_running();
my
@joinable
= MCE::Hobo->list_joinable();
$_
->
join
()
for
@hobos
;
my
$hobo
= mce_async {
foreach
(
@files
) { ... } };
$hobo
->
join
();
if
(
my
$err
=
$hobo
->error()) {
warn
(
"Hobo error: $err\n"
);
}
# Get a hobo's object
$hobo
= MCE::Hobo->self();
# Get a hobo's ID
$tid
= MCE::Hobo->tid();
# "$$.tid"
$tid
=
$hobo
->tid();
$pid
= MCE::Hobo->pid();
# $$
$pid
=
$hobo
->pid();
# Test hobo objects
if
(
$hobo1
==
$hobo2
) {
...
}
# Give other hobos a chance to run
MCE::Hobo->yield();
MCE::Hobo->yield(0.05);
# Wantarray-aware return context
my
(
$value1
,
$value2
) =
$hobo
->
join
();
my
$value
=
$hobo
->
join
();
# Check hobo's state
if
(
$hobo
->is_running()) {
sleep
1;
}
if
(
$hobo
->is_joinable()) {
$hobo
->
join
();
}
# Send a signal to a hobo
$hobo
->
kill
(
'SIGUSR1'
);
# Exit a hobo
MCE::Hobo->
exit
();
DESCRIPTION
A hobo is a migratory worker inside the machine that carries the asynchronous gene. Hobos are equipped with threads
-like capability for running code asynchronously. Unlike threads, each hobo is a unique process to the underlying OS. The IPC is managed by MCE::Shared
, which runs on all major platforms including Cygwin.
MCE::Hobo
may be used as a standalone or together with MCE
including running alongside threads
.
The following is a parallel demonstration.
use
strict;
use
warnings;
use
MCE::Hobo;
# usage: head -20 file.txt | perl script.pl
# my $aa = MCE::Shared->array(); # OO with on-demand deref @{}
# my $ha = MCE::Shared->hash(); # OO with on-demand deref %{}
# my $oh = MCE::Shared->ordhash(); # OO with on-demand deref %{}
# my $va = MCE::Shared->scalar(); # OO interface only: ->set, ->get
my
$ifh
= MCE::Shared->handle(
"<"
, \
*STDIN
);
# shared
my
$ofh
= MCE::Shared->handle(
">"
, \
*STDOUT
);
my
$ary
= MCE::Shared->array();
sub
parallel_task {
my
(
$id
) =
@_
;
while
( <
$ifh
> ) {
printf
{
$ofh
}
"[ %4d ] %s"
, $.,
$_
;
# $ary->[ $. - 1 ] = "[ ID $id ] read line $.\n" ); # dereference
$ary
->set( $. - 1,
"[ ID $id ] read line $.\n"
);
# OO faster
}
}
my
$hobo1
= MCE::Hobo->new(
"parallel_task"
, 1 );
my
$hobo2
= MCE::Hobo->new( \
¶llel_task
, 2 );
my
$hobo3
= MCE::Hobo->new(
sub
{ parallel_task(3) } );
$_
->
join
for
MCE::Hobo->list();
# search array -- single IPC
my
@vals
=
$ary
->vals(
"val =~ / ID 2 /"
);
{
*STDERR
}
join
(
""
,
@vals
);
OVERRIDING DEFAULTS
The following list options which may be overridden when loading the module.
There is a simpler way to enable Sereal. The following will attempt to use Sereal if available, otherwise defaults to Storable for serialization.
API DOCUMENTATION
- $hobo = MCE::Hobo->create( { posix_exit => 1 }, FUNCTION, ARGS )
- $hobo = MCE::Hobo->create( FUNCTION, ARGS )
- $hobo = MCE::Hobo->new( FUNCTION, ARGS )
-
This will create a new hobo that will begin execution with function as the entry point, and optionally ARGS for list of parameters. It will return the corresponding MCE::Hobo object, or undef if hobo creation failed.
Options may be specified via a hash structure. At this time,
posix_exit
is the only option supported which callsPOSIX::_exit(0)
when finished. The default isCORE::exit(0)
. Setposix_exit
to avoid all END and destructor processing.FUNCTION may either be the name of a function, an anonymous subroutine, or a code ref.
my
$hobo
= MCE::Hobo->create(
"func_name"
, ... );
# or
my
$hobo
= MCE::Hobo->create(
sub
{ ... }, ... );
# or
my
$hobo
= MCE::Hobo->create( \
&func
, ... );
The
new()
method is an alias forcreate()
. - mce_async { BLOCK } ARGS;
- mce_async { BLOCK };
-
mce_async
runs the block asynchronously similarly toMCE::Hobo-
create()>. It returns the hobo object, or undef if hobo creation failed.my
$hobo
= mce_async {
foreach
(
@files
) { ... } };
$hobo
->
join
();
if
(
my
$err
=
$hobo
->error()) {
warn
(
"Hobo error: $err\n"
);
}
- $hobo->join()
-
This will wait for the corresponding hobo to complete its execution. In non-voided context,
join()
will return the value(s) of the entry point function.The context (void, scalar or list) for the return value(s) for
join
is determined at the time of joining and mostly wantarray-aware.my
$hobo1
= MCE::Hobo->create(
sub
{
my
@results
=
qw(foo bar baz)
;
return
(
@results
);
});
my
@res1
=
$hobo1
->
join
;
# ( foo, bar, baz )
my
$res1
=
$hobo1
->
join
;
# baz
my
$hobo2
= MCE::Hobo->create(
sub
{
return
'foo'
;
});
my
@res2
=
$hobo2
->
join
;
# ( foo )
my
$res2
=
$hobo2
->
join
;
# foo
- $hobo1->equal( $hobo2 )
-
Tests if two hobo objects are the same hobo or not. Hobo comparison is based on process IDs. This is overloaded to the more natural forms.
if
(
$hobo1
==
$hobo2
) {
print
(
"Hobos are the same\n"
);
}
# or
if
(
$hobo1
!=
$hobo2
) {
print
(
"Hobos differ\n"
);
}
- $hobo->error()
-
Hobos are executed in an
eval
context. This method will returnundef
if the hobo terminates normally. Otherwise, it returns the value of$@
associated with the hobo's execution status in itseval
context. - $hobo->exit()
-
This sends
'SIGQUIT'
to the hobo object, notifying hobo to exit. It returns the hobo object to allow for method chaining. It is important to join later if not immediately to not leave a zombie or defunct process.$hobo
->
exit
()->
join
();
...
$hobo
->
join
();
# later
- MCE::Hobo->exit()
-
A hobo can be exited at any time by calling
MCE::Hobo-
exit()>. This behaves the same asexit(status)
when called from the main process. - MCE::Hobo->finish()
-
This class method is called automatically by
END
, but may be called explicitly. Two shared objects toMCE::Shared
are destroyed. An error is emmitted via croak if there are active hobos not yet joined.MCE::Hobo->create(
'task1'
,
$_
)
for
1 .. 4;
$_
->
join
for
MCE::Hobo->list();
MCE::Hobo->create(
'task2'
,
$_
)
for
1 .. 4;
$_
->
join
for
MCE::Hobo->list();
MCE::Hobo->create(
'task3'
,
$_
)
for
1 .. 4;
$_
->
join
for
MCE::Hobo->list();
MCE::Hobo->finish();
- $hobo->is_running()
-
Returns true if a hobo is still running.
- $hobo->is_joinable()
-
Returns true if the hobo has finished running and not yet joined.
- $hobo->kill( 'SIG...' )
-
Sends the specified signal to the hobo. Returns the hobo object to allow for method chaining. As with
exit
, it is important to join eventually if not immediately to not leave a zombie or defunct process.$hobo
->
kill
(
'SIG...'
)->
join
();
The following is a parallel demonstration comparing
MCE::Shared
againstRedis
andRedis::Fast
on a Fedora 23 VM. Joining is after all workers have been notified to quit.use
Redis;
use
Redis::Fast;
use
MCE::Hobo;
use
MCE::Shared;
my
$redis
= Redis->new();
my
$rfast
= Redis::Fast->new();
my
$array
= MCE::Shared->array();
sub
parallel_redis {
my
(
$_redis
) =
@_
;
my
(
$count
,
$quit
,
$len
) = (0, 0);
# instead, use a flag to exit loop
$SIG
{
'QUIT'
} =
sub
{
$quit
= 1 };
while
(1) {
$len
=
$_redis
->rpush(
'list'
,
$count
++);
last
if
$quit
;
}
$count
;
}
sub
parallel_array {
my
(
$count
,
$quit
,
$len
) = (0, 0);
# do not exit from inside handler
$SIG
{
'QUIT'
} =
sub
{
$quit
= 1 };
while
(1) {
$len
=
$array
->
push
(
$count
++);
last
if
$quit
;
}
$count
;
}
sub
benchmark_this {
my
(
$desc
,
$num_hobos
,
$timeout
,
$code
,
@args
) =
@_
;
my
(
$start
,
$total
) = (
time
(), 0);
MCE::Hobo->new(
$code
,
@args
)
for
1..
$num_hobos
;
sleep
$timeout
;
# joining is not immediate; ok
$_
->
kill
(
'QUIT'
)
for
MCE::Hobo->list();
# joining later; ok
$total
+=
$_
->
join
()
for
MCE::Hobo->list();
printf
"$desc <> duration: %0.03f secs, count: $total\n"
,
time
() -
$start
;
sleep
0.2;
}
benchmark_this(
'Redis '
, 8, 5.0, \
¶llel_redis
,
$redis
);
benchmark_this(
'Redis::Fast'
, 8, 5.0, \
¶llel_redis
,
$rfast
);
benchmark_this(
'MCE::Shared'
, 8, 5.0, \
¶llel_array
);
- $hobo->list()
-
Returns a list of all hobos not yet joined.
- $hobo->list_running()
-
Returns a list of all hobos that are still running.
- $hobo->list_joinable()
-
Returns a list of all hobos that have completed running. Thus, ready to be joined without blocking.
- MCE::Hobo->self()
-
Class method that allows a hobo to obtain it's own MCE::Hobo object.
- $hobo->pid()
- $hobo->tid()
-
Returns the ID of the hobo. TID is composed of process and thread IDs together as a string value.
PID: $$
TID:
"$$.tid"
- MCE::Hobo->pid()
- MCE::Hobo->tid()
-
Class methods that allows a hobo to obtain its own ID.
- MCE::Hobo->yield( floating_seconds )
-
Let this hobo yield CPU time to other hobos. By default, the class method calls
sleep(0.0002)
on Unix including Cygwin andsleep(0.001)
on Windows.MCE::Hobo->yield();
MCE::Hobo->yield(0.05);
CREDITS
The inspiration for MCE::Hobo
comes from wanting threads
-like behavior for processes. Both can run side-by-side including safe-use by MCE workers. Likewise, the documentation resembles threads
.
SEE ALSO
INDEX
AUTHOR
Mario E. Roy, <marioeroy AT gmail DOT com>