MCE-1.499_001

Changes for version 1.499_001 - 2013-09-22

MCE 1.5 is underway. I have one method to add to MCE::Queue. In the meantime, the following have been completed code-wise.
- Will work on documentation in the next 2 weeks and add additional examples to highlight the various forms of parallel computing with MCE.
- Regards, ;) Mario
MCE.pm (8 data channels for up to 3x IPC handling)
- e.g. 8 workers calling sendto in MCE 1.4 sends 80,000 per second. That is now over 250,000 per second in MCE 1.5. That is just the beginning. The following is not a good benchmark due to printf formatting being done inside the preserve order code block run by the manager. Foreach is able to do 40,000 and forseq 78,000 when commenting out the printf statement. In any event, IPC is much faster in MCE 1.5.
  - (Before) (After)
  - Parallel::Loops 600 600 $mce->foreach 21,500 34,000 (+ 12,500) $mce->forseq 64,000 70,000 (+ 6,000) $mce->forchunk 450,000 465,000 (+ 15,000)
MCE::Flow, MCE::Grep, MCE::Loop, MCE::Map, MCE::Queue, MCE::Stream
- How did I forget... The gather option... Take a look at the MCE::Flow pod page when you have a chance.
MCE 1.5 is backwards compatible with 1.4.x and lower.
All public methods can be called directly using the package name and method e.g. MCE->wid, MCE->run.
- MCE->new( max_workers => 'auto', user_func => sub { my $wid = MCE->wid; MCE->sendto('STDOUT', "Hello from $wid\n"); } );
- MCE->run;
New example (interval.pl). More to come for the final 1.5 release.
New options (gather, interval, task_name).
- Changes to other options are described below.
- The task_end option can now be specified at the top level. See docs for use case.
- Input_data can be specified inside the first task instead of having to specify this at the top level. Note that input_data is ignored if specified for tasks having task_id > 0.
  - user_tasks => [{ input_data => \@list, ... },{ ... }
New public methods chunk_id, gather, freeze, thaw, yield, task_name, print, printf, say
Optimized the egrep.pl and wc.pl examples. These run much faster. Try these out against large log files. Both examples fly.
Barrier synchronization update. Two sockets are utilized instead of two lock files. This, at last, now works wonderfully under the Cygwin environment. For threading, the removal of the 2 lock files allows one to have a greater number of threads within the limits of ulimit. Previously, only about 1/3rd of ulimit -n was possible. One can now spawn just under 1/2 of ulimit -- int(ulimit -n / 2 - 10).
Removed the logic for determining MAX_OPEN_FILES and MAX_USER_PROCS. MCE no longer limits on max_workers allowed.
Code re-factor work. Added a private method _validate_runstate called by various methods. Organized the code slightly such as placement of methods.
Localize the input scalar $_ prior to calling user_func. Folks can use $_ for input_data and sequence of numbers. Added section to docs explaining DEFAULT INPUT SCALAR.