NAME
Bencher::Scenario::ListUtil - Benchmark List::Util (XS) vs PP implementation(s)
VERSION
This document describes version 0.04 of Bencher::Scenario::ListUtil (from Perl distribution Bencher-Scenario-ListUtil), released on 2016-06-26.
SYNOPSIS
To run benchmark with default option:
% bencher -m ListUtilTo run module startup overhead benchmark:
% bencher --module-startup -m ListUtilFor more options (dump scenario, list/include/exclude/add participants, list/include/exclude/add datasets, etc), see bencher or run bencher --help.
BENCHMARKED MODULES
Version numbers shown below are the versions used when running the sample benchmark.
List::Util 1.45
PERLANCAR::List::Util::PP 0.04
BENCHMARK PARTICIPANTS
List::Util::max (perl_code) [numeric]
Function call template:
List::Util::max(@{<list>})PERLANCAR::List::Util::PP::max (perl_code) [numeric]
Function call template:
PERLANCAR::List::Util::PP::max(@{<list>})List::Util::maxstr (perl_code) [stringy]
Function call template:
List::Util::maxstr(@{<list>})PERLANCAR::List::Util::PP::maxstr (perl_code) [stringy]
Function call template:
PERLANCAR::List::Util::PP::maxstr(@{<list>})List::Util::min (perl_code) [numeric]
Function call template:
List::Util::min(@{<list>})PERLANCAR::List::Util::PP::min (perl_code) [numeric]
Function call template:
PERLANCAR::List::Util::PP::min(@{<list>})List::Util::minstr (perl_code) [stringy]
Function call template:
List::Util::minstr(@{<list>})PERLANCAR::List::Util::PP::minstr (perl_code) [stringy]
Function call template:
PERLANCAR::List::Util::PP::minstr(@{<list>})
BENCHMARK DATASETS
num10
num100
num1000
str10
str100
str1000
SAMPLE BENCHMARK RESULTS
Run on: perl: v5.22.2, CPU: Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz (4 cores), OS: GNU/Linux Debian version 8.0, OS kernel: Linux version 3.16.0-4-amd64.
Benchmark with default options (bencher -m ListUtil):
+-----------------------------------+---------+-----------+-----------+------------+---------+---------+
| participant | dataset | rate (/s) | time (μs) | vs_slowest | errors | samples |
+-----------------------------------+---------+-----------+-----------+------------+---------+---------+
| PERLANCAR::List::Util::PP::maxstr | num1000 | 5650 | 177 | 1 | 5.3e-08 | 20 |
| PERLANCAR::List::Util::PP::minstr | num1000 | 6170 | 162 | 1.09 | 5.3e-08 | 20 |
| PERLANCAR::List::Util::PP::maxstr | str1000 | 8200 | 120 | 1.4 | 2.1e-07 | 20 |
| PERLANCAR::List::Util::PP::minstr | str1000 | 9200 | 110 | 1.6 | 6.4e-07 | 20 |
| List::Util::maxstr | num1000 | 10000 | 98 | 1.8 | 1.1e-07 | 20 |
| List::Util::minstr | num1000 | 10300 | 96.7 | 1.83 | 8e-08 | 20 |
| PERLANCAR::List::Util::PP::max | num1000 | 12000 | 85 | 2.1 | 1.1e-07 | 20 |
| PERLANCAR::List::Util::PP::min | num1000 | 13700 | 73.2 | 2.42 | 2.7e-08 | 20 |
| List::Util::minstr | str1000 | 18800 | 53.3 | 3.32 | 2.4e-08 | 24 |
| List::Util::maxstr | str1000 | 19700 | 50.7 | 3.49 | 2.7e-08 | 20 |
| List::Util::min | num1000 | 49000 | 20 | 8.7 | 2.7e-08 | 20 |
| List::Util::max | num1000 | 51000 | 19.6 | 9.03 | 6.7e-09 | 20 |
| PERLANCAR::List::Util::PP::maxstr | num100 | 64000 | 16 | 11 | 2.7e-08 | 20 |
| PERLANCAR::List::Util::PP::minstr | num100 | 66700 | 15 | 11.8 | 6.7e-09 | 20 |
| PERLANCAR::List::Util::PP::maxstr | str100 | 81578 | 12.258 | 14.446 | 8.1e-11 | 20 |
| PERLANCAR::List::Util::PP::minstr | str100 | 90600 | 11 | 16 | 8.9e-09 | 25 |
| PERLANCAR::List::Util::PP::max | num100 | 114000 | 8.8 | 20.1 | 3.3e-09 | 20 |
| List::Util::minstr | num100 | 115160 | 8.6838 | 20.392 | 1.1e-11 | 22 |
| List::Util::maxstr | num100 | 115158 | 8.68376 | 20.392 | 0 | 20 |
| PERLANCAR::List::Util::PP::min | num100 | 128200 | 7.802 | 22.7 | 9.3e-11 | 20 |
| List::Util::minstr | str100 | 186000 | 5.39 | 32.9 | 1.4e-09 | 27 |
| List::Util::maxstr | str100 | 194847 | 5.13224 | 34.5034 | 0 | 21 |
| List::Util::min | num100 | 475130 | 2.1047 | 84.135 | 1.2e-11 | 20 |
| List::Util::max | num100 | 500000 | 2 | 88 | 3.3e-09 | 20 |
| PERLANCAR::List::Util::PP::maxstr | num10 | 505000 | 1.98 | 89.5 | 8.3e-10 | 20 |
| PERLANCAR::List::Util::PP::minstr | num10 | 560000 | 1.8 | 99 | 2.7e-09 | 31 |
| PERLANCAR::List::Util::PP::maxstr | str10 | 630000 | 1.6 | 110 | 2.2e-09 | 25 |
| PERLANCAR::List::Util::PP::minstr | str10 | 690000 | 1.45 | 122 | 4.2e-10 | 20 |
| PERLANCAR::List::Util::PP::max | num10 | 840000 | 1.19 | 148.8 | 7.1e-11 | 20 |
| PERLANCAR::List::Util::PP::min | num10 | 910000 | 1.1 | 160 | 1.6e-09 | 21 |
| List::Util::maxstr | num10 | 938600 | 1.065 | 166.2 | 1.2e-11 | 20 |
| List::Util::minstr | num10 | 940200 | 1.064 | 166.5 | 1.2e-11 | 20 |
| List::Util::minstr | str10 | 1450000 | 0.691 | 256 | 2.1e-10 | 20 |
| List::Util::maxstr | str10 | 1490000 | 0.67 | 264 | 6.2e-10 | 20 |
| List::Util::max | num10 | 2600000 | 0.38 | 470 | 8.3e-10 | 20 |
| List::Util::min | num10 | 2750000 | 0.363 | 488 | 2.1e-10 | 20 |
+-----------------------------------+---------+-----------+-----------+------------+---------+---------+Benchmark module startup overhead (bencher -m ListUtil --module-startup):
+---------------------------+-----------+------------------------+------------+---------+---------+
| participant | time (ms) | mod_overhead_time (ms) | vs_slowest | errors | samples |
+---------------------------+-----------+------------------------+------------+---------+---------+
| List::Util | 5.3 | 2.8 | 1 | 4.8e-05 | 21 |
| PERLANCAR::List::Util::PP | 5.2 | 2.7 | 1 | 3.9e-05 | 20 |
| perl -e1 (baseline) | 2.5 | 0 | 2.1 | 1.1e-05 | 20 |
+---------------------------+-----------+------------------------+------------+---------+---------+DESCRIPTION
EARLY VERSION, ONLY A FEW FUNCTIONS HAVE BEEN BENCHMARKED.
For max*/min*, in general the XS version are about 3x faster than PP.
Packaging a benchmark script as a Bencher scenario makes it convenient to include/exclude/add participants/datasets (either via CLI or Perl code), send the result to a central repository, among others . See Bencher and bencher (CLI) for more details.
HOMEPAGE
Please visit the project's homepage at https://metacpan.org/release/Bencher-Scenario-ListUtil.
SOURCE
Source repository is at https://github.com/perlancar/perl-Bencher-Scenario-ListUtil.
BUGS
Please report any bugs or feature requests on the bugtracker website https://rt.cpan.org/Public/Dist/Display.html?Name=Bencher-Scenario-ListUtil
When submitting a bug or request, please include a test-file or a patch to an existing test-file that illustrates the bug or desired feature.
AUTHOR
perlancar <perlancar@cpan.org>
COPYRIGHT AND LICENSE
This software is copyright (c) 2016 by perlancar@cpan.org.
This is free software; you can redistribute it and/or modify it under the same terms as the Perl 5 programming language system itself.