NAME
Bencher::Scenario::SetComparison - Benchmark set comparison
VERSION
This document describes version 0.002 of Bencher::Scenario::SetComparison (from Perl distribution Bencher-Scenario-SetComparison), released on 2016-10-19.
SYNOPSIS
To run benchmark with default option:
% bencher -m SetComparisonTo run module startup overhead benchmark:
% bencher --module-startup -m SetComparisonFor more options (dump scenario, list/include/exclude/add participants, list/include/exclude/add datasets, etc), see bencher or run bencher --help.
DESCRIPTION
TODO: include more set modules.
TODO: compare complex elements.
Packaging a benchmark script as a Bencher scenario makes it convenient to include/exclude/add participants/datasets (either via CLI or Perl code), send the result to a central repository, among others . See Bencher and bencher (CLI) for more details.
BENCHMARKED MODULES
Version numbers shown below are the versions used when running the sample benchmark.
Test::Deep 1.120
Set::Tiny 0.04
BENCHMARK PARTICIPANTS
Test::Deep (perl_code)
Code template:
state $set = Test::Deep::set(@{<set>}); Test::Deep::eq_deeply(<array>, $set)Set::Tiny (perl_code) [simple-elements]
Code template:
state $set1 = Set::Tiny->new(@{<array>}); state $set2 = Set::Tiny->new(@{<set>}); $set1->symmetric_difference($set2)->size == 0 ? 1:0
BENCHMARK DATASETS
elems=10num
elems=100num
elems=200num
SAMPLE BENCHMARK RESULTS
Run on: perl: v5.24.0, CPU: Intel(R) Core(TM) i5-2400 CPU @ 3.10GHz (4 cores), OS: GNU/Linux Debian version 8.0, OS kernel: Linux version 3.16.0-4-amd64.
Benchmark with default options (bencher -m SetComparison):
#table1#
+-------------+--------------+-----------+-----------+------------+-----------+---------+
| participant | dataset | rate (/s) | time (ms) | vs_slowest | errors | samples |
+-------------+--------------+-----------+-----------+------------+-----------+---------+
| Test::Deep | elems=200num | 4.18 | 239 | 1 | 0.00021 | 20 |
| Test::Deep | elems=100num | 17 | 60 | 4 | 6.3e-05 | 20 |
| Test::Deep | elems=10num | 1500 | 0.67 | 360 | 9.6e-07 | 20 |
| Set::Tiny | elems=200num | 15000 | 0.067 | 3600 | 1.1e-07 | 20 |
| Set::Tiny | elems=100num | 30000 | 0.033 | 7200 | 6.2e-08 | 23 |
| Set::Tiny | elems=10num | 190000 | 0.0054 | 45000 | 6.7e-09 | 20 |
+-------------+--------------+-----------+-----------+------------+-----------+---------+Benchmark module startup overhead (bencher -m SetComparison --module-startup):
#table2#
+---------------------+-----------+------------------------+------------+-----------+---------+
| participant | time (ms) | mod_overhead_time (ms) | vs_slowest | errors | samples |
+---------------------+-----------+------------------------+------------+-----------+---------+
| Test::Deep | 26 | 20 | 1 | 0.00018 | 21 |
| Set::Tiny | 6.4 | 0.4 | 4.1 | 2.1e-05 | 20 |
| perl -e1 (baseline) | 6 | 0 | 5 | 6.6e-05 | 20 |
+---------------------+-----------+------------------------+------------+-----------+---------+BENCHMARK NOTES
Test::Deep is slow :)
HOMEPAGE
Please visit the project's homepage at https://metacpan.org/release/Bencher-Scenario-SetComparison.
SOURCE
Source repository is at https://github.com/perlancar/perl-Bencher-Scenario-SetComparison.
BUGS
Please report any bugs or feature requests on the bugtracker website https://rt.cpan.org/Public/Dist/Display.html?Name=Bencher-Scenario-SetComparison
When submitting a bug or request, please include a test-file or a patch to an existing test-file that illustrates the bug or desired feature.
SEE ALSO
Bencher::Scenario::BagComparison
Bencher::Scenario::SetOperationModules
AUTHOR
perlancar <perlancar@cpan.org>
COPYRIGHT AND LICENSE
This software is copyright (c) 2016 by perlancar@cpan.org.
This is free software; you can redistribute it and/or modify it under the same terms as the Perl 5 programming language system itself.