NAME
Bencher::Scenario::DataComparisonModules::Compare - Benchmark data comparison modules
VERSION
This document describes version 0.003 of Bencher::Scenario::DataComparisonModules::Compare (from Perl distribution Bencher-Scenarios-DataComparisonModules), released on 2018-08-12.
SYNOPSIS
To run benchmark with default option:
% bencher -m DataComparisonModules::CompareTo run module startup overhead benchmark:
% bencher --module-startup -m DataComparisonModules::CompareFor more options (dump scenario, list/include/exclude/add participants, list/include/exclude/add datasets, etc), see bencher or run bencher --help.
DESCRIPTION
Packaging a benchmark script as a Bencher scenario makes it convenient to include/exclude/add participants/datasets (either via CLI or Perl code), send the result to a central repository, among others . See Bencher and bencher (CLI) for more details.
BENCHMARKED MODULES
Version numbers shown below are the versions used when running the sample benchmark.
Data::Cmp 0.003
Data::Compare 1.25
Data::Diff 0.01
BENCHMARK PARTICIPANTS
Data::Compare::Compare (perl_code)
Function call template:
Data::Compare::Compare(<d1>, <d2>)Data::Cmp::cmp_data (perl_code)
Function call template:
Data::Cmp::cmp_data(<d1>, <d2>)Test::Deep::NoTest::eq_deeply (perl_code)
Code template:
use Test::Deep::NoTest; eq_deeply(<d1>, <d2>)Data::Comparator::data_comparator (perl_code)
Function call template:
Data::Comparator::data_comparator(<d1>, <d2>)Data::Diff::diff (perl_code)
Code template:
use Data::Diff; Data::Diff->new(<d1>, <d2>)
BENCHMARK DATASETS
simple scalar
array len=10
array len=1000
hash keys=10
hash keys=1000
SAMPLE BENCHMARK RESULTS
Run on: perl: v5.26.1, CPU: Intel(R) Core(TM) M-5Y71 CPU @ 1.20GHz (2 cores), OS: GNU/Linux LinuxMint version 18.3, OS kernel: Linux version 4.10.0-38-generic.
Benchmark with default options (bencher -m DataComparisonModules::Compare):
#table1#
+-----------------------------------+----------------+-----------+-----------+------------+---------+---------+
| participant | dataset | rate (/s) | time (ms) | vs_slowest | errors | samples |
+-----------------------------------+----------------+-----------+-----------+------------+---------+---------+
| Data::Diff::diff | array len=1000 | 160 | 6.2 | 1 | 1.4e-05 | 20 |
| Data::Comparator::data_comparator | hash keys=1000 | 190 | 5.1 | 1.2 | 6.5e-06 | 20 |
| Data::Diff::diff | hash keys=1000 | 190 | 5.1 | 1.2 | 6.9e-06 | 20 |
| Data::Compare::Compare | hash keys=1000 | 209 | 4.78 | 1.29 | 4e-06 | 20 |
| Data::Comparator::data_comparator | array len=1000 | 220 | 4.6 | 1.3 | 9.4e-06 | 21 |
| Data::Compare::Compare | array len=1000 | 290 | 3.4 | 1.8 | 6.1e-06 | 20 |
| Test::Deep::NoTest::eq_deeply | hash keys=1000 | 370 | 2.7 | 2.3 | 4.2e-06 | 20 |
| Data::Cmp::cmp_data | hash keys=1000 | 620 | 1.6 | 3.8 | 4.1e-06 | 20 |
| Data::Cmp::cmp_data | array len=1000 | 1000 | 0.96 | 6.4 | 2.8e-06 | 25 |
| Test::Deep::NoTest::eq_deeply | array len=1000 | 1300 | 0.74 | 8.3 | 5.9e-06 | 21 |
| Test::Deep::NoTest::eq_deeply | hash keys=10 | 6100 | 0.16 | 38 | 2.7e-07 | 20 |
| Test::Deep::NoTest::eq_deeply | array len=10 | 8200 | 0.12 | 50 | 2.4e-07 | 25 |
| Data::Diff::diff | array len=10 | 14000 | 0.069 | 89 | 1.1e-07 | 20 |
| Data::Comparator::data_comparator | hash keys=10 | 18000 | 0.057 | 110 | 1.1e-07 | 20 |
| Data::Diff::diff | hash keys=10 | 18000 | 0.054 | 110 | 1e-07 | 22 |
| Data::Compare::Compare | hash keys=10 | 18000 | 0.054 | 110 | 1.2e-07 | 25 |
| Data::Comparator::data_comparator | array len=10 | 19000 | 0.052 | 120 | 1.1e-07 | 20 |
| Data::Compare::Compare | array len=10 | 26000 | 0.038 | 160 | 5.2e-08 | 21 |
| Data::Cmp::cmp_data | hash keys=10 | 63000 | 0.016 | 390 | 2.7e-08 | 20 |
| Data::Cmp::cmp_data | array len=10 | 78300 | 0.0128 | 482 | 5.5e-09 | 29 |
| Test::Deep::NoTest::eq_deeply | simple scalar | 167000 | 0.00597 | 1030 | 1.6e-09 | 22 |
| Data::Diff::diff | simple scalar | 216980 | 0.0046087 | 1336 | 4.5e-11 | 21 |
| Data::Comparator::data_comparator | simple scalar | 311000 | 0.00322 | 1910 | 1.7e-09 | 20 |
| Data::Compare::Compare | simple scalar | 440000 | 0.0023 | 2700 | 4.1e-09 | 21 |
| Data::Cmp::cmp_data | simple scalar | 1100000 | 0.00092 | 6700 | 2.7e-09 | 24 |
+-----------------------------------+----------------+-----------+-----------+------------+---------+---------+Benchmark module startup overhead (bencher -m DataComparisonModules::Compare --module-startup):
#table2#
+---------------------+-----------+------------------------+------------+---------+---------+
| participant | time (ms) | mod_overhead_time (ms) | vs_slowest | errors | samples |
+---------------------+-----------+------------------------+------------+---------+---------+
| Data::Compare | 26.1 | 19.9 | 1 | 2.5e-05 | 20 |
| Data::Diff | 17.8 | 11.6 | 1.47 | 9.6e-06 | 20 |
| Test::Deep::NoTest | 16.5 | 10.3 | 1.58 | 1.3e-05 | 20 |
| Data::Comparator | 12 | 5.8 | 2.18 | 6.5e-06 | 20 |
| Data::Cmp | 9.97 | 3.77 | 2.62 | 8.9e-06 | 21 |
| perl -e1 (baseline) | 6.2 | 0 | 4.2 | 2.3e-05 | 20 |
+---------------------+-----------+------------------------+------------+---------+---------+To display as an interactive HTML table on a browser, you can add option --format html+datatables.
HOMEPAGE
Please visit the project's homepage at https://metacpan.org/release/Bencher-Scenarios-DataComparisonModules.
SOURCE
Source repository is at https://github.com/perlancar/perl-Bencher-Scenarios-DataComparisonModules.
BUGS
Please report any bugs or feature requests on the bugtracker website https://rt.cpan.org/Public/Dist/Display.html?Name=Bencher-Scenarios-DataComparisonModules
When submitting a bug or request, please include a test-file or a patch to an existing test-file that illustrates the bug or desired feature.
AUTHOR
perlancar <perlancar@cpan.org>
COPYRIGHT AND LICENSE
This software is copyright (c) 2018 by perlancar@cpan.org.
This is free software; you can redistribute it and/or modify it under the same terms as the Perl 5 programming language system itself.