NAME

Bencher::Scenario::SetOperationModules - Benchmark Perl set operation (union, intersection, diff, symmetric diff) modules

VERSION

This document describes version 0.10 of Bencher::Scenario::SetOperationModules (from Perl distribution Bencher-Scenario-SetOperationModules), released on 2017-01-25.

SYNOPSIS

To run benchmark with default option:

% bencher -m SetOperationModules

To run module startup overhead benchmark:

% bencher --module-startup -m SetOperationModules

For more options (dump scenario, list/include/exclude/add participants, list/include/exclude/add datasets, etc), see bencher or run bencher --help.

DESCRIPTION

Packaging a benchmark script as a Bencher scenario makes it convenient to include/exclude/add participants/datasets (either via CLI or Perl code), send the result to a central repository, among others . See Bencher and bencher (CLI) for more details.

BENCHMARKED MODULES

Version numbers shown below are the versions used when running the sample benchmark.

Array::AsObject 1.02

Array::Set 0.05

Array::Utils 0.5

List::MoreUtils 0.416

List::MoreUtils::PP 0.416

Set::Array 0.30

Set::Object 1.35

Set::Scalar 1.29

Set::Tiny 0.04

BENCHMARK PARTICIPANTS

  • Array::Utils::unique (perl_code) [op:union]

    Code template:

    &Array::Utils::unique(<set1>, <set2>)
  • Set::Scalar::union (perl_code) [op:union]

    Code template:

    my $s1 = Set::Scalar->new(@{<set1>}); my $s2 = Set::Scalar->new(@{<set2>}); $s1->union($s2)
  • List::MoreUtils::PP::uniq (perl_code) [op:union]

    Function call template:

    List::MoreUtils::PP::uniq(@{<set1>}, @{<set2>})
  • List::MoreUtils::XS::uniq (perl_code) [op:union]

    Function call template:

    List::MoreUtils::uniq(@{<set1>}, @{<set2>})
  • Array::Set::set_union (perl_code) [op:union]

    Function call template:

    Array::Set::set_union(<set1>, <set2>)
  • Set::Array::union (perl_code) [op:union]

    Code template:

    my $s1 = Set::Array->new(@{<set1>}); my $s2 = Set::Array->new(@{<set2>}); $s1->union($s2)
  • Array::AsObject::union (perl_code) [op:union]

    Code template:

    my $s1 = Array::AsObject->new(@{<set1>}); my $s2 = Array::AsObject->new(@{<set2>}); $s1->union($s2, 1)
  • Set::Object::union (perl_code) [op:union]

    Code template:

    my $s1 = Set::Object->new(@{<set1>}); my $s2 = Set::Object->new(@{<set2>}); $s1->union($s2)
  • Set::Tiny::union (perl_code) [op:union]

    Code template:

    my $s1 = Set::Tiny->new(@{<set1>}); my $s2 = Set::Tiny->new(@{<set2>}); $s1->union($s2)
  • Array::Utils::array_diff (perl_code) [op:symdiff]

    Code template:

    &Array::Utils::array_diff(<set1>, <set2>)
  • Set::Scalar::symmetric_difference (perl_code) [op:symdiff]

    Code template:

    my $s1 = Set::Scalar->new(@{<set1>}); my $s2 = Set::Scalar->new(@{<set2>}); $s1->symmetric_difference($s2)
  • List::MoreUtils::PP::singleton (perl_code) [op:symdiff]

    Function call template:

    List::MoreUtils::PP::singleton(@{<set1>}, @{<set2>})
  • List::MoreUtils::XS::singleton (perl_code) [op:symdiff]

    Function call template:

    List::MoreUtils::singleton(@{<set1>}, @{<set2>})
  • Array::Set::set_symdiff (perl_code) [op:symdiff]

    Function call template:

    Array::Set::set_symdiff(<set1>, <set2>)
  • Set::Array::symmetric_difference (perl_code) [op:symdiff]

    Code template:

    my $s1 = Set::Array->new(@{<set1>}); my $s2 = Set::Array->new(@{<set2>}); $s1->symmetric_difference($s2)
  • Array::AsObject::symmetric_difference (perl_code) [op:symdiff]

    Code template:

    my $s1 = Array::AsObject->new(@{<set1>}); my $s2 = Array::AsObject->new(@{<set2>}); $s1->symmetric_difference($s2)
  • Set::Object::symmetric_difference (perl_code) [op:symdiff]

    Code template:

    my $s1 = Set::Object->new(@{<set1>}); my $s2 = Set::Object->new(@{<set2>}); $s1->symmetric_difference($s2)
  • Set::Tiny::symmetric_difference (perl_code) [op:symdiff]

    Code template:

    my $s1 = Set::Tiny->new(@{<set1>}); my $s2 = Set::Tiny->new(@{<set2>}); $s1->symmetric_difference($s2)
  • Array::Utils::array_minus (perl_code) [op:diff]

    Code template:

    &Array::Utils::array_minus(<set1>, <set2>)
  • Set::Scalar::difference (perl_code) [op:diff]

    Code template:

    my $s1 = Set::Scalar->new(@{<set1>}); my $s2 = Set::Scalar->new(@{<set2>}); $s1->difference($s2)
  • Array::Set::set_diff (perl_code) [op:diff]

    Function call template:

    Array::Set::set_diff(<set1>, <set2>)
  • Set::Array::difference (perl_code) [op:diff]

    Code template:

    my $s1 = Set::Array->new(@{<set1>}); my $s2 = Set::Array->new(@{<set2>}); $s1->difference($s2)
  • Array::AsObject::difference (perl_code) [op:diff]

    Code template:

    my $s1 = Array::AsObject->new(@{<set1>}); my $s2 = Array::AsObject->new(@{<set2>}); $s1->difference($s2)
  • Set::Object::difference (perl_code) [op:diff]

    Code template:

    my $s1 = Set::Object->new(@{<set1>}); my $s2 = Set::Object->new(@{<set2>}); $s1->difference($s2)
  • Set::Tiny::difference (perl_code) [op:diff]

    Code template:

    my $s1 = Set::Tiny->new(@{<set1>}); my $s2 = Set::Tiny->new(@{<set2>}); $s1->difference($s2)
  • Array::Utils::intersect (perl_code) [op:intersect]

    Code template:

    &Array::Utils::intersect(<set1>, <set2>)
  • Set::Scalar::intersection (perl_code) [op:intersect]

    Code template:

    my $s1 = Set::Scalar->new(@{<set1>}); my $s2 = Set::Scalar->new(@{<set2>}); $s1->intersection($s2)
  • Array::Set::set_intersect (perl_code) [op:intersect]

    Function call template:

    Array::Set::set_intersect(<set1>, <set2>)
  • Set::Array::intersection (perl_code) [op:intersect]

    Code template:

    my $s1 = Set::Array->new(@{<set1>}); my $s2 = Set::Array->new(@{<set2>}); $s1->intersection($s2)
  • Array::AsObject::intersection (perl_code) [op:intersect]

    Code template:

    my $s1 = Array::AsObject->new(@{<set1>}); my $s2 = Array::AsObject->new(@{<set2>}); $s1->intersection($s2, 1)
  • Set::Object::intersection (perl_code) [op:intersect]

    Code template:

    my $s1 = Set::Object->new(@{<set1>}); my $s2 = Set::Object->new(@{<set2>}); $s1->intersection($s2)
  • Set::Tiny::intersection (perl_code) [op:intersect]

    Code template:

    my $s1 = Set::Tiny->new(@{<set1>}); my $s2 = Set::Tiny->new(@{<set2>}); $s1->intersection($s2)

BENCHMARK DATASETS

  • num10

  • num100

SAMPLE BENCHMARK RESULTS

Run on: perl: v5.24.0, CPU: Intel(R) Core(TM) M-5Y71 CPU @ 1.20GHz (2 cores), OS: GNU/Linux LinuxMint version 17.3, OS kernel: Linux version 3.19.0-32-generic.

Benchmark with default options (bencher -m SetOperationModules):

#table1#
+---------------------------------------+---------+-----------+------------+------------+---------+---------+
| participant                           | dataset | rate (/s) | time (ms)  | vs_slowest |  errors | samples |
+---------------------------------------+---------+-----------+------------+------------+---------+---------+
| Array::AsObject::symmetric_difference | num100  |      79   | 13         |     1      | 9.1e-05 |      20 |
| Array::AsObject::intersection         | num100  |     160   |  6.2       |     2      | 6.1e-05 |      20 |
| Array::AsObject::difference           | num100  |     330   |  3.1       |     4.1    | 4.3e-06 |      22 |
| Set::Array::intersection              | num100  |     440   |  2.3       |     5.6    |   7e-06 |      36 |
| Set::Scalar::intersection             | num100  |    1410   |  0.711     |    17.9    | 4.3e-07 |      20 |
| Set::Scalar::union                    | num100  |    1500   |  0.68      |    19      | 9.1e-07 |      20 |
| Set::Scalar::difference               | num100  |    1550   |  0.644     |    19.7    | 2.1e-07 |      20 |
| Set::Scalar::symmetric_difference     | num100  |    2120   |  0.472     |    26.9    | 4.3e-07 |      20 |
| Array::AsObject::symmetric_difference | num10   |    4000   |  0.3       |    50      | 2.9e-06 |      23 |
| Set::Object::symmetric_difference     | num100  |    7000   |  0.14      |    89      | 2.3e-07 |      28 |
| Array::AsObject::intersection         | num10   |    7666   |  0.130446  |    97.3485 | 3.4e-11 |      22 |
| Set::Scalar::intersection             | num10   |    7700   |  0.13      |    98      | 2.2e-07 |      29 |
| Array::AsObject::union                | num100  |    7900   |  0.13      |   100      | 2.3e-07 |      28 |
| Set::Scalar::union                    | num10   |    8000   |  0.12      |   100      | 1.9e-07 |      25 |
| Set::Object::intersection             | num100  |    8100   |  0.12      |   100      | 1.8e-07 |      28 |
| Set::Scalar::difference               | num10   |    8200   |  0.12      |   100      | 2.3e-07 |      27 |
| Set::Array::symmetric_difference      | num100  |    9000   |  0.11      |   110      | 4.3e-07 |      20 |
| Set::Array::difference                | num100  |   10000   |  0.097     |   130      | 1.3e-07 |      20 |
| Set::Object::union                    | num100  |   10000   |  0.095     |   130      | 1.1e-07 |      20 |
| Set::Tiny::symmetric_difference       | num100  |   12000   |  0.086     |   150      |   1e-07 |      22 |
| Set::Object::difference               | num100  |   12000   |  0.082     |   150      | 1.2e-07 |      24 |
| Set::Array::union                     | num100  |   12000   |  0.08      |   160      | 9.7e-08 |      24 |
| Set::Tiny::difference                 | num100  |   13000   |  0.079     |   160      | 1.1e-07 |      20 |
| Set::Tiny::union                      | num100  |   13000   |  0.079     |   160      | 1.9e-07 |      31 |
| Set::Scalar::symmetric_difference     | num10   |   13000   |  0.0769    |   165      | 2.7e-08 |      20 |
| Set::Tiny::intersection               | num100  |   15000   |  0.068     |   190      |   8e-08 |      20 |
| Array::Set::set_symdiff               | num100  |   14800   |  0.0676    |   188      | 2.7e-08 |      20 |
| Array::Set::set_intersect             | num100  |   15551.5 |  0.0643027 |   197.484  | 9.8e-12 |      22 |
| Array::Set::set_union                 | num100  |   17300   |  0.0578    |   220      | 2.7e-08 |      20 |
| Array::AsObject::difference           | num10   |   18000   |  0.0555    |   229      | 2.7e-08 |      20 |
| List::MoreUtils::PP::singleton        | num100  |   18000   |  0.055     |   230      | 1.1e-07 |      20 |
| Array::Utils::array_diff              | num100  |   19000   |  0.0526    |   241      | 2.5e-08 |      23 |
| Array::Set::set_diff                  | num100  |   21000   |  0.047     |   270      | 6.4e-08 |      22 |
| Array::Utils::array_minus             | num100  |   21800   |  0.0459    |   276      | 1.2e-08 |      24 |
| Array::Utils::intersect               | num100  |   21884.2 |  0.0456952 |   277.901  | 3.4e-11 |      20 |
| List::MoreUtils::PP::uniq             | num100  |   24000   |  0.042     |   300      | 5.3e-08 |      20 |
| Set::Array::intersection              | num10   |   25605   |  0.039055  |   325.15   | 1.8e-10 |      20 |
| List::MoreUtils::XS::singleton        | num100  |   30088.2 |  0.0332356 |   382.083  |   0     |      20 |
| List::MoreUtils::XS::uniq             | num100  |   44000   |  0.023     |   560      | 2.7e-08 |      20 |
| Set::Object::symmetric_difference     | num10   |   45000   |  0.022     |   570      | 2.7e-08 |      20 |
| Set::Array::difference                | num10   |   47000   |  0.021     |   590      | 5.2e-08 |      27 |
| Set::Array::symmetric_difference      | num10   |   49000   |  0.02      |   630      | 2.7e-08 |      30 |
| Array::AsObject::union                | num10   |   53498   |  0.018692  |   679.36   | 4.8e-11 |      29 |
| Set::Object::intersection             | num10   |   66400   |  0.0151    |   843      | 6.7e-09 |      20 |
| Set::Array::union                     | num10   |   66600   |  0.015     |   846      | 6.7e-09 |      20 |
| Set::Object::union                    | num10   |   80000   |  0.013     |  1000      | 1.3e-08 |      20 |
| Set::Object::difference               | num10   |   92700   |  0.0108    |  1180      | 3.3e-09 |      20 |
| Set::Tiny::symmetric_difference       | num10   |   97000   |  0.01      |  1200      | 1.3e-08 |      21 |
| Set::Tiny::difference                 | num10   |  110000   |  0.0091    |  1400      | 2.6e-08 |      26 |
| Set::Tiny::union                      | num10   |  110000   |  0.0091    |  1400      |   1e-08 |      20 |
| Array::Set::set_symdiff               | num10   |  127000   |  0.0079    |  1610      | 2.8e-09 |      29 |
| Set::Tiny::intersection               | num10   |  130000   |  0.00767   |  1660      | 3.2e-09 |      22 |
| Array::Set::set_intersect             | num10   |  133000   |  0.0075    |  1690      | 3.3e-09 |      20 |
| Array::Set::set_union                 | num10   |  140000   |  0.0069    |  1800      | 8.9e-09 |      25 |
| Array::Utils::array_diff              | num10   |  156400   |  0.0063939 |  1986.1    | 3.4e-11 |      36 |
| List::MoreUtils::PP::singleton        | num10   |  170000   |  0.0061    |  2100      | 6.7e-09 |      20 |
| Array::Set::set_diff                  | num10   |  167000   |  0.00598   |  2120      |   5e-09 |      20 |
| Array::Utils::unique                  | num100  |  170000   |  0.0057    |  2200      | 6.7e-09 |      20 |
| Array::Utils::intersect               | num10   |  192000   |  0.00521   |  2440      | 1.7e-09 |      20 |
| Array::Utils::array_minus             | num10   |  192000   |  0.0052    |  2440      | 1.3e-09 |      32 |
| List::MoreUtils::PP::uniq             | num10   |  214000   |  0.00468   |  2710      | 1.7e-09 |      20 |
| List::MoreUtils::XS::singleton        | num10   |  303000   |  0.0033    |  3840      | 1.6e-09 |      22 |
| List::MoreUtils::XS::uniq             | num10   |  380000   |  0.0027    |  4800      | 6.7e-09 |      20 |
| Array::Utils::unique                  | num10   |  585600   |  0.001708  |  7436      | 9.5e-11 |      32 |
+---------------------------------------+---------+-----------+------------+------------+---------+---------+

Benchmark module startup overhead (bencher -m SetOperationModules --module-startup):

#table2#
+---------------------+------------------------------+--------------------+----------------+-----------+------------------------+------------+---------+---------+
| participant         | proc_private_dirty_size (MB) | proc_rss_size (MB) | proc_size (MB) | time (ms) | mod_overhead_time (ms) | vs_slowest |  errors | samples |
+---------------------+------------------------------+--------------------+----------------+-----------+------------------------+------------+---------+---------+
| Set::Array          | 3.1                          | 6.4                | 22             |      21   |                   15   |        1   | 5.7e-05 |      21 |
| Array::AsObject     | 2.2                          | 5.6                | 19             |      21   |                   15   |        1   | 8.1e-05 |      20 |
| Set::Object         | 1                            | 4.4                | 16             |      17   |                   11   |        1.2 | 9.2e-05 |      20 |
| Set::Scalar         | 1.2                          | 4.5                | 16             |      16   |                   10   |        1.3 | 5.4e-05 |      20 |
| List::MoreUtils     | 1.1                          | 4.5                | 16             |      13   |                    7   |        1.6 | 9.2e-05 |      20 |
| List::MoreUtils::PP | 1.6                          | 5                  | 19             |       9.9 |                    3.9 |        2.1 | 6.1e-05 |      20 |
| Array::Set          | 2.7                          | 6                  | 24             |       9   |                    3   |        2.3 | 4.6e-05 |      20 |
| Set::Tiny           | 0.93                         | 4.3                | 16             |       7.1 |                    1.1 |        2.9 | 2.3e-05 |      21 |
| Array::Utils        | 1.9                          | 5.4                | 19             |       7   |                    1   |        3   | 4.9e-05 |      21 |
| perl -e1 (baseline) | 0.9                          | 4                  | 20             |       6   |                    0   |        4   | 8.5e-05 |      20 |
+---------------------+------------------------------+--------------------+----------------+-----------+------------------------+------------+---------+---------+

To display as an interactive HTML table on a browser, you can add option --format html+datatables.

HOMEPAGE

Please visit the project's homepage at https://metacpan.org/release/Bencher-Scenario-SetOperationModules.

SOURCE

Source repository is at https://github.com/perlancar/perl-Bencher-Scenario-SetOperationModules.

BUGS

Please report any bugs or feature requests on the bugtracker website https://rt.cpan.org/Public/Dist/Display.html?Name=Bencher-Scenario-SetOperationModules

When submitting a bug or request, please include a test-file or a patch to an existing test-file that illustrates the bug or desired feature.

SEE ALSO

Benchmark::Featureset::SetOps

Excluded modules: Set::Bag (expects hashes instead of arrays), Set::SortedArray (members are sorted).

AUTHOR

perlancar <perlancar@cpan.org>

COPYRIGHT AND LICENSE

This software is copyright (c) 2017 by perlancar@cpan.org.

This is free software; you can redistribute it and/or modify it under the same terms as the Perl 5 programming language system itself.