/ 
App-BloomUtils-0.001
River stage one • 1 direct dependent • 1 total dependent
/ gen-bloom-filter

NAME

gen-bloom-filter - Generate bloom filter

VERSION

This document describes version 0.001 of gen-bloom-filter (from Perl distribution App-BloomUtils), released on 2018-03-22.

SYNOPSIS

Usage:

% gen-bloom-filter [options]

DESCRIPTION

You supply lines of text from STDIN and it will output the bloom filter bits on STDOUT. You can also customize num_bits (m) and num_hashes (k). Some rules of thumb to remember:

  • One byte per item in the input set gives about a 2% false positive rate. So if you expect two have 1024 elements, create a 1KB bloom filter with about 2% false positive rate. For other false positive rates:

    1% - 9.6 bits per item 0.1% - 14.4 bits per item 0.01% - 19.2 bits per item

  • Optimal number of hash functions is 0.7 times number of bits per item.

  • What is an acceptable false positive rate? This depends on your needs.

Ref: https://corte.si/posts/code/bloom-filter-rules-of-thumb/index.html

OPTIONS

* marks required options.

Main options

--num-bits=s, -m

Default value:

80000

The default is 80000 (generates a ~10KB bloom filter). If you supply 10,000 items (meaning 1 byte per 1 item) then the false positive rate will be ~2%. If you supply fewer items the false positive rate is smaller and if you supply more than 10,000 items the false positive rate will be higher.

--num-hashes=s, -k

Default value:

5.7

Output options

--format=s

Choose output format, e.g. json, text.

Default value:

undef
--json

Set output format to json.

--naked-res

When outputing as JSON, strip result envelope.

Default value:

0

By default, when outputing as JSON, the full enveloped result is returned, e.g.:

[200,"OK",[1,2,3],{"func.extra"=>4}]

The reason is so you can get the status (1st element), status message (2nd element) as well as result metadata/extra result (4th element) instead of just the result (3rd element). However, sometimes you want just the result, e.g. when you want to pipe the result for more post-processing. In this case you can use `--naked-res` so you just get:

[1,2,3]

Other options

--help, -h, -?

Display help message and exit.

--version, -v

Display program's version and exit.

COMPLETION

This script has shell tab completion capability with support for several shells.

bash

To activate bash completion for this script, put:

complete -C gen-bloom-filter gen-bloom-filter

in your bash startup (e.g. ~/.bashrc). Your next shell session will then recognize tab completion for the command. Or, you can also directly execute the line above in your shell to activate immediately.

It is recommended, however, that you install modules using cpanm-shcompgen which can activate shell completion for scripts immediately.

tcsh

To activate tcsh completion for this script, put:

complete gen-bloom-filter 'p/*/`gen-bloom-filter`/'

in your tcsh startup (e.g. ~/.tcshrc). Your next shell session will then recognize tab completion for the command. Or, you can also directly execute the line above in your shell to activate immediately.

It is also recommended to install shcompgen (see above).

other shells

For fish and zsh, install shcompgen as described above.

HOMEPAGE

Please visit the project's homepage at https://metacpan.org/release/App-BloomUtils.

SOURCE

Source repository is at https://github.com/perlancar/perl-App-BloomUtils.

BUGS

Please report any bugs or feature requests on the bugtracker website https://rt.cpan.org/Public/Dist/Display.html?Name=App-BloomUtils

When submitting a bug or request, please include a test-file or a patch to an existing test-file that illustrates the bug or desired feature.

SEE ALSO

bloom-filter-calculator.

AUTHOR

perlancar <perlancar@cpan.org>

COPYRIGHT AND LICENSE

This software is copyright (c) 2018 by perlancar@cpan.org.

This is free software; you can redistribute it and/or modify it under the same terms as the Perl 5 programming language system itself.