NAME

App::dupfind::Threaded::MapReduce::Digest - Map-reduce version of digest_dups method and worker thread for it

VERSION

version 0.140230

DESCRIPTION

Overrides the digest_dups method from App::dupfind::Common and implements an worker thread routine that is invoked therein. In this threaded version of digest_dups, the set of same-size file groupings is mapped as a task and sent to the main map reducer logic engine implemented in App::dupfind::Threaded::MapReduce. The outcome of that multithreaded map-reduce operation is a list of true duplicates (or no duplicates if none were found).

Please don't use this module by itself. It is for internal use only.

METHODS

digest_dups

Calls the map-reduce logic on the $size_dups hashref, providing a wrapped coderef calling out to _digest_worker that will be invoked by the map-reduce engine for same-size size file groupings

This overrides the digest_dups method in App::dupfind::Common

_digest_woker

Does the calculating of digests in order to determine uniqueness of files in a same-size file grouping, storing information in a shared hashref and recording statistics as it progresses. A caching mechanism is used in order to avoid re-hashing of file content that has already been seen, yielding a very consequential performance gain.