NAME
WWW::Crawler::Mojo::Queue::Memory - Crawler queue with memory
SYNOPSIS
DESCRIPTION
Crawler queue with memory.
ATTRIBUTES
This class inherits all methods from WWW::Crawler::Mojo::Queue and implements following new ones.
cap
Capacity of queue, indecating how many jobs can be kept in queue at a time. If you enqueue over capacity, the oldest job will be automatically disposed.
jobs
jobs.
redundancy_storage
A hash ref in which the class keeps DONE flags for each jobs in order to avoid to perform resembling jobs multiple times.
# Mark a job as DONE
$queue->redundancy_storage->{$job->digest} = 1;
# Delete the mark
delete($queue->redundancy_storage->{$job->digest});METHODS
This class inherits all methods from WWW::Crawler::Mojo::Queue class and implements following new ones.
dequeue
Implementation for WWW::Crawler::Mojo::Queue interface.
enqueue
Implementation for WWW::Crawler::Mojo::Queue interface.
length
Implementation for WWW::Crawler::Mojo::Queue interface.
next
Implementation for WWW::Crawler::Mojo::Queue interface.
requeue
Implementation for WWW::Crawler::Mojo::Queue interface.
shuffle
Implementation for WWW::Crawler::Mojo::Queue interface.
AUTHOR
Keita Sugama, <sugama@jamadam.com>
COPYRIGHT AND LICENSE
Copyright (C) Keita Sugama.
This program is free software; you can redistribute it and/or modify it under the same terms as Perl itself.