/ 
Sub-Throttler-0.1.0
River stage one • 1 direct dependent • 1 total dependent
/ Sub::Throttler

NAME

Sub::Throttler - Rate limit sync and async function calls

SYNOPSIS

# Load throttling engine
use Sub::Throttler qw( throttle_it );

# Enable throttling for existing sync/async functions/methods
throttle_it('Mojo::UserAgent::get');
throttle_it('Mojo::UserAgent::post');

# Load throttling algorithms
use Sub::Throttler::Limit;
use Sub::Throttler::Rate::EV;

# Configure throttling algorithms
my $throttle_parallel_requests
    = Sub::Throttler::Limit->new(limit => 5);
my $throttle_request_rate
    = Sub::Throttler::Rate::EV->new(period => 0.1, limit => 10);

# Apply configured limits to selected functions/methods with
# throttling support
$throttle_parallel_requests->apply_to_methods(Mojo::UserAgent => qw( get ));
$throttle_request_rate->apply_to_methods(Mojo::UserAgent => qw( get post ));

DESCRIPTION

This module provide sophisticated throttling framework which let you delay execution of any sync and async functions/methods based on any rules you need.

You can use core features of this framework with usual sync application or with application based on any event loop, but some throttling algorithms may require specific event loop (like Sub::Throttler::Rate::EV).

The "SYNOPSIS" shows basic usage example, but there are a lot of advanced features: define which and how many resources each function/method use depending not only on it name but also on it params, normal and high-priority queues for throttled functions/methods, custom wrappers with ability to free unused limits, write your own functions/methods with smart support for throttling, implement own throttling algorithms, save and restore current limits between different executions of your app.

Basic use case: limit rate for downloading urls. Advanced use case: apply complex limits for using remote API, which depends on how many items you process (i.e. on some params of API call), use high-priority queue to ensure login/re-login API call will be executed ASAP before any other delayed API calls, cancel unused limits in case of failed API call to avoid needless delay for next API calls, save/restore used limits to avoid occasional exceeding the quota because of crash/restart of your app.

ALGORITHMS / PLUGINS

Sub::Throttler::Limit implement algorithm to throttle based on quantity of used resources/limits. For example, it will let you limit an amount of simultaneous tasks. Also it's good base class for your own algorithms.

Sub::Throttler::Rate::EV implement algorithm to throttle based on rate (quantity of used resources/limits per some period of time). For example, it will let you control maximum calls/sec and burst rate (by choosing between "1000 calls per 1 second" and "10 calls per 0.01 second" limits).

HOW THROTTLING WORKS

To be able to throttle some function/method it should support throttling. This mean it either should be implemented using this module (see "throttle_me" and "throttle_me_asap"), or you should replace original function/method with special wrapper. Simple (but limited) way to do this is use "throttle_it" and "throttle_it_asap" helpers. If simple way doesn't work for you then you can implement "custom wrapper" yourself.

Next, not all functions/methods with throttling support will be actually throttled - you should configure which of them and how exactly should be throttled by using throttling algorithm objects.

Then, when some function/method with throttling support is called, it will try to acquire some resources (see below) - on success it will be immediately executed, otherwise it execution will be delayed until these resources will be available. When it's finished it should explicitly free used resources (if it was unable to do it work - for example because of network error - it may cancel resources as unused) - to let some other (delayed until these resources will be available) function/method runs.

HOW TO CONTROL LIMITS / RESOURCES

When you configure throttling for some function/method you define which "resources" and how much of them it needs to run. The "resource" is just any string, and in simple case all throttled functions/methods will use same string (say, "default") and same quantity of this "resource": 1.

# this algorithm allow using up to 5 "resources" of same name
$throttle = Sub::Throttler::Limit->new(limit => 5);

# this is same:
$throttle->apply_to_functions('Package::func');
# as this:
$throttle->apply_to(sub {
    my ($this, $name, @params) = @_;
    if (!$this && $name eq 'Package::func') {
        return 'default', 1;    # require 1 resource named "default"
    }
    return;                     # do not throttle other functions/methods
});

But in complex cases you can (based on name of function or class name or exact object and method name, and their parameters) define several "resources" with different quantities of each.

$throttle->apply_to(sub {
    my ($this, $name, @params) = @_;
    if (ref $this && $this eq $target_object && $name eq 'method') {
        # require 2 "some" and 10 "other" resources
        return ['some','other'], [2,10];
    }
    return;                     # do not throttle other functions/methods
});

It's allowed to "apply" same $throttle instance to same functions/methods more than once if you won't require same resource name more than once for same function/method.

How exactly these "resources" will be acquired and released depends on used algorithm.

PRIORITY / QUEUES

There are two separate queues for delayed functions/methods: normal and high-priority "asap" queue. Functions/methods in "asap" queue will be executed before any (even delayed before them) function/method in normal queue which require same resources to run. But if there are not enough resources to run function/method from high-priority queue and enough to run from normal queue - function/method from normal queue will be run.

Which function/method will use normal and which "asap" queue is defined by that function/method (or it wrapper) implementation.

Delayed methods in queue use weak references to their objects, so these objects doesn't kept alive only because of these delayed method calls. If these objects will be destroyed then their delayed methods will be silently removed from queue.

EXPORTS

Nothing by default, but all documented functions can be explicitly imported.

Use tag :ALL to import all of them.

If you developing plugin for this module you can use tag :plugin to import throttle_add, throttle_del and throttle_flush.

INTERFACE

Enable throttling for existing functions/methods

throttle_it
my $orig_func = throttle_it('func');
my $orig_func = throttle_it('Some::func2');

This helper is able to replace with wrapper either sync function/method or async function/method which receive callback in last parameter.

That wrapper will call $done->() (release used resources, see "throttle_me" for details about it) after sync function/method returns or just before callback of async function/method will be called.

If given function name without package it will look for that function in caller's package.

Return reference to original function or throws if given function is not exists.

throttle_it_asap
my $orig_func = throttle_it_asap('func');
my $orig_func = throttle_it_asap('Some::func2');

Same as "throttle_it" but use high-priority "asap" queue.

custom wrapper

If you want to call $done->() after async function/method callback or want to cancel unused resources in some cases by calling $done->(0) you should implement custom wrapper instead of using "throttle_it" or "throttle_it_asap" helpers.

Throttling anonymous function is not supported, that's why you need to use string eval instead of *Some::func = sub { 'wrapper' }; here.

# Example wrapper for sync function which called in scalar context and
# return false when it failed to do it work (we want to cancel
# unused resources in this case).
my $orig_func = \&Some::func;
eval <<'EOW';
no warnings 'redefine';
sub Some::func {
    my $done = &throttle_me || return;
    my $result = $orig_func->(@_);
    if ($result) {
        $done->();
    } else {
        $done->(0);
    }
    return $result;
}
EOW

# Example wrapper for async method which receive callback in first
# parameter and call it with error message when it failed to do it
# work (we want to cancel unused resources in this case); we also want
# to call $done->() after callback and use "asap" queue.
my $orig_method = \&Class::method;
eval <<'EOW';
no warnings 'redefine';
sub Class::method {
    my $done = &throttle_me_asap || return;
    my $self = shift;
    my $orig_cb = shift;
    my $cb = sub {
        my ($error) = @_;
        $orig_cb->(@_);
        if ($error) {
            $done->(0);
        } else {
            $done->();
        }
    };
    $self->$orig_method($cb, @_);
}
EOW

Writing functions/methods with support for throttling

To have maximum control over some function/method throttling you should write that function yourself (in some cases it's enough to write "custom wrapper" for existing function/method). This will let you control when exactly it should release used resources or cancel unused resources to let next delayed function/method run as soon as possible.

throttle_me
sub func {
    my $done = &throttle_me || return;
    my (@params) = @_;
    if ('unable to do the work') {
        $done->(0);
        return;
    }
    ...
    $done->();
}
sub method {
    my $done = &throttle_me || return;
    my ($self, @params) = @_;
    if ('unable to do the work') {
        $done->(0);
        return;
    }
    ...
    $done->();
}

You should use it exactly as it shown in these examples: it should be called using form &throttle_me because it needs to modify your function/method's @_.

If your function/method should be delayed because of throttling it will return false, and you should interrupt your function/method. Otherwise it'll acquire "resources" needed to run your function/method and return callback which you should call later to release these resources.

If your function/method has done it work (and thus "used" these resources) $done should be called without parameters $done->() or with one true param $done->(1); if it hasn't done it work (and thus "not used" these resources) it's better to call it with one false param $done->(0) to cancel these unused resources and give a chance for another function/method to reuse them.

If you forget to call $done - you'll get a warning (and chances are you'll soon run out of resources because of this and new throttled functions/methods won't be run anymore), if you call it more than once - exception will be thrown.

Anonymous functions are not supported.

throttle_me_asap

Same as "throttle_me" except use &throttle_me_asap:

my $done = &throttle_me_asap || return;

This will make this function/method use high-priority "asap" queue instead of normal queue.

done_cb
my $cb = done_cb($done, sub {
    my (@params) = @_;
    ...
});

my $cb = done_cb($done, sub {
    my ($extra1, $extra2, @params) = @_;
    ...
}, $extra1, $extra2);

my $cb = done_cb($done, $object, 'method');
sub Class::Of::That::Object::method {
    my ($self, @params) = @_;
    ...
}

my $cb = done_cb($done, $object, 'method', $extra1, $extra2);
sub Class::Of::That::Object::method {
    my ($self, $extra1, $extra2, @params) = @_;
    ...
}

This is a simple helper function used to make sure you won't forget to call $done->() in your async function/method with throttling support.

First parameter must be $done callback, then either callback function or object and name of it method, and then optionally any extra params for that callback function/object's method.

Returns callback, which when called will first call $done->() and then given callback function or object's method with any extra params (if any) followed by it own params.

Example:

# use this:
sub download {
    my $done = &throttle_me || return;
    my ($url) = @_;
    $ua->get($url, done_cb($done, sub {
        my ($ua, $tx) = @_;
        ...
    }));
}
# instead of this:
sub download {
    my $done = &throttle_me || return;
    my ($url) = @_;
    $ua->get($url, sub {
        my ($ua, $tx) = @_;
        $done->();
        ...
    });
}

Implementing throttle algorithms/plugins

It's recommended to inherit your algorithm from Sub::Throttler::Limit.

Each plugin must provide these methods (they'll be called by throttling engine):

sub acquire {
    my ($self, $id, $key, $quantity) = @_;
    # try to acquire $quantity of resources named $key for
    # function/method identified by $id
    if ('failed to acquire') {
        return;
    }
    return 1;   # resource acquired
}
sub release {
    my ($self, $id) = @_;
    # release resources previously acquired for $id
    if ('some resources was freed') {
        throttle_flush();
    }
}
sub release_unused {
    my ($self, $id) = @_;
    # cancel unused resources previously acquired for $id
    if ('some resources was freed') {
        throttle_flush();
    }
}

While trying to find out is there are enough resources to run some delayed function/method throttling engine may call release_unused() immediately after successful acquire() - if it turns out some other resource needed for same function/method isn't available.

throttle_add
throttle_add($throttle_plugin, sub {
    my ($this, $name, @params) = @_;
    # $this is undef or a class name or an object
    # $name is a function or method name
    # @params is function/method params
    ...
    return undef;                   # OR
    return $key;                    # OR
    return ($key,$quantity);        # OR
    return \@keys;                  # OR
    return (\@keys,\@quantities);
});

This function usually used to implement helper methods in algorithm like "apply_to" in Sub::Throttler::Limit, "apply_to_functions" in Sub::Throttler::Limit, "apply_to_methods" in Sub::Throttler::Limit. But if algorithm doesn't implement such helpers it may be used directly by user to apply some algorithm instance to selected functions/methods.

throttle_del
throttle_del();
throttle_del($throttle_plugin);

Undo previous "throttle_add" calls with $throttle_plugin in first param or all of them if given no param. This is rarely useful, usually you setup throttling when your app initializes and then doesn't change it.

throttle_flush
throttle_flush();

Algorithm must call it each time quantity of some resources increases (so there is a chance one of delayed functions/methods can be run now).

BUGS AND LIMITATIONS

No bugs have been reported.

Throttling anonymous functions is not supported.

SUPPORT

Please report any bugs or feature requests through the web interface at http://rt.cpan.org/NoAuth/ReportBug.html?Queue=Sub-Throttler. I will be notified, and then you'll automatically be notified of progress on your bug as I make changes.

You can also look for information at:

AUTHOR

Alex Efros <powerman@cpan.org>

LICENSE AND COPYRIGHT

Copyright 2014 Alex Efros <powerman@cpan.org>.

This program is distributed under the MIT (X11) License: http://www.opensource.org/licenses/mit-license.php

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.