NAME

Promises::Cookbook::Recursion - Examples of recursive asynchronous operations

VERSION

version 1.02

SYNOPSIS

package MyClass;

use Promises backend => ['AE'], 'deferred';

sub new                 {...}
sub process             {...}
sub is_finished         {...}
sub fetch_next_from_db  {...} # returns a promise

sub fetch_all {
    my $self     = shift;
    my $deferred = deferred;

    $self->_fetch_loop($deferred);
    return $deferred->promise;
}

sub _fetch_loop {
    my ($self,$deferred) = @_;
    if ( $self->is_finished ) {
        $deferred->resolve;
        return
    }
    $self->fetch_next_from_db
         ->then( sub { $self->process(@_) })
         ->done(
            sub { $self->_fetch_loop($deferred) }
            sub { $deferred->reject(@_) }
         );
}

package main;

my $cv  = AnyEvent->condvar;
my $obj = MyClass->new(...);
$obj->fetch_all->then(
    sub { $cv->send(@_)          },
    sub { $cv->croak('ERROR',@_) }
);

$cv->recv;

DESCRIPTION

While collect() allows you to wait for multiple promises which are executing in parallel, sometimes you need to execute each step in order, by using promises recursively. For instance:

1. Fetch next page of results

2. Process page of results

3. If there are no more results, return success

4. Otherwise, goto step 1

However, recursion can result in very deep stacks and out of memory conditions. There are two important steps for dealing with recursion effectively.

The first is to use one of the event-loop backends:

use Promises backend => ['AE'], 'deferred';

While the default Promises::Deferred implementation calls the then() callbacks synchronously, the event-loop backends call the callbacks asynchronously in the context of the event loop.

However, each promise passes its return value on to the next promise etc, so you still end up using a lot of memory with recursion. We can avoid this by breaking the chain.

In our example, all we care about is whether all the steps in our process completed successfully or not. Each execution of steps 1 to 4 is independent. Step 1 does not need to receive the return value from step 4.

We can break the chain by using done() instead of then(). While then() returns a new promise to continue the chain, done() will execute either the success callback or the error callback and return an empty list, breaking the chain and rolling back the stack.

To work through the code in the "SYNOPSIS":

sub fetch_all {
    my $self     = shift;
    my $deferred = deferred;

    $self->_fetch_loop($deferred);
    return $deferred->promise;
}

The $deferred variable (and the promise that we return to the caller) will either be resolved once all results have been fetched and processed by the _fetch_loop(), or rejected if an error occurs at any stage of execution.

sub _fetch_loop {
    my ($self,$deferred) = @_;

    if ( $self->is_finished ) {
        $deferred->resolve;
        return;
    }

If is_finished returns a true value (eg there are no more results to fetch), then we can resolve our promise, indicating success, and exit the loop.

    $self->fetch_next_from_db
         ->then( sub { $self->process(@_) })
         ->done(
            sub { $self->_fetch_loop($deferred) }
            sub { $deferred->reject(@_) }
         );
}

Otherwise we fetch the next page of results aynchronously from the DB and process them. If either of these steps (fetching or processing) fails, then we signal failure by rejecting our deferred promise and exiting the loop. If there is no failure, we recurse back into our loop by calling _fetch_loop() again.

However,this recursion happens asynchronously. What this code actually does is to schedule the call to _fetch_loop() in the next tick of the event loop. And because we used done() instead of then(), we don't wait around for the return result but instead return immediately, exiting the current execution, discarding the return results and rolling back the stack.

AUTHOR

Stevan Little <stevan.little@iinteractive.com>

COPYRIGHT AND LICENSE

This software is copyright (c) 2019, 2017, 2014, 2012 by Infinity Interactive, Inc..

This is free software; you can redistribute it and/or modify it under the same terms as the Perl 5 programming language system itself.