/ 
POE-Component-Curl-Multi-0.18
River stage one • 1 direct dependent • 1 total dependent
/ POE::Component::Curl::Multi

NAME

POE::Component::Curl::Multi - a fast HTTP POE component

VERSION

version 0.18

SYNOPSIS

use strict;
use warnings;
use HTTP::Request::Common qw[GET];
use POE qw[Component::Curl::Multi];

$!=1;

my @urls = ( 'https://api.github.com/repos/git/git/tags',
             'http://this.is.made.up.stuff/',
             'http://www.cpan.org/',
             'http://www.google.com/', );

my $curl = POE::Component::Curl::Multi->spawn(
  Alias => 'curl',
  FollowRedirects => 5,
  Max_Concurrency => 10,
);

POE::Session->create(
  package_states => [
    main => [qw(_start _response)],
  ],
);

$poe_kernel->run();
exit 0;

sub _start {
  $poe_kernel->post( 'curl', 'request', '_response', GET($_) ) for @urls;
  return;
}

sub _response {
  my ($request_packet, $response_packet) = @_[ARG0, ARG1];
  use Data::Dumper;
  local $Data::Dumper::Indent=1;
  warn Dumper( $response_packet->[0] );
  return;
}

DESCRIPTION

POE::Component::Curl::Multi is an HTTP user-agent for POE. It lets other sessions run while HTTP transactions are being processed, and it lets several HTTP transactions be processed in parallel.

It uses Net::Curl internally to provide access to libcurl for fast performance. It strives to be API compatible(ish) with POE::Component::Client::HTTP.

It is inspired by AnyEvent::Curl::Multi.

CONSTRUCTOR

spawn

Starts an instance of the component. Takes a number of options:

alias

Set an alias for this instance of the component. There is no default.

timeout

Specifies a timeout, in seconds, for each request, defaults to 180.

followredirects

Specifies how many redirects (e.g. 302 Moved) to follow. If not specified defaults to 0, and thus no redirection is followed.

agent

Can either be a string or an ARRAYREF of strings. This will be used as UserAgent header in requests. If an ARRAYREF is provided one of them will be picked randomly to send.

See http://curl.haxx.se/libcurl/c/curl_easy_setopt.html#CURLOPTUSERAGENT

proxy

Specify a proxy to use.

See http://curl.haxx.se/libcurl/c/curl_easy_setopt.html#CURLOPTPROXY

max_concurrency

Specify the maximum number of concurrent requests, a value of 0 means no limit will be imposed. The default is 0.

curl_debug

Enable libcurl's verbosity.

See http://curl.haxx.se/libcurl/c/curl_easy_setopt.html#CURLOPTVERBOSE

Returns an object that accepts some methods as documented below.

AVAILABLE METHODS

session_id

Takes no arguments. Returns the ID of the component's session.

shutdown

Responds to all pending requests with 408 (request timeout), and then shuts down the component and all subcomponents.

pending_requests_count

Returns the number of requests currently being processed.

cancel

Cancel a specific HTTP request. Requires a reference to the original request (blessed or stringified) so it knows which one to cancel.

ACCEPTED EVENTS

Sessions communicate asynchronously with the component. They post requests to it, and it posts responses back.

request

Requests are posted to the component's request state. They include an HTTP::Request object which defines the request. For example:

$kernel->post(
  'ua', 'request',            # http session alias & state
  'response',                 # my state to receive responses
  GET('http://poe.perl.org'), # a simple HTTP request
  'unique id',                # a tag to identify the request
  'progress',                 # an event to indicate progress
  'http://1.2.3.4:80/'        # proxy to use for this request
);

This invocation is compatible with POE::Component::Client::HTTP.

You may also send either an arrayref or hashref to the request state with the following parameters:

request

A HTTP::Request object which defines the request.

response

Either a string specifying the state in the sender session to receive responses or, alternatively, a POE::Session postback that will be invoked with responses.

tag

A tag to identify the request.

progress

An optional handler, if specified the component will provide progress metrics (see sample handler below).

proxy

Specify a proxy to use. This overrides the proxy set with spawn, if applicable.

See http://curl.haxx.se/libcurl/c/curl_easy_setopt.html#CURLOPTPROXY

session

Specify a POE::Session object, ID or alias to send responses to instead of the sending session. If a postback is used for response, this option will be ignored.

pending_requests_count

Returns the number of requests currently being processed. To receive the return value, it must be invoked with $kernel->call().

my $count = $kernel->call('ua' => 'pending_requests_count');

This is also available as a method on the object returned by spawn

my $count = $curl->pending_requests_count();

shutdown

Responds to all pending requests with 408 (request timeout), and then shuts down the component and all subcomponents.

SENT EVENTS

response handler

In addition to all the usual POE parameters, HTTP responses come with two list references:

my ($request_packet, $response_packet) = @_[ARG0, ARG1];

$request_packet contains a reference to the original HTTP::Request object. This is useful for matching responses back to the requests that generated them.

my $http_request_object = $request_packet->[0];
my $http_request_tag    = $request_packet->[1]; # from the 'request' post

$response_packet contains a reference to the resulting HTTP::Response object.

my $http_response_object = $response_packet->[0];

Please see the HTTP::Request and HTTP::Response manpages for more information.

progress handler

The example progress handler shows how to calculate a percentage of download completion.

sub progress_handler {
  my $gen_args  = $_[ARG0];    # args passed to all calls
  my $call_args = $_[ARG1];    # args specific to the call

  my $req = $gen_args->[0];    # HTTP::Request object being serviced
  my $tag = $gen_args->[1];    # Request ID tag from.
  my $got = $call_args->[0];   # Number of bytes retrieved so far.
  my $tot = $call_args->[1];   # Total bytes to be retrieved.

  my $percent = $got / $tot * 100;

  printf(
    "-- %.0f%% [%d/%d]: %s\n", $percent, $got, $tot, $req->uri()
  );

  return;
}

STREAMING

This component does not (yet) support POE::Component::Client::HTTP's streaming options.

CLIENT HEADERS

POE::Component::Curl::Multi sets its own response headers with additional information. All of its headers begin with "X-PCCH".

X-PCCH-Errmsg

POE::Component::Curl::Multi may fail because of an internal client error rather than an HTTP protocol error. X-PCCH-Errmsg will contain a human readable reason for client failures, should they occur.

The text of X-PCCH-Errmsg may also be repeated in the response's content.

X-PCCH-Peer

This response header is not yet supported.

ATTRIBUTION

POE::Component::Curl::Multi is based on both

POE::Component::Client::HTTP by Rocco Caputo, Rob Bloodgood and Martijn van Beers

and

AnyEvent::Curl::Multi by Michael S. Fischer

SEE ALSO

Net::Curl

AnyEvent::Curl::Multi

POE::Component::Client::HTTP

AUTHOR

Chris Williams

COPYRIGHT AND LICENSE

This software is copyright (c) 2020 by Chris Williams, Michael S. Fischer, Rocco Caputo, Rob Bloodgood and Martijn van Beers.

This is free software; you can redistribute it and/or modify it under the same terms as the Perl 5 programming language system itself.