The Perl Toolchain Summit 2025 Needs You: You can help 🙏 Learn more

 / 
LWP-Protocol-Net-Curl-0.027
River stage one • 1 direct dependent • 2 total dependents
/ LWP::Protocol::Net::Curl

NAME

LWP::Protocol::Net::Curl - the power of libcurl in the palm of your hands!

VERSION

version 0.027

SYNOPSIS


            

              
              
#!/usr/bin/env perl;
use common::sense;
use LWP::Protocol::Net::Curl;
use WWW::Mechanize;
...

DESCRIPTION

Drop-in replacement for LWP, WWW::Mechanize and their derivatives to use Net::Curl as a backend.

Advantages:

  • support ftp/ftps/http/https/sftp/scp protocols out-of-box (secure layer require libcurl to be compiled with TLS/SSL/libssh2 support)

  • support SOCKS4/5 proxy out-of-box

  • connection persistence and DNS cache (independent from LWP::ConnCache)

  • lightning-fast HTTP compression and redirection

  • lower CPU usage: this matters if you fork() multiple downloader instances

  • asynchronous threading via Coro::Select (see eg/async.pl)

  • at last but not least: 100% compatible with both LWP and WWW::Mechanize test suites!

LIBCURL INTERFACE

You may query which LWP protocols are implemented through Net::Curl by accessing @LWP::Protocol::Net::Curl::implements or %LWP::Protocol::Net::Curl::implements.

By default, every protocol listed in that array will be implemented via LWP::Protocol::Net::Curl. It is possible to import only specific protocols:


            

              
              
use LWP::Protocol::Net::Curl takeover => 0;
LWP::Protocol::implementor(https => 'LWP::Protocol::Net::Curl');

The default value of takeover option is true, resulting in exactly the same behavior as in:


            

              
              
use LWP::Protocol::Net::Curl takeover => 0;
LWP::Protocol::implementor($_ => 'LWP::Protocol::Net::Curl')
    for @LWP::Protocol::Net::Curl::implements;

Default curl_easy_setopt() options can be set during initialization:


            

              
              
use LWP::Protocol::Net::Curl
    encoding    => '',  # use HTTP compression by default
    referer     => 'http://google.com/',
    verbose     => 1;   # make libcurl print lots of stuff to STDERR

Or during runtime, using special HTTP headers (prefixed by X-CurlOpt-):


            

              
              
use LWP::Protocol::Net::Curl;
use LWP::UserAgent;
my $ua = LWP::UserAgent->new;
my $res = $ua->get(
    'https://metacpan.org/',
    X_CurlOpt_Verbose => 1,
);

Options set this way have the lowest precedence. For instance, if WWW::Mechanize sets the Referer: by it's own, the value you defined above won't be used.

DEBUGGING

Quickly enable libcurl verbose mode via PERL5OPT environment variable:


            

              
              
PERL5OPT=-MLWP::Protocol::Net::Curl=verbose,1 perl your-script.pl

Bonus: it works even if you don't include the use LWP::Protocol::Net::Curl line!

TODO

  • better implementation for non-HTTP protocols

  • more tests

  • expose the inner guts of libcurl while handling encoding/redirects internally

  • revise Net::Curl::Multi "event loop" code

BUGS

  • sometimes still complains about Attempt to free unreferenced scalar: SV 0xdeadbeef during global destruction.

  • in "async mode", each LWP::UserAgent instance "blocks" until all requests finish

  • parallel requests via Coro::Select are very inefficient; consider using YADA if you're into event-driven parallel user agents

  • Net::Curl::Share support is disabled on threaded Perl builds

SEE ALSO

AUTHOR

Stanislaw Pusep <stas@sysd.org>

COPYRIGHT AND LICENSE

This software is copyright (c) 2025 by Stanislaw Pusep.

This is free software; you can redistribute it and/or modify it under the same terms as the Perl 5 programming language system itself.

CONTRIBUTORS

  • José Joaquín Atria <jjatria@gmail.com>

  • Nick Kostyria <kostirya@gmail.com>

  • Peter Williams <pjwilliams@gmail.com>