NAME

PPI::XS::Tokenizer - C++ replaction for the PPI Tokenizer

SYNOPSIS

use PPI::XS::Tokenizer;
use PPI;

DESCRIPTION

This is a C++ port for the PPI Tokenizer, aimed to make it faster. It build to be identical replacement, and as the are no user serviciable parts in the original tokenizer, there are none here either.

This package is compatible with PPI version 1.213

The effort was seeded be the Padre - the Perl IDE project, mentored by Adam Kennedy, the Perl-C++ binding was done by Steffen Mueller, and the tokenizer itself was written by Shmuel Fomberg.

Benchmark

Code:

timethis( 1000, sub { PPI::Document->new(".../lib/PPI/Node.pm") } );

Results:

with PPI::XS::Tokenizer
timethis 1000: 206 wallclock secs (203.29 usr +  0.67 sys = 203.96 CPU) @  4.90/s (n=1000)
Pure perl:
timethis 1000: 260 wallclock secs (257.66 usr +  0.28 sys = 257.94 CPU) @  3.88/s (n=1000)

SEE ALSO

AUTHOR

Shmuel Fomberg, <semuelf@cpan.org>

Steffen Mueller, <smueller@cpan.org>

COPYRIGHT AND LICENSE

Copyright (C) 2010 by Shmuel Fomberg

This library is free software; you can redistribute it and/or modify it under the same terms as Perl itself, either Perl version 5.8.0 or, at your option, any later version of Perl 5 you may have available.