NAME

TAPx::Parser - Parse TAP output

VERSION

Version 0.40

SYNOPSIS

use TAPx::Parser;

my $parser = TAPx::Parser->new( { source => $source } );

while ( my $result = $parser->next ) {
    print $result->as_string;
}

DESCRIPTION

TAPx::Parser is designed to produce a proper parse of TAP output. It is ALPHA code and should be treated as such. The interface is now solid, but it is still subject to change.

For an example of how to run tests through this module, see the simple harnesses examples/.

METHODS

Class methods

`new`

my $parser = TAPx::Parser->new(\%args);

Returns a new TAPx::Parser object.

The arguments should be a hashref with one of the following keys:

source

This is the preferred method of passing arguments to the constructor. To determine how to handle the source, the following steps are taken.

If the source contains a newline, it's assumed to be a string of raw TAP output.

If the source is a reference, it's assumed to be something to pass to the TAPx::Parser::Iterator constructor. This is used internally and you should not use it.

Otherwise, the parser does a -e check to see if the source exists. If so, it attempts to execute the source and read the output as a stream. This is by far the preferred method of using the parser.
```
foreach my $file ( @test_files ) {
    my $parser = TAPx::Parser->new( { source => $file } );
    # do stuff with the parser
}
```
tap

The value should be the complete TAP output.
stream

The value should be a code ref. Every every time the reference is called, it should return a chunk of TAP. When no more tap is available, it should return undef.

The following keys are optional.

callback

If present, each callback corresponding to a given result type will be called with the result as the argument if the run method is used:

my %callbacks = (
    test    => \&test_callback,
    plan    => \&plan_callback,
    comment => \&comment_callback,
    bailout => \&bailout_callback,
    unknown => \&unknown_callback,
);

my $aggregator = TAPx::Parser::Aggregator->new;
foreach my $file ( @test_files ) {
    my $stream = TAPx::Parser::Source::Perl->new($file);
    my $parser = TAPx::Parser->new(
        {
            stream    => $stream,
            callbacks => \%callbacks,
        }
    );
    $parser->run;
    $aggregator->add( $file, $parser );
}

switches

If using a Perl file as a source, optional switches may be passed which will be used when invoking the perl executable.
```
my $parser = TAPx::Parser->new( {
    source   => $test_file,
    switches => '-Ilib',
} );
```

Instance methods

`next`

my $parser = TAPx::Parser->new( { stream => $stream } );
while ( my $result = $parser->next ) {
    print $result->as_string, "\n";
}

This method returns the results of the parsing, one result at a time. Note that it is destructive. You can't rewind and examine previous results.

Each result returned is a subclass of TAPx::Parser::Results. See that module and related classes for more information on how to use them.

`run`

$parser->run;

This method merely runs the parser and parses all of the TAP. If callbacks are used, it will attempt to call the appropriate callback with the TAP result as the argument.

INDIVIDUAL RESULTS

If you've read this far in the docs, you've seen this:

while ( my $result = $parser->next ) {
    print $result->as_string;
}

Each result returned is a TAPx::Parser::Result subclass, referred to as result types.

Result types

Basically, you fetch individual results from the TAP. The five types, with examples of each, are as follows:

Plan
```
1..42
```

Test

ok 3 - We should start with some foobar!

Comment
```
# Hope we don't use up the foobar.
```
Bailout
```
Bail out!  We ran out of foobar!
```
Unknown
```
... yo, this ain't TAP! ...
```

Each result fetched is a result object of a different type. There are common methods to each result object and different types may have methods unique to their type. Sometimes a type method may be overridden in a subclass, but its use is guaranteed to be identical.

Common type methods

`type`

Returns the type of result, such as comment or test.

`as_string`

Prints a string representation of the token. This might not be the exact output, however. Tests will have test numbers added if not present, TODO and SKIP directives will be capitalized and, in general, things will be cleaned up. If you need the original text for the token, see the raw method.

`raw`

Returns the original line of text which was parsed.

`is_plan`

Indicates whether or not this is the test plan line.

`is_test`

Indicates whether or not this is a test line.

`is_comment`

Indicates whether or not this is a comment.

`is_bailout`

Indicates whether or not this is bailout line.

`is_unknown`

Indicates whether or not the current line could be parsed.

`is_ok`

if ( $result->is_ok ) { ... }

Reports whether or not a given result has passed. Anything which is not a test result returns true. This is merely provided as a convenient shortcut which allows you to do this:

my $parser = TAPx::Parser->new( { source => $source } );
while ( my $result = $parser->next ) {
    # only print failing results
    print $result->as_string unless $result->is_ok;
}

`plan` methods

if ( $result->is_plan ) { ... }

If the above evaluates as true, the following methods will be available on the $result object.

`plan`

if ( $result->is_plan ) {
   print $result->plan;
}

This is merely a synonym for as_string.

`tests_planned`

my $planned = $result->tests_planned;

Returns the number of tests planned. For example, a plan of 1..17 will cause this method to return '17'.

`directive`

my $directive = $result->directive;

If a SKIP directive is included with the plan, this method will return it.

1..0 # SKIP: why bother?

`explanation`

my $explanation = $result->explanation;

If a SKIP directive was included with the plan, this method will return the explanation, if any.

`commment` methods

if ( $result->is_comment ) { ... }

If the above evaluates as true, the following methods will be available on the $result object.

`comment`

if ( $result->is_comment ) {
    my $comment = $result->comment;
    print "I have something to say:  $comment";
}

`bailout` methods

if ( $result->is_bailout ) { ... }

If the above evaluates as true, the following methods will be available on the $result object.

`explanation`

if ( $result->is_bailout ) {
    my $explanation = $result->explanation;
    print "We bailed out because ($explanation)";
}

If, and only if, a token is a bailout token, you can get an "explanation" via this method. The explanation is the text after the mystical "Bail out!" words which appear in the tap output.

`unknown` methods

if ( $result->is_unknown ) { ... }

There are no unique methods for unknown results.

`test` methods

if ( $result->is_test ) { ... }

If the above evaluates as true, the following methods will be available on the $result object.

`ok`

my $ok = $result->ok;

Returns the literal text of the ok or not ok status.

`number`

my $test_number = $result->number;

Returns the number of the test, even if the original TAP output did not supply that number.

`description`

my $description = $result->description;

Returns the description of the test, if any. This is the portion after the test number but before the directive.

`directive`

my $directive = $result->directive;

Returns either TODO or SKIP if either directive was present for a test line.

`explanation`

my $explanation = $result->explanation;

If a test had either a TODO or SKIP directive, this method will return the accompanying explantion, if present.

not ok 17 - 'Pigs can fly' # TODO not enough acid

For the above line, the explanation is not enough acid.

`is_ok`

if ( $result->is_ok ) { ... }

Returns a boolean value indicating whether or not the test passed. Remember that for TODO tests, the test always passes.

Note: this was formerly passed. The latter method is deprecated and will issue a warning.

`is_actual_ok`

if ( $result->is_actual_ok ) { ... }

Returns a boolean value indicating whether or not the test passed, regardless of its TODO status.

Note: this was formerly actual_passed. The latter method is deprecated and will issue a warning.

`has_skip`

if ( $result->has_skip ) { ... }

Returns a boolean value indicating whether or not this test had a SKIP directive.

`has_todo`

if ( $result->has_todo ) { ... }

Returns a boolean value indicating whether or not this test had a TODO directive.

Note that TODO tests always pass. If you need to know whether or not they really passed, check the is_actual_ok method.

TOTAL RESULTS

After parsing the TAP, there are many methods available to let you dig through the results and determine what is meaningful to you.

`plan`

my $plan = $parser->plan;

Returns the test plan, if found.

`passed`

my @passed = $parser->passed; # the test numbers which passed
my $passed = $parser->passed; # the number of tests which passed

This method lets you know which (or how many) tests passed. If a test failed but had a TODO directive, it will be counted as a passed test.

`failed`

my @failed = $parser->failed; # the test numbers which failed
my $failed = $parser->failed; # the number of tests which failed

This method lets you know which (or how many) tests failed. If a test passed but had a TODO directive, it will be counted as a failed test.

`actual_passed`

# the test numbers which actually passed
my @actual_passed = $parser->actual_passed;

# the number of tests which actually passed
my $actual_passed = $parser->actual_passed;

This method lets you know which (or how many) tests actually passed, regardless of whether or not a TODO directive was found.

`actual_ok`

This method is a synonym for actual_passed.

`actual_failed`

# the test numbers which actually failed
my @actual_failed = $parser->actual_failed;
# the number of tests which actually failed
my $actual_failed = $parser->actual_failed;

This method lets you know which (or how many) tests actually failed, regardless of whether or not a TODO directive was found.

`todo`

my @todo = $parser->todo; # the test numbers with todo directives
my $todo = $parser->todo; # the number of tests with todo directives

This method lets you know which (or how many) tests had TODO directives.

`todo_failed`

# the test numbers which unexpectedly succeeded
my @todo_failed = $parser->todo_failed;
# the number of tests which unexpectedly succeeded 
my $todo_failed = $parser->todo_failed;

This method lets you know which (or how many) tests actually passed but were declared as "TODO" tests.

`skipped`

my @skipped = $parser->skipped; # the test numbers with SKIP directives
my $skipped = $parser->skipped; # the number of tests with SKIP directives

This method lets you know which (or how many) tests had SKIP directives.

`is_good_plan`

if ( $parser->is_good_plan ) { ... }

Returns a boolean value indicating whether or not the number of tests planned matches the number of tests run.

Note: this was formerly good_plan. The latter method is deprecated and will issue a warning.

And since we're on that subject ...

`tests_planned`

print $parser->tests_planned;

Returns the number of tests planned, according to the plan. For example, a plan of '1..17' will mean that 17 tests were planned.

`tests_run`

print $parser->tests_run;

Returns the number of tests which actually were run. Hopefully this will match the number of $parser->tests_planned.

exit

$parser->exit;

Once the parser is done, this will return the exit status. If the parser ran an executable, it returns the exit status of the executable.

wait

$parser->wait;

Once the parser is done, this will return the wait status. If the parser ran an executable, it returns the wait status of the executable. Otherwise, this mererely returns the exit status.

`parse_errors`

my @errors = $parser->parse_errors; # the parser errors
my $errors = $parser->parse_errors; # the number of parser_errors

Fortunately, all TAP output is perfect. In the event that it is not, this method will return parser errors. Note that a junk line which the parser does not recognize is not an error. This allows this parser to handle future versions of TAP. The following are all TAP errors reported by the parser:

Misplaced plan

The plan (for example, '1..5'), must only come at the beginning or end of the TAP output.
No plan

Gotta have a plan!

More than one plan

1..3
ok 1 - input file opened
not ok 2 - first line of the input valid # todo some data
ok 3 read the rest of the file
1..3

Right. Very funny. Don't do that.

Test numbers out of sequence

1..3
ok 1 - input file opened
not ok 2 - first line of the input valid # todo some data
ok 2 read the rest of the file

That last test line above should have the number '3' instead of '2'.

Note that it's perfectly acceptable for some lines to have test numbers and others to not have them. However, when a test number is found, it must be in sequence. The following is also an error:

1..3
ok 1 - input file opened
not ok - first line of the input valid # todo some data
ok 2 read the rest of the file

But this is not:

1..3
ok  - input file opened
not ok - first line of the input valid # todo some data
ok 3 read the rest of the file

CALLBACKS

As mentioned earlier, a "callback" key may be added may be added to the TAPx::Parser constructor. If present, each callback corresponding to a given result type will be called with the result as the argument if the run method is used. The callback is expected to be a subroutine reference (or anonymous subroutine) which is invoked with the parser result as its argument.

my %callbacks = (
    test    => \&test_callback,
    plan    => \&plan_callback,
    comment => \&comment_callback,
    bailout => \&bailout_callback,
    unknown => \&unknown_callback,
);

my $aggregator = TAPx::Parser::Aggregator->new;
foreach my $file ( @test_files ) {
    my $stream = TAPx::Parser::Source::Perl->new($file);
    my $parser = TAPx::Parser->new(
        {
            stream    => $stream,
            callbacks => \%callbacks,
        }
    );
    $parser->run;
    $aggregator->add( $file, $parser );
}

There are, at the present time, seven keys allowed for callbacks. These keys are case-sensitive.

1 test

Invoked if $result->is_test returns true.

2 plan

Invoked if $result->is_plan returns true.

3 comment

Invoked if $result->is_comment returns true.

4 bailout

Invoked if $result->is_unknown returns true.

5 unknown

Invoked if $result->is_unknown returns true.

6 ELSE

If a result does not have a callback defined for it, this callback will be invoked. Thus, if all five of the previous result types are specified as callbacks, this callback will never be invoked.

7 ALL

This callback will always be invoked and this will happen for each result after one of the above six callbacks is invoked. For example, if Term::ANSIColor is loaded, you could use the following to color your test output:

my %callbacks = (
    test => sub {
        my $test = shift;
        if ( $test->is_ok && not $test->directive ) {
            # normal passing test
            print color 'green';
        }
        elsif ( !$test->is_ok ) {    # even if it's TODO
            print color 'white on_red';
        }
        elsif ( $test->has_skip ) {
            print color 'white on_blue';

        }
        elsif ( $test->has_todo ) {
            print color 'white';
        }
    },
    ELSE => sub {
        # plan, comment, and so on (anything which isn't a test line)
        print color 'black on_white';
    },
    ALL => sub {
        # now print them
        print shift->as_string;
        print color 'reset';
        print "\n";
    },
);

See examples/tprove_color for an example of this.

##############################################################################

TAP GRAMMAR

NOTE: This grammar is slightly out of date. There's still some discussion about it and a new one will be provided when we have things better defined.

The TAPx::Parser does not use a formal grammar because TAP is essentially a stream-based protocol. In fact, it's quite legal to have an infinite stream. For the same reason that we don't apply regexes to streams, we're not using a formal grammar here. Instead, we parse the TAP in lines.

For purposes for forward compatability, any result which does not match the following grammar is currently referred to as TAPx::Parser::Result::Unknown. It is not a parse error.

A formal grammar would look similar to the following:

(* 
    For the time being, I'm cheating on the EBNF by allowing 
    certain terms to be defined by POSIX character classes by
    using the following syntax:

      digit ::= [:digit:]

    As far as I am away, that's not valid EBNF.  Sue me.  I
    didn't know how to write "char" otherwise (Unicode issues).  
    Suggestions welcome.
*)

(* POSIX character classes and other terminals *)

digit              ::= [:digit:]
character          ::= ([:print:] - "\n")
positiveInteger    ::= ( digit - '0' ) {digit}
nonNegativeInteger ::= digit {digit}


tap         ::= plan lines | lines plan {comment}

plan        ::= '1..' nonNegativeInteger "\n"

lines       ::= line {line}

line        ::= (comment | test | junk ) "\n"

tests       ::= test {test}

test        ::= status positiveInteger? description? directive?

status      ::= 'not '? 'ok '

description ::= (character - (digit | '#')) {character - '#'}

directive   ::= '#' ( 'TODO' | 'SKIP' ) ' ' {character}

comment     ::= '#' {character}

junk        ::= character {character}

BACKWARDS COMPATABILITY

The Perl-QA list attempted to ensure backwards compatability with Test::Harnes. However, there are some minor differences.

Differences

TODO plans

A little-known feature of Test::Harness is that it supported TODO lists in the plan:
```
1..2 todo 2
ok 1 - We have liftoff
not ok 2 - Anti-gravity device activated
```
Under Test::Harness, test number 2 would pass because it was listed as a TODO test on the plan line. However, we are not aware of anyone actually using this feature and hard-coding test numbers is discouraged because it's very easy to add a test and break the test number sequence. This makes test suites very fragile. Instead, the following should be used:
```
1..2
ok 1 - We have liftoff
not ok 2 - Anti-gravity device activated # TODO
```
Unexpectedly succeeding TODO tests now fail

At least as late as Test::Harness 2.63, the following would be reported as a passing test:
```
1..2
ok 1 - We have liftoff
ok 2 - Anti-gravity device activated # TODO
```
However, this meant that TODOs were innappropriately sprinkled through the code and Test::Harness would simply list that one test had unexpectedly succeeded and the poor test author would have to hunt through his test output to find out which one. This is particularly problematic when working in large teams and finding out that another programmer has already implemented a feature elsewhere and your tests may or may not be correct.

Further, with TAPx::Parser, a primitive harness which only reports failing tests should look like the following, even though it's wrong:
```
my $parser = TAPx::Parser->new( { source => $test_file } );
while ( my $result = $parser->next ) {
    print $result->as_string if ! $result->is_ok;
}
```
That works, but if we had stuck with the old Test::Harness behavior, it would not have reported the unexpectedly succeeding tests unless the harness author remembered to test for $result->todo_failed.

ACKNOWLEDGEMENTS

All of the following have helped. Bug reports, patches, (im)moral support, or just words of encouragement have all been forthcoming.

Michael Schwern
Andy Lester
chromatic
GEOFFR
Shlomi Fish
Torsten Schoenfeld
Jerry Gay
Aristotle
Adam Kennedy
Yves Orton
Adrian Howard
Sean & Lil
Andreas J. Koenig

AUTHOR

Curtis "Ovid" Poe, <ovid@cpan.org>

BUGS

Please report any bugs or feature requests to bug-tapx-parser@rt.cpan.org, or through the web interface at http://rt.cpan.org/NoAuth/ReportBug.html?Queue=TAPx-Parser. I will be notified, and then you'll automatically be notified of progress on your bug as I make changes.

COPYRIGHT & LICENSE

This program is free software; you can redistribute it and/or modify it under the same terms as Perl itself.

To install TAPx::Parser, copy and paste the appropriate command in to your terminal.

cpanm

cpanm TAPx::Parser

CPAN shell

perl -MCPAN -e shell
install TAPx::Parser

For more information on module installation, please visit the detailed CPAN module installation guide.

	Global
`s`	Focus search bar
`?`	Bring up this help dialog

	GitHub
`g` `p`	Go to pull requests
`g` `i`	go to github issues (only if github is preferred repository)

	POD
`g` `a`	Go to author
`g` `c`	Go to changes
`g` `i`	Go to issues
`g` `d`	Go to dist
`g` `r`	Go to repository/SCM
`g` `s`	Go to source
`g` `b`	Go to file browse

	Search terms
module: (e.g. module:Plugin)
distribution: (e.g. distribution:Dancer auth)
author: (e.g. author:SONGMU Redis)
version: (e.g. version:1.00)

NAME

VERSION

SYNOPSIS

DESCRIPTION

METHODS

Class methods

new

Instance methods

next

run

INDIVIDUAL RESULTS

Result types

Common type methods

type

as_string

raw

is_plan

is_test

is_comment

is_bailout

is_unknown

is_ok

plan methods

plan

tests_planned

directive

explanation

commment methods

comment

bailout methods

explanation

unknown methods

test methods

ok

number

description

directive

explanation

is_ok

is_actual_ok

has_skip

has_todo

TOTAL RESULTS

plan

passed

failed

actual_passed

actual_ok

actual_failed

todo

todo_failed

skipped

is_good_plan

tests_planned

tests_run

exit

wait

parse_errors

CALLBACKS

TAP GRAMMAR

BACKWARDS COMPATABILITY

Differences

ACKNOWLEDGEMENTS

AUTHOR

BUGS

COPYRIGHT & LICENSE

Module Install Instructions

`new`

`next`

`run`

`type`

`as_string`

`raw`

`is_plan`

`is_test`

`is_comment`

`is_bailout`

`is_unknown`

`is_ok`

`plan` methods

`plan`

`tests_planned`

`directive`

`explanation`

`commment` methods

`comment`

`bailout` methods

`explanation`

`unknown` methods

`test` methods

`ok`

`number`

`description`

`directive`

`explanation`

`is_ok`

`is_actual_ok`

`has_skip`

`has_todo`

`plan`

`passed`

`failed`

`actual_passed`

`actual_ok`

`actual_failed`

`todo`

`todo_failed`

`skipped`

`is_good_plan`

`tests_planned`

`tests_run`

`parse_errors`