NAME

Games::Golf::TestSuite - An object that can run a suite of tests.

SYNOPSIS

use Games::Golf::TestSuite;

my $test   = new Games::Golf::TestSuite( "hole" );

# try a list of Games::Golf::Entry objects against the test
# the tests will be run only on untested entries
$test->run ( @entries );

# tests a single entry, and return the result
my $result = $test->check( $entry );

DESCRIPTION

!!FIXME!! Games::Golf::TestSuite

Methods

$test = Games::Golf::TestSuite->new( "hole.t" );

Constructs a Games::Golf::TestSuite object with the tests held in the file given as an argument.

A test suite is a perl script written as follow:

# first test: does the script at least compile?
$test->compile;
# check 
$in   = 'some input to give the script';
$out  = 'the script expected output';
$err  = '';    # don't expect anything on STDERR
$exit = undef; # don't care for exit code
$test->aioee( $args, $in, $out, $err, $exit );

# use other Games::Golf::TestSuite method
# to create as many subtests as needed

$test->limit( time => 30, stdout => 1024, stderr => 1024 );

Sets limits on the scripts under test.

!!FIXME!! More documentation here.

time

A time limit for each test (default unlimited).

stdout and stderr

A data limit for stdout and stderr (default unlimited).

$test->run( @entries );

Run the testsuite on the given list of Games::Golf::Entry objects. This method simply loops over @entries with check() method.

If the Games::Golf::Entry object has already been tested, it will not be tested again.

$test->check( $entry );

Run the testsuite on a single Games::Golf::Entry object, update the object and return the results of the test. The result attribute of the Games::Golf::Entry object is updated.

Available methods for testing scripts

These tests are Games::Golf::TestSuite methods that hole makers can use in their hole test scripts.

$test->compile;

Does the player script at least compile? This does count as a test in the testsuite.

$test->aioee( $args, $in, $out, $err, $exit );

Given $args and $in, the script should output $out on STDOUT, $err on STDERR and exit with code $exit. If you don't care about any of the three outputs (stdout, stderr or exit code), just pass "" or undef instead.

WARNING! This sub is a minimal workaround, and may deadlock on big input sets, since we do not check via select if we can write.

use constant SELECT_TIMEOUT => 0.05;

my $file = $self->{file} . ".pl";
my ($pid, $ec);

# Restrictions to apply
my %limit = 
  ( time   => 30,           # 30 Seconds
    stdout => 1000000,      # 1  Megabyte
    stderr => 1000000,      # 1  Megabyte
    @_
  );

croak "Invalid parameters passed!"
  unless (scalar keys %limit == 3);

# Storage for data read
my ($stdout, $stderr) = ("", "");

# Use exceptions
eval {
    # Start code under test
    local (*IN, *OUT, *ERR);
    $pid  = open3(\*IN, \*OUT, \*ERR, "$^X $file");

    # Use select to monitor filehandles
    my $select = IO::Select->new;
    $select->add(\*OUT, \*ERR); # do not check \*IN.

    # Do writing.
    print IN $in;
    close IN;

    waitpid($pid, 0);
    $ec = $? >> 8;

    # Do reading
    while ( my @ready = $select->can_read(SELECT_TIMEOUT) ) {

        foreach my $fh ( @ready ) {
            # from STDOUT
            if ($fh == \*OUT) {
                $stdout .= <OUT>;
            }
            # from STDERR
            elsif ($fh == \*ERR) {
                $stderr .= <ERR>;
            }
            $select->remove($fh) if eof($fh);
        }
    }
    #waitpid($pid, 0);

    close OUT;
    close ERR;
};

# Handle exceptions (TODO)
#if ($@) {
#die $@;
#}

Available tests for testing subs

If the hole requires a sub, the testsuite must first create the sub with makesub(), and then can use it.

The sub is stored in the sub attribute of the Games::Golf::TestSuite object, and is named after the hole, in the Games::Golf::TestSuite::Sandbox package (in case the sub calls itself).

$test->makesub;

Create a subroutine named after the hole in the sandbox. It can then be used through $test->sub( ... ) or by calling it directly by its name (the hole name).

This does count as a test in the testsuite. The sub should at least be valid Perl, shouldn't it?

$test->sub( ... );

Call the subroutine defined with makesub().

$test->ok( $result, $expected );

Similar to the ok() sub used in the Test module. The two parameters are in scalar context.

The following examples are straight from Test.pm:

$test->ok(0,1);             # failure: '0' ne '1'
$test->ok('broke','fixed'); # failure: 'broke' ne 'fixed'
$test->ok('fixed','fixed'); # success: 'fixed' eq 'fixed'
$test->ok('fixed',qr/x/);   # success: 'fixed' =~ qr/x/

$test->ok(sub { 1+1 }, 2);  # success: '2' eq '2'
$test->ok(sub { 1+1 }, 3);  # failure: '2' ne '3'
$test->ok(0, int(rand(2));  # (just kidding :-)

$test->ok 'segmentation fault', '/(?i)success/'; # regex match

Other test for the code itself

These tests are Games::Golf::TestSuite methods that hole makers can use in their hole test scripts.

The subtests use the code stored in attribute code, and update the attribute result.

$test->not_string( $s );

Test that the code in the entry doesn't contain the string $s.

$test->not_match( $regex );

Test that the code in the entry doesn't match $regex.

$test->not_op( $op );

Test that the code in the entry doesn't use the given opcode. (TODO)

BUGS

Please report all bugs to:

http://rt.cpan.org/NoAuth/Bugs.html?Dist=Games-Golf

TODO

Arrange for the entry script to be run with use strict; and warnings. (right now, they are run in a no strict, no warnings sandbox.)

AUTHORS

Philippe 'BooK' Bruhat <book@cpan.org>

Dave Hoover <dave@redsquirreldesign.com>

Steffen Müller <games-golf@steffen-mueller.net>

Jonathan E. Paton <jonathanpaton@yahoo.com>

Jérôme Quelin <jquelin@cpann.org>

Eugène Van der Pijll <E.C.vanderPijll@phys.uu.nl>

COPYRIGHT

This module is free software. It may be used, redistributed and/or modified under the same terms as Perl itself.

SEE ALSO

perl, Games::Golf, Games::Golf::Entry.

cpanm Games::Golf

perl -MCPAN -e shell
install Games::Golf