NAME

Object::Exercise - Generic execution & benchmark harness for method calls.

SYNOPSIS

use Object::Exercise;

my @operationz =
(
  [
    [ method arg arg arg ... ]          # method and arguments
    [ 1 ],                              # expected value
  ],

  [
    qw( method arg arg arg )            # just check for $@
  ],

  [
    [ qw( method expected to fail ) ]   # continue on failure
    [],
  ],

  [
    [ $coderef, @argz ],                # $obj->$coderef( @argz )
    [ ( 1 .. 10 )     ],                # expected value
    'Coderef returns list'              # hardwired message
  ]
);

# You can push the operations through an class:

$exercise->( 'YourClass', @test_opz );    # YourClass->method( @argz )

# or an object:

my $object = YourClass->new( @whatever );

$object->prepare_for_test( @more_args );

$exercise->( $object,     @test_opz );    # $object->method( @argz )

DESCRIPTION

This package exports a single subroutine , exercise, which functions as an OO execution loop.

$execute is a subroutine reference that takes a list of arguments. The first element in that list is an object of the class being tested. The remaining elements are a list of operations, each of which is an array reference.

Each operation consists of a method call and the method's arguments. Each method call is dispatched using the object, optionally comparing the return value to some pre-defined result.

Exceptions are trapped and logged. The last operation can be re-executed if it fails.

All operations are passed in as arrayrefs. They can be nested either to store a return value and test to run, or to hold a list consisting of a method name and its arguments.

Rationale

The setup code for a typical test file is frequently repetitive. We have to code for the object and each of a collection of method calls. We frequently have to check return values and exception statuses.

This leads to blocks of code like this:

my $obj = Package::New->( ... );

if( defined ( my $return = eval { $obj->method_1( @args_1 ) } ) )
{
  @$return == 3 or die "...";

  cmp_deeply $return, [ ... ], "Failed comparing @argz_1: ...";

}
elsif( $@ )
{
  die "Failed execution of method_1 ...";
}
else
{
  die "Undef returned from method_1 ..."
}

eval { $obj->method_2( @args_2 ) };

if( $@ )
{
  die "Failed execution of method_2 ...";
}
...

The only thing that really varies about any of these are the return values, method name, and arguments.

Object::Exercise reduces all of this to a list of methods and arguments, with optional data validation:

[ method => @args ],      # single flat list in arrayref

or

[
  [ method => @args ]     # same method + arguments
  [ 3 ],                  # with added return value check
],

In both cases $@ is checked on return; in the second case Test::Deep::cmp_deeply is used to validate the returned data.

Test vs. Run-Only Operations

There are two types of operations: tests and run-only. Tests have a hard-coded value that is compared with the method call's return value; the return value of a run-only operation is ignored.

Tests

These are nested arrayrefs:

[
  [ $method => @args  ],
  [ expected return   ],
  'optional message'
],

The return value can be any sort of structure but must be enclosed in an arrayref. The test is run via:

my $result = [ $object->$method( @argz ) ];

cmp_deeply $result, $expected, $message;

This leaves any method called in a list context with the result put into an arryref. This means that the expected value for a call that returns arrayrefs will look like:

[
  [ $method => @argz ],
  [ # outer arrayref stored return value

    [ # return value is itself an arrayref ]
  ],
],

If the method returns hashrefs in list context then use something like:

[
  [ $method => @argz ],
  [ # outer arrayref stored return value

    { # return value is itself an hashref }
  ],
],

The default ok message is formed by joining the method and arguments on whitespace. This can lead to prove issuing lines like:

ok save foobar HASH(0x123456) (999)

but usually gives at least recognizable results.

To override this, simply supply a message of your own:

[
  [ $method => @argz ],
  [ { # return value is itself an hashref } ],
  'Remember: This should return a hashref!'       # your message
],

Testing Known Failures

Sometimes it is useful to test how the code handles invalid requests. In these cases the test will fail. Normally, executing a method that returns with $@ set will be logged as a failed test. If the expected value is an empty array ref (i.e., nothing was expected back) then the $@ will be logged as passed.

These tests look like:

[
  [ qw( method designed to fail ) ]
  '',
]

This will give a message like:

ok save foobar HASH(0x123456) expected failure (999)

Run-Only Items

These consist simply of a method and its arguments:

[ method => arg, arg, ...  ],

A method with no arguments is a one-liner:

[ method ]

which leads to:

$object->$method()

These are called in a void context, so if the method checks wantarray it will get undef. This may affect the execution of some methods, but usually will not (normal tests are wantarray ? a : b without the separate test for defined).

Coderefs

Coderef's are dispatched as standard method calls:

my $coderef = sub { ... };  # or \&somesub


[
    [ $coderef, @argz ],
    [ ... ]
]

is executed as:

$obj->$coderef( @argz )

this allows dispatching the object outside of its class, say to a utility function that does some extra data checking or logging.

Re-Running Failed Operations

Operations are deemed to fail if they raise an exception (i.e., $@ is set at their completion) or if the return value does not compare deeply to expected data (if provided).

In either case, it is often helpful to examine the failed operation. This is accomplished here by wrapping each exectution in a closure like:

my $cmd
= sub
{
  $DB::single = 1 if $debug;

  $obj->$method( @argz )
};

These closures are eval-ed one at a time and then compared to expected values as necessary. If the operation raises an exception or the test fails then $debug is set to true and a breakpoint is set in the main loop. This allows code run in the perl debugger to re-execute the failed operation in single-step mode and see exactly what failed without having to single-step through all of the successful operations.

For example:

perl -d harness_code.t;

will stop execution at the first failed operation, allowing a single s to step into the $obj-$method( @argz )> call.

Harness Directives

There are times when you want to control the execution or harness arguments as it is running. The directives are processed by the harness itself. These can set a breakpoint prior to calling one of the methods, adjust the verbosity, set the continue switch, or set an object value.

Breakpoints

It is sometimes helpful to stop the execution of code before it fails in order to examine its execution before the failure. Any non-ref entry in the data will print the text and set the debug flag to true. After that every operation will halt at the $obj-$method(...)> line.

For example, this will print the message Check why... and stop at the method call:

[
  ...

  'Check why foo returns 2 instead of 3',

  [
    [ qw( frobnicate foo ) ],
    [ 3 ],
  ]
],

EXAMPLES

my @testz =
(
  # evaluate expected failures

  # modify is expected to fail, but the empty
  # arrayref is a signal that nothing is expected
  # back from the test.

  [
    [],
    [ modify => ( label => $field2, 'xyz' ) ],
  ],

  # false label shouldn't change anything

  [
    [],                                         # expect failure
    [ modify => ( label => $field2, ''    ) ],
  ],

  [
    [ qw( ijk                             ) ],  # expect return of 'ijk'
    [ lookup => ( label => $field2        ) ],
  ],

  [
    [],                                         # expect failure
    [ modify => ( label => $field2, 0     ) ],
  ],

  [
    [ qw( ijk                             ) ],  # expect string 'ijk' again
    [ lookup => ( label => $field2        ) ],
  ],

);

Gives output:

ok 10 - lookup label f.10e => ijk
Bogus label: field label 'xyz' used by 'f.10d'
ok 11 - modify label f.10e xyz => expected exception
Bogus modify label: false label '' (f.10e)
ok 12 - modify label f.10e  => expected exception
Bogus modify label: false label '0' (f.10e)

DEBUG

Re-run a failed operation:

$ perl -d ./t/some-test.t;

ok ...
ok ...
ok ...

...

Failed execution:
<failure message>

47:       0
  DB<1> &$cmd

CM::TxDB::Metadata::t::Harness::CODE(0x8ada248)(/home/slembark/sandbox/Cheetahmail/spot/branches/dev_1_0_0/lib/CM/TxDB/Metadata/t/Harness.pm:184):
184:              $obj->$method( @argz )
  DB<<2>> s

The tests can also be run in benchmark mode via:

BENCHMARK=1 perl t/foo.t;

which will skip loading Test::More and run the operations via:

eval { $obj->$method( @$argz ) }

timing the entire exeuction via benchmark.

AUTHOR

Steven Lembark <lembark@wrkhors.com>

COPYRIGHT

Copyright (C) 2007 Steven Lembark. This code is released under the same terms as Perl-5.8.

cpanm Object::Exercise

perl -MCPAN -e shell
install Object::Exercise