NAME

Fennec::Manual::Developer - Developer manual for extending or enhancing Fennec.

DESCRIPTION

Guide to developing plugins or enhancements for Fennec.

CHAPTER 1 - INNER WORKINGS

Fennec starts by loading a runner, even in standalone. This runner does the following:

STARTUP PHASE

Gather process information
Take over Test::Builder if needed
Start a collector to gather results
Initialize output handlers
Create a fork controller aka threader (NOT USING I-THREADS): Threader should not be confused with perls horrible threading.

TESTING PHASE

The following is run for each test file the runner finds. For standalone tests it occurs only once; for only the one file. The runner may do this in multiple processes, one per file.

Run test file: Creates test groups and workflow objects, as well as running any package level asserts.
Process Workflows: Workflow codeblocks are run as methods and produce additional test groups.
Run all test groups from all workflows: All the test groups defined in the file and the workflows are randomized and run as methods. The file process may run multiple groups at once in several processes.

CLEANUP PHASE

The cleanup phase tears everything down after tests have been run.

Wait on child processes
Shutdown output handles

CHAPTER 2 - NAMESPACES

Fennec: This is the primary Fennec namespace.
Fennec::Workflow: This is the namespace for workflow implementations. Things like RSPEC an Case live here.
Fennec::Util: Used for misc utility items.
Fennec::TestSet: TestSet is the class used to implement test group methods, and any workflow test grouping.
Fennec::TestFile: The base class for all Fennec test classes. Custom file types may subclass this for their own base classes.
Fennec::Parser: Namespace for Devel::Declare Magic.
Fennec::Output: This is where output objects such as results, diag, etc. live. These are the objects that get serialized and sent to the handlers in the runner.
Fennec::Handler: This is where output handlers go. These objects take the output objects and present them to the user in a useful way such as TAP.
Fennec::FileType: The namespace for filetype classes which are used to load test files.
Fennec::Collector: Namespace for collectors which funnel results to the parent thread.
Fennec::Base: This is where various base classes go.
Fennec::Assert: This is the namespace for libraries of assert functions.

CHAPTER 3 - TESTING FENNEC ITSELF

Fennec provides a module that makes it easy to test itself. Anyone who has written libraries for Test::Builder has probably used the nightmare that is Test::Builder::Tester. Let me assure you Fennec learned from this nightmare and does not repeat it.

Fennec provides Fennec::Assert::Interceptor which can override the collector and intercept result objects without sending them to the main process. This library provides the capture() function for this task. Here is a snippet from t/Fennec/Assert/Core/Simple.pm showing its use:

package TEST::Fennec::Assert::Simple;
use strict;
use warnings;
use Fennec asserts => [qw/ Core Interceptor /];

tests 'ok' => sub {
    my $output = capture {
        ok( 1, 'pass' );
        ok( 0, 'fail' );
        ok( 1 );
        ok( 0 );
    };

    is( @$output, 4, "4 results" );

    is( $output->[0]->pass, 1, "passed" );
    is( $output->[0]->name, 'pass', 'name' );

    is( $output->[1]->pass, 0, "failed" );
    is( $output->[1]->name, 'fail', 'name' );

    is( $output->[2]->pass, 1, "passed" );
    is( $output->[2]->name, 'nameless test', 'name' );

    is( $output->[3]->pass, 0, "failed" );
    is( $output->[3]->name, 'nameless test', 'name' );
};

Each item will be one of the subclasses of Fennec::Output, such as Fennec::Output::Result and Fennec::Output::Diag.

CHAPTER 4 - WRITING ASSERTION LIBRARIES

Assertion libraries are the most common testing tool type written. Fennec makes the task of writing assertion libraries very simple. Fennec provies utilities that let you focus on your assertion library instead of worrying about the framework details.

CHAPTER 4.1 - WRITING NEW LIBRARIES

An assertion library must first import Fennec::Assert, this will turn the calling package into an Exporter::Declare subclass. This will also provide several helper functions:

util

Used to export functions that will not generate result objects.

All of the following are valid:

# Export and define seperately
util 'my_util';
sub my_util { ... }

# export and declare together
util my_util => sub { ... };

# export and declare using Devel::Declare shortcuts
util my_util { ... }

# export and declare attaching Devel::Declare magic
util my_util PARSER { ... }

tester

Used to export functions that generate results. This will wrap your function with code to provide diagnostics, line numbers, package names, and other contextal information you dont want figure out yourself.

# Export and define seperately
tester 'my_assert';
sub my_assert { ... }

# export and declare together
tester my_assert => sub { ... };

# export and declare using Devel::Declare shortcuts
tester my_assert { ... }

# export and declare attaching Devel::Declare magic
tester my_assert PARSER { ... }

result

Use this function in your asserts to generate a result object. It will build the result object, add context, and send the result to the collector.

In most cases you simply provide a name and a boolean:

result(
    pass   => $BOOL,
    name   => $NAME,
);

Here are all options:

result(
    # Required:
    pass => $BOOL,

    # Optional:
    name   => $NAME,
    stderr => \@ERRORS,
    stdout => \@NOTES,

    # Filled in automatically, listed here so you can override
    line           => $LINE_NO,
    file           => $FILE_NAME,
    todo           => $REASON,
    skip           => $REASON,
    testset_name   => $NAME_OF_TEST_GROUP,
    workflow_stack => \@NAMES,
);

Only specify file and line if Fennec gets it wrong. Deducing these is not something you want to figure out yourself.

diag

Messages that should always be displayed.

diag( $message1, $message2, @other_messages )

note

Messages that should be displayed in verbose mode.

note( $message1, $message2, @other_messages )

test_caller

Returnes a hash with line and file as keys. Used to find context for results.

my %context = test_caller();

tb_wrapper

Used to improve Test::Builder asserts. See Chapter 4.2 for more details

COMPLETE EXAMPLE

package Fennec::Assert::MyAssert;
use strict;
use warnings;
use Fennec::Assert;

tester my_ok {
    my ( $ok, $name ) = @_;
    result(
        pass => $ok ? 1 : 0,
        name => $name || 'nameless test',
    );
}

util log_time {
    diag( "Time: " . time());
}

1;

CHAPTER 4.2 - WRAPPING TEST::BUILDER BASED TOOLS

Wrapping a Test::Builder library is exceedingly simple:

use Fennec::Assert
import the functions you wish to wrap
use tb_wrapper() to wrap each function
export the resulting coderefs using tester() or util()

EXAMPLE

package Fennec::Assert::TBCore::More;
use strict;
use warnings;

use Fennec::Assert;
use Fennec::Output::Result;
require Test::More;

our @LIST = qw/ ok is isnt like unlike cmp_ok can_ok isa_ok new_ok pass fail
                use_ok require_ok is_deeply /;

for my $name ( @LIST ) {
    no strict 'refs';
    next unless Test::More->can( $name );
    tester( $name => tb_wrapper( \&{ 'Test::More::' . $name }));
}

1;

CHAPTER 5 - WRITING CUSTOM OUTPUT HANDLERS

An output handler must subclass Fennec::Handler. The baseclass provides a generic constructor which you may override if you wish. You can also hook into construction by writing an init method. Handlers are constructed with no arguments.

The primary method to implement is handle( $output ), here is a complete list of methods that you may want to implement:

handle( $output ): Every output object generated is passed to each handler via this method. Every item passed in will be a subclass of Fennec::Output. You should familiarize yourself with the 3 main output types, Fennec::Output::Result, Fennec::Output::Diag, Fennec::Output::Note. You should handle these three types as well as handle unknown types generically.
finish(): This is called with no arguments when testing is complete and no more results will be recieved. This is your last chance to do anything.
start(): This is called when Fennec starts testing.
starting_file( $filename ): This will be called whenever a process starts running a new test file.
fennec_error( @errors ): This is called when fennec encounters an error not related to testing.

EXAMPLE

This is an example handler that ignores everything but errors, which it writes to a file.

package Fennec::Handler::MyHandler;
use strict;
use warnings;

use base 'Fennec::Handler';

use Fennec::Util::Accessors;
Accessors qw/ fh count files /;

sub start {
    my $self = shift;
    open( my $fh, ">", "failures.log" ) || die( "Error opening failure log: $!" );
    $self->fh( $fh );
    $self->count( 0 );
    $self->files( {} );
}

sub handle {
    my $self = shift;
    my ( $item ) = @_;
    return unless $item
           && $item->isa( 'Fennec::Output::Result' )
           && !$item->pass;

    $self->count( $self->count + 1 );
    my $file = $item->file;
    $file =~ s|.*(perl_lib/)|$1|g;
    $self->files->{ $file }++;

    my $fh = $self->fh;
    print $fh <<EOT
===========================================================
File: @{[ $item->file ]}
Test: @{[ $item->name() ]}
Workflow Stack: @{[ join( ', ', @{ $item->workflow_stack() || [] })]}
Err:
@{[ join( "\n", @{ $item->stderr || [] })]}
EOT
}

sub finish {
    my $self = shift;
    my $fh = $self->fh;
    print $fh "=======END========\n";
    print $fh "$_: " . $self->files->{$_} . "\n" for keys %{ $self->files };
    print $fh "\nTotal Errors: " . $self->count . "\n";
    close( $fh );
}

sub fennec_error {
    my $self = shift;
    my $fh = $self->fh;
    print $fh "=====FENNEC ERRORS====\n", @_;
}

1;

CHAPTER 6 - WRITING CUSTOM COLLECTORS

The collector is an abstracted IPC interface. All collectors must subclass Fennec::Collector. There are 2 methods to implement:

write( $output ): This method takes a single output object. This method needs to pass the object to the main thread.
@outputs = cull(): This should return a list of output objects collected from the child processes.

You may also override these, but you must remember to call SUPER.

start(): Called in the parent process to initialize the handlers.
finish(): Called in the parent process to shutdown the handlers.

You should familiarize yourself with Fennec::Collector, and avoid overriding any other methods.

CHAPTER 7 - WRITING CUSTOM FILE TYPES

You must implement the following methods.

$bool = $class->valid_file( $filename ): Return true if the given filename is a valid file of the type you expect.
$package = $obj->load_file(): Must load the test file. This must also make sure that a package name is generated for the file. When finished the method must return the name of the package for the test file. This package must also be a subclass of Fennec::TestFile.
@list = $obj->paths(): A list of paths, relative to the project root, in which to search for test files.

TRUNCATED SYNOPSIS

package Fennec::FileType::MyType;
use strict;
use warnings;

use base 'Fennec::FileType';

sub valid_file {
    my $class = shift;
    my ( $file ) = @_;
    ...
    return $bool;
}

sub load_file {
    my $self = shift;
    my $file = $self->filename;
    ...
    return $package;
}

sub paths {qw( t/ ... )}

1;

CHAPTER 8 - WRITING CUSTOM WORKFLOWS

CHAPTER 8.1 - WHY WRITE A WORKFLOW

In fennec a workflow is a way to structure tests. There are many popular testing frameworks that owe their popularity to how they structure their tests. A notable example is Ruby's RSPEC. Structured tests can simplify your job as a tester.

Within a workflow you can define child workflows, and groups of tests. Ultimately your workflow will return a list of testset objects. You can be as creative as you want in designing your workflow. You can also subclass Fennec::TestSet or use Fennec::TestSet::SubSet to wrap groups with buildup or teardown functions.

Using the Fennec::Workflow system it was a minimal effort to write an RSPEC like workflow. Less than 100 lines of code in SPEC.pm at the time this was written. If you do not like how Fennec implements SPEC you can probably implement it your own quite easily.

CHAPTER 8.2 - OVERVIEW

When a test file is loaded Fennec::Runner creates a root workflow, this root workflow is an Fennec::Workflow object. Every testset or workflow defined in the test file is a blessed method. The blessed methods are passed to a parent workflow. Those defined outside of a workflow are passed to a 'root' workflow.

A test classes root workflow is stored inthe classes meta object. Calling $test_class->fennec_meta() will return the meta object for a testclass. The meta object has 2 methods for workflows: $meta->root_workflow() which always returns the root workflow, and $meta->workflow() which returns the highest workflow on the stack, also known as the 'current' workflow.

Any testsets or workflows defined inside the method of a parent workflow are passed to that parent workflow. Each workflow object must inherit or implement the add_item() method which takes a workflow or testset. Testsets are retrieved from the workflows after they have been traversed to the deepest level.

Example:

package MyTest;
use Fennec workflows => [ 'MyWorkflow' ];

my_workflow parent => sub {
    tests 'parent tests' => {
        ...
    };
    my_workflow nested => sub {
        tests 'child tests' => {
            ...
        };
    };
};

1;

When Fennec runs it will load the file, it will create a root workflow, the workflow sub named 'parent' will be blessed and passed to the root workflow as a child workflow. The workflow sub 'nested' will be blessed as a workflow and passed to 'parent'. While this is going on the testsets will also be blessed and passed into their parent workflows.

CHAPTER 8.3 - CUSTOM WORKFLOW SYNOPSIS

package Fennec::Workflow::MyWorkflow;
use strict;
use warnings;

use Fennec::Workflow qw/:subclass/;

build_hook { ... };

build_with 'my_workflow';

sub testsets {
    my $self = shift;
    ...
    return @testsets;
}

sub add_item {
    my $self = shift;
    my ($item) = @_;

    if ( we_handle( $item )) { ... }
    else { $self->SUPER::add_item( $item )}
}

1;

CHAPTER 8.4 - METHODS THAT CAN BE OVERRIDEN

Overriding these is optional, not overriding them will result in a workflow that acts just like the root workflow.

@testsets = $wf->testsets()

testsets() should return an array with 0 or more testset objects, or objects that subclass testset. It is also responsible for returning the testsets from child workflows. If this method does not get testsets from child workflows they will not be run (which will generate a warning).

$wf->build()

The inherited method sets $wf as the current workflow, it then runs the method that was blessed into as the workflow object. Overriding this is not recommended, but may be necessary for some complicated workflows.

You should always call $self->built(1) at the end of your custom build() method. Not doing this may cause problems with items being added to a workflow AFTER build().

$wf->build_children()

This should rarely need to be overriden, calls $child->build() on all child workflows. If you override add_items to add items other than workflows and testsets, or to disallow adding items at all then you will probably want to override this to reflect the change.

$wf->add_item( $item )

Add $item to the workflow. The inherited method will add workflows or testsets, it will throw an exception for anything else. You can access the added items via $self->_workflows() and $self->_testsets().

If you override this method you should die with a useful message if $self->built() is true.

CHAPTER 8.5 - HELPFUL FUNCTIONS

build_with( $name )
build_with( $name, $class ): Exports a method ($name) that when called will create an instance of $class, or the class in which build_with() was called. The exported method will take a name and codeblock as arguments. After being build the instance will be added to the current in which it was defined.
build_hook { ... }
build_hook( sub { ... }): Add code that should be run just after building the workflows and just before running the tests.

CHAPTER 8.6 - OTHER METHODS TO KNOW

import(): Fennec::Workflow has a complicated import() method, in order to simplify it all classes that sublcass Fennec::Workflow have a new import() method exported to their package. It is important that you do not try to override import(), or that you are at least aware that you cannot call $wf->SUPER::import() and get the expected behavior. Defining your own import() method will also throw a redefine warning.
@wfs = $wf->workflows(): Returns a list of all the workflows added as children.
@testsets = $wf->testsets(): Returns a list of all the testsets in this workflow, and all of its children.
$tf = $wf->testfile(): Returns the Fennec::TestFile object currently being run.
$pwf = $wf->parent(): Returns the parent workflow object to which this one is a child, the root workflow will return the TestFile object.
$testsets = $wf->_testsets()
$wf->_testsets( \@testsets ): Get/Set the list of testsets, if you override add_item() and never caller SUPER::add_item() then you will need to manually add TestSets to the arrayref returned by _testsets().
$workflows = $wf->_workflows()
$wf->_workflows( \@workflows ): Get/Set the list of workflows, if you override add_item() and never caller SUPER::add_item() then you will need to manually add Workflows to the arrayref returned by _workflows().
run_tests(): In a normal Fennec run this will only be called on the root Workflow object. Overriding this in your subclass will have NO EFFECT.
$wf->add_items( @items ): Calls $wf->add_item() for each item in @items.

CHAPTER 9 - RUNNER PLUGINS

It is also possible to create plugins that extend or modify the runner itself. There are a few ways in which you can hook into the runner.

CHAPTER 9.1 - CONFIGURATION OPTIONS

Fennec::Runner can export the add_config() function. This function allows you to add configuration items to the runner. These items can be listed in t/Fennec.t along with the usual ones. This will create the config option itself, as well as an accessor on the runner singleton by which it can be accessed.

add_config $CONFIG_NAME

add_config $CONFIG_NAME => $default

add_config $CONFIG_NAME => ( %OPTIONS )

Options:

default => $VALUE

default => sub { my ($data) = @_; ... }

Specify the default value when none is specified. Can either be scalar value (including refs) or a coderef. The only argument provided to the coderef is the $data hash. The $data hash contains all the options that have been calculated so far for the runner.

You absolutely should not modify the $data hash in coderefs passed to this parameter. The only fields guaranteed to be set are those listed in the depends option.

env_override => $VALUE

Set to false in order to disable environment overrides to this parameter. When not specified it defaults to true. When true (numeric) this parameter can be overriden using the FENNEC_[NAME] envireonment variable.

You can also specify a string name that will be used instead of FENNEC_[NAME].

modify => sub { my ($value, $data) = @_; ... }

This hook can be used to modify the value the user provides. It can also modify the default value that you provide. The first argument is the unaltered value. The second argument is the $data hash. See the section for the 'default' option for more info about $data.

depends => \@LIST

Arrayref of config options that must be calculated before this one.

CHAPTER 9.2 - TEST HOOKS

Test hooks are run after the root workflow has been instantiated, but before it has been processed. Test hooks are coderefs that receive the runner and test object as arguments.

Fennec::Runner can export the add_test_hook() function. This function allows you to add test hooks to the runner.

add_test_hook( sub { my( $runner, $test ) = @_; ... })

CHAPTER 9.3 - FINISH HOOKS

These are run just before the threader and collector are stopped. These are coderefs that recieve no arguments. Fennec::Runner can export the add_finish_hook() function. This function allows you to add finnish hooks to the runner.

add_finish_hook(sub {...})

AUTHORS

Chad Granum exodist7@gmail.com

COPYRIGHT

Fennec is free software; Standard perl licence.

Fennec is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the license for more details.

To install Fennec, copy and paste the appropriate command in to your terminal.

cpanm

cpanm Fennec

CPAN shell

perl -MCPAN -e shell
install Fennec

For more information on module installation, please visit the detailed CPAN module installation guide.

	Global
`s`	Focus search bar
`?`	Bring up this help dialog

	GitHub
`g` `p`	Go to pull requests
`g` `i`	go to github issues (only if github is preferred repository)

	POD
`g` `a`	Go to author
`g` `c`	Go to changes
`g` `i`	Go to issues
`g` `d`	Go to dist
`g` `r`	Go to repository/SCM
`g` `s`	Go to source
`g` `b`	Go to file browse

	Search terms
module: (e.g. module:Plugin)
distribution: (e.g. distribution:Dancer auth)
author: (e.g. author:SONGMU Redis)
version: (e.g. version:1.00)

NAME

DESCRIPTION

CHAPTER 1 - INNER WORKINGS

STARTUP PHASE

TESTING PHASE

CLEANUP PHASE

CHAPTER 2 - NAMESPACES

CHAPTER 3 - TESTING FENNEC ITSELF

CHAPTER 4 - WRITING ASSERTION LIBRARIES

CHAPTER 4.1 - WRITING NEW LIBRARIES

COMPLETE EXAMPLE

CHAPTER 4.2 - WRAPPING TEST::BUILDER BASED TOOLS

EXAMPLE

CHAPTER 5 - WRITING CUSTOM OUTPUT HANDLERS

EXAMPLE

CHAPTER 6 - WRITING CUSTOM COLLECTORS

CHAPTER 7 - WRITING CUSTOM FILE TYPES

TRUNCATED SYNOPSIS

CHAPTER 8 - WRITING CUSTOM WORKFLOWS

CHAPTER 8.1 - WHY WRITE A WORKFLOW

CHAPTER 8.2 - OVERVIEW

CHAPTER 8.3 - CUSTOM WORKFLOW SYNOPSIS

CHAPTER 8.4 - METHODS THAT CAN BE OVERRIDEN

CHAPTER 8.5 - HELPFUL FUNCTIONS

CHAPTER 8.6 - OTHER METHODS TO KNOW

CHAPTER 9 - RUNNER PLUGINS

CHAPTER 9.1 - CONFIGURATION OPTIONS

CHAPTER 9.2 - TEST HOOKS

CHAPTER 9.3 - FINISH HOOKS

AUTHORS

COPYRIGHT

Module Install Instructions