/ 
Fennec-0.020
River stage one • 9 direct dependents • 9 total dependents
/ Fennec::Manual::Mission

NAME

Fennec::Manual::Mission - Why Fennec is here

WHY WRITE A NEW TEST FRAMEWORK

Perl testing frameworks currently leave a lot to be desired. There are several projects in the works to try and improve how perl testing is done. These projects usually focus on solving one or two of the problems, sometimes these solutions are incompatible with eachother, or current testing packages. Fennec is an attempt to make a testing framework upon which intercompatible solutions to these problems can be built.

Fennec provides a solid base that is highly extendable. It allows for the writing of custom nestable workflows (like RSPEC), Custom Asserts (like Test::Exception), Custom output handlers (Alternatives to TAP), Custom file types, and custom result passing (collectors). In Fennec all test files are objects. Fennec also solves the forking problem, thats it, forking just plain works.

Fennec tries to play nicely with Test::Builder. It will not interfer with tests that are written using Test::Builder based tools. There is also a wrapper that makes it possible to use Test::Builder based assert packages (like Test::Warn) within Fennec tests. Many Test::Builder tools will work as expected even without the wrapper.

Fennec core does not attempt to solve all the current problems. However it does take them all under consideration and tries to provide extendability to simplify building such solutions. Its not so much Fennec solves your problem as it is that Fennec lets you solve your problem while allowing you to use solutions to the other problems at the same time.

GOALS

Ease of entry

Fennec must be easy to simply drop into an existing project. It must also be easy to use when starting a project. The most common use cases should be simple to implement, with the ability to bring in more complicated items as needed.

Distributability

Fennec must be cross-platform. It should rely on as little non-core dependancies as possible. That is not to say it should homebrew solutions to complicated problems, but rather it should avoid complicated problems where simple solutions are not available.

People will avoid Fennec if there are too many dependancies that are not typically part of a linux distribution. As well we want Fennec to be available for use in a wide range of projects, if it has a deep dependancy tree we increase the chance of eventually making an interdependancy.

When there is a well solved complicated problem a dependancy should still be used. We do not want to take the road of implementing large parts of cpan internally. This is a judgement call and a tradeoff that must be considered whenever a new dependancy is suggested.

This only applies to core Fennec. Extentions are free to depend on whatever they please.

Extendability

Fennec must open itself to extentions in every way possible. If someone needs a hook added we will look at it. Currently every way we can think of that one might wish to extend fennec has been written accordingly.

Compatability

Fennec should work with prove. It should also play nicely with Test::Builder.

PROBLEMS WITH OTHER SOLUTIONS

Some of the big issues being tackled by other projects are as follows.

TODO: Link to projects

Alternate Workflows

Alternate workflows including RSPEC like testing.

Alternate Output

Currently TAP is the standard, and for most cases it is sufficient. The problem is that most solutions give no thought to those who might want an alternate format.

Assertions

These include Test::More and Test::Exception. Most of these are made using Test::Builder, and unless your testing solution plays nicely with TB you will probably have to write your own set of assertions, this can require re-writing well tested solutions to common problems.

Forking

Ability to fork and have both processes generate results. There are 2 current problems preventing forking tests from working properly. Test::Builder includes numbers in its TAP output. When you fork you have duplicate numbers, or out of order numbers. Note - Test::Builder2 apparently solves this problem.

The other problem is much more sinister, and harder to detect. When you fork in perl all processes have the same STDOUT and STDERR output handles. It is possible that 2 processes can 'fight' and output results at the same time. When this happens you can have part of one result, followed by a different result, before the rest of the first result appears. The more tests you have the more likely you are to see this. This bug would be completely random and unpredictable.

NOTE (4/24/2010) The second problem listed is proving difficult to prove:

In the past I wrote a script to verify this problem, however I was unable to find that proof when writing fennec. I have since written another test script in an attempt to demonstrate this again. My new script was unable to demonstrate the problem.

Rather than hiding my lack of proof, I present it here, and freely admit the second forking problem may not in fact be a problem. I would love to have someone more familiar with the IO systems confirm or disprove the problem.

My failed attempt can be found here: http://gist.github.com/raw/378112/4b74dfb6aa453a9df944ac8489b8825b6bc072bb/outputtest.pl

AUTHORS

Chad Granum exodist7@gmail.com

COPYRIGHT

Copyright (C) 2010 Chad Granum

Fennec is free software; Standard perl licence.

Fennec is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the license for more details.