NAME
Config::Model::Tester - Test framework for Config::Model
SYNOPSIS
# in t/foo.t
use warnings;
use strict;
use Config::Model::Tester ;
use ExtUtils::testlib;
my $arg = shift || '';
my $test_only_model = shift || '';
my $do = shift ;
run_tests($arg, $test_only_model, $do) ;DESCRIPTION
This class provides a way to test configuration models with tests files. This class was designed to tests several models and several tests cases per model.
A specific layout for test files must be followed
Simple test file layout
t/model_tests.d
|-- fstab-examples
| |-- t0
| \-- t1
\-- fstab-test-conf.plIn the example above, we have 1 model to test: fstab.
Test specification is written in fatab-test-conf.pl file. Test cases are plain files in fstab-examples. fstab-test-conf.pl will contain instruction so that each file will be used as a /etc/fstab test case.
fatab-test-conf.pl can contain specifications for more test case. Each test case will require a new file in fstab-examples directory.
Test file layout for multi-file configuration
t/model_tests.d
|-- dpkg-examples
| \-- libversion
| \-- debian
| |-- changelog
| |-- compat
| |-- control
| |-- copyright
| |-- rules
| |-- source
| | \-- format
| \-- watch
\-- dpkg-test-conf.plIn the example above, the test specification is written in dpkg-test- conf.pl. Dpkg layout requires several files per test case. dpkg-test- conf.pl will contain instruction so that each directory under dpkg- examples will be used.
Test file layout depending on system
t/model_tests.d/
|-- ssh-examples
\-- basic
| |-- system_ssh_config
| \-- user_ssh_config
\-- ssh-test-conf.plIn this example, the layout of the configuration files depend on the system. For instance, system ssh_config is stored in /etc/ssh on Linux, and directly in /etc on MacOS.
ssh-test-conf.pl will specify the target path of each file. I.e.:
$home_for_test = $^O eq 'darwin' ? '/Users/joe'
: '/home/joe' ;
# ...
setup => {
'system_ssh_config' => {
'darwin' => '/etc/ssh_config',
'default' => '/etc/ssh/ssh_config',
},
'user_ssh_config' => "$home_for_test/.ssh/config"Basic test specification
Each model test is specified in <model>-test-conf.pl. This file contains a set of global variable. (yes, global variables are often bad ideas in programs, but they are handy for tests):
# config file name (used to copy test case into test wr_root directory)
$conf_file_name = "fstab" ;
# config dir where to copy the file
#$conf_dir = "etc" ;
# home directory for this test
$home_for_test = '/home/joe' ;Here, t0 file will be copied in wr_root/test-t0/etc/fstab.
# config model name to test
$model_to_test = "Fstab" ;
# list of tests
@tests = (
{
# test name
name => 't0',
# add optional specification here for t0 test
},
{
name => 't1',
# add optional specification here for t1 test
},
);
1; # to keep Perl happyInternal tests
$model can also be called to create models to test specific Config::Model behaviors. Use with caution.
Test specification with arbitrary file names
In some models (e.g. Multistrap, the config file is chosen by the user. In this case, the file name must be specified for each tests case:
$model_to_test = "Multistrap";
@tests = (
{
name => 'arm',
config_file => '/home/foo/my_arm.conf',
check => {},
},
);test scenario
Each subtest follow a sequence explained below. Each step of this sequence may be altered by adding specification in the test case:
Setup test in
wr_root/<subtest name>/. If your configuration file layout depend on the target system, you will have to specify the path usingsetupparameter:setup => { 'file_name_in_examples_dir' => { 'darwin' => '/etc/foo', # macosx 'default' => '/etc/bar' # others }, 'another_file_in_examples_dir' => $computed_path }Create configuration instance, load config data and check its validity. Use
load_check => 'no'if your file is not valid.Check for config data warning. You should pass the list of expected warnings. E.g.
load_warnings => [ qr/Missing/, (qr/deprecated/) x 3 , ],Use an empty array_ref to masks load warnings.
Optionally load configuration data. You should design this config data to suppress any error or warning mentioned above. E.g:
load => 'binary:seaview Synopsis="multiplatform interface for sequence alignment"',Optionally, call apply_fixes:
apply_fix => 1,Call dump_tree to check the validity of the data. Use
dump_errorsif you expect issues:dump_errors => [ # the issues the fix that will be applied qr/mandatory/ => 'Files:"*" Copyright:0="(c) foobar"', qr/mandatory/ => ' License:FOO text="foo bar" ! Files:"*" License short_name="FOO" ' ],Likewise, specify any expected warnings (note the list must contain only
qrstuff):dump_warnings => [ (qr/deprecated/) x 3 ],You can tolerate any dump warning this way:
dump_warnings => undef ,Run specific content check to verify that configuration data was retrieved correctly:
check => [ 'fs:/proc fs_spec', "proc" , 'fs:/proc fs_file', "/proc" , 'fs:/home fs_file', "/home", ],You can run check using different check modes (See "fetch( ... )" in Config::Model::Value) by passing a hash ref instead of a scalar :
check => [ 'sections:debian packages:0' , { qw/mode layered value dpkg-dev/}, ''sections:base packages:0', { qw/mode layered value gcc-4.2-base/}, ],The whole hash content (except "value") is passed to grab and fetch
Verify annotation extracted from the configuration file comments:
verify_annotation => { 'source Build-Depends' => "do NOT add libgtk2-perl to build-deps (see bug #554704)", 'source Maintainer' => "what a fine\nteam this one is", },Write back the config data in
wr_root/<subtest name>/. Note that write back is forced, so the tested configuration files are written back even if the configuration values were not changed during the test.You can skip warning when writing back with:
no_warnings => 1,Check the content of the written files(s) with Test::File::Contents:
file_content => { "/home/foo/my_arm.conf" => "really big string" , } file_contents_like => { "/home/foo/my_arm.conf" => qr/should be there/ , } file_contents_unlike => { "/home/foo/my_arm.conf" => qr/should NOT be there/ , }Check added or removed configuration files. If you expect changes, specify a subref to alter the file list:
file_check_sub => sub { my $list_ref = shift ; # file added during tests push @$list_ref, "/debian/source/format" ; };Copy all config data from
wr_root/<subtest name>/towr_root/<subtest name>-w/. This steps is necessary to check that configuration written back has the same content as the original configuration.Create another configuration instance to read the conf file that was just copied (configuration data is checked.)
Compare data read from original data.
Run specific content check on the written config file to verify that configuration data was written and retrieved correctly:
wr_check => { 'fs:/proc fs_spec', "proc" , 'fs:/proc fs_file', "/proc" , 'fs:/home fs_file', "/home", },Like the
checkitem explained above, you can run check using different check modes.
running the test
Run all tests:
perl -Ilib t/model_test.tBy default, all tests are run on all models.
You can pass arguments to t/model_test.t:
a bunch of letters. 't' to get test traces. 'e' to get stack trace in case of errors, 'l' to have logs. All other letters are ignored. E.g.
# run with log and error traces perl -Ilib t/model_test.t elThe model name to tests. E.g.:
# run only fstab tests perl -Ilib t/model_test.t x fstabThe required subtest E.g.:
# run only fstab tests t0 perl -Ilib t/model_test.t x fstab t0
Examples
See http://config-model.hg.sourceforge.net/hgweb/config-model/config-model/file/tip/config-model-core/t/model_tests.d/debian-dpkg-copyright-test-conf.pl
AUTHOR
Dominique Dumont, (ddumont at cpan dot org)