NAME
Rosetta::Validator - A common comprehensive test suite to run against all Engines
DEPENDENCIES
Perl Version: 5.006
Standard Modules: none
Nonstandard Modules:
Rosetta 0.38COPYRIGHT AND LICENSE
This file is part of the Rosetta database portability library.
Rosetta is Copyright (c) 1999-2004, Darren R. Duncan. All rights reserved. Address comments, suggestions, and bug reports to perl@DarrenDuncan.net, or visit "http://www.DarrenDuncan.net" for more information.
Rosetta is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License (GPL) version 2 as published by the Free Software Foundation (http://www.fsf.org/). You should have received a copy of the GPL as part of the Rosetta distribution, in the file named "LICENSE"; if not, write to the Free Software Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA.
Linking Rosetta statically or dynamically with other modules is making a combined work based on Rosetta. Thus, the terms and conditions of the GPL cover the whole combination. As a special exception, the copyright holders of Rosetta give you permission to link Rosetta with independent modules, regardless of the license terms of these independent modules, and to copy and distribute the resulting combined work under terms of your choice, provided that every copy of the combined work is accompanied by a complete copy of the source code of Rosetta (the version of Rosetta used to produce the combined work), being distributed under the terms of the GPL plus this exception. An independent module is a module which is not derived from or based on Rosetta, and which is fully useable when not linked to Rosetta in any form.
Any versions of Rosetta that you modify and distribute must carry prominent notices stating that you changed the files and the date of any changes, in addition to preserving this original copyright notice and other credits. Rosetta is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
While it is by no means required, the copyright holders of Rosetta would appreciate being informed any time you create a modified version of Rosetta that you are willing to distribute, because that is a practical way of suggesting improvements to the standard version.
SYNOPSIS
This example demonstrates how Rosetta::Validator could be used in the standard test suite for an Engine module; in fact, this example is a simplified version of the actual t/*.t file for the Rosetta::Engine::Generic module.
#!/usr/bin/perl
use strict; use warnings;
use Rosetta::Validator;
BEGIN { $| = 1; }
my $test_num = 0;
sub print_result {
my ($result) = @_;
$test_num ++;
my ($feature_key, $feature_status, $feature_desc_msg, $val_error_msg, $eng_error_msg) =
@{$result}{'FEATURE_KEY', 'FEATURE_STATUS', 'FEATURE_DESC_MSG', 'VAL_ERROR_MSG', 'ENG_ERROR_MSG'};
my $result_str =
($feature_status eq 'PASS' ? "ok $test_num (PASS)" :
$feature_status eq 'FAIL' ? "not ok $test_num (FAIL)" :
"ok $test_num (SKIP)").
" - $feature_key - ".object_to_string( $feature_desc_msg ).
($val_error_msg ? ' - '.object_to_string( $val_error_msg ) : '').
($eng_error_msg ? ' - '.object_to_string( $eng_error_msg ) : '');
print "$result_str\n";
}
sub object_to_string {
my ($message) = @_;
if( ref($message) and UNIVERSAL::isa( $message, 'Rosetta::Interface' ) ) {
$message = $message->get_error_message();
}
if( ref($message) and UNIVERSAL::isa( $message, 'Locale::KeyedText::Message' ) ) {
my $translator = Locale::KeyedText->new_translator( ['Rosetta::Engine::Generic::L::',
'Rosetta::Validator::L::', 'Rosetta::L::', 'SQL::Routine::L::'], ['en'] );
my $user_text = $translator->translate_message( $message );
unless( $user_text ) {
return( "internal error: can't find user text for a message: ".
$message->as_string()." ".$translator->as_string() );
}
return( $user_text );
}
return( $message ); # if this isn't the right kind of object
}
eval {
my $total_tests_per_invoke = Rosetta::Validator->total_possible_tests();
print "1..$total_tests_per_invoke\n";
my $config_filepath = shift( @ARGV ); # set from first command line arg; '0' means none
my $rh_config = {};
if( $config_filepath ) {
$rh_config = do $config_filepath;
}
my $trace_to_stdout = shift( @ARGV ) ? 1 : 0; # set from second command line arg
my $validator = Rosetta::Validator->new();
$trace_to_stdout and $validator->set_trace_fh( \*STDOUT );
$validator->set_engine_name( 'Rosetta::Engine::Generic' );
$validator->set_engine_config_options( $rh_config );
$validator->perform_tests();
foreach my $result (@{$validator->get_test_results()}) {
print_result( $result );
}
};
$@ and print "TESTS ABORTED: ".object_to_string( $@ ); # errors in test suite itself, or core modules
1;Here is the content of an example configuration file that one can use with said test script:
my $rh_config = {
'product_code' => 'MySQL',
'local_dsn' => 'test',
'login_name' => 'jane',
'login_pass' => 'pwd',
};DESCRIPTION
The Rosetta::Validator Perl 5 module is a common comprehensive test suite to run against all Rosetta Engines. You run it against a Rosetta Engine module to ensure that the Engine and/or the database behind it implements the parts of the Rosetta API that your application needs, and that the API is implemented correctly. Rosetta::Validator is intended to guarantee a measure of quality assurance (QA) for Rosetta, so your application can use the database access framework with confidence of safety.
Alternately, if you are writing a Rosetta Engine module yourself, Rosetta::Validator saves you the work of having to write your own test suite for it. You can also be assured that if your module passes Rosetta::Validator's approval, then your module can be easily swapped in for other Engine modules by your users, and that any changes you make between releases haven't broken something important.
Rosetta::Validator would be used similarly to how Sun has an official validation suite for Java Virtual Machines to make sure they implement the official Java specification.
For reference and context, please see the FEATURE SUPPORT VALIDATION documentation section in the core "Rosetta" module.
Note that, as is the nature of test suites, Rosetta::Validator will be getting regular updates and additions, so that it anticipates all of the different ways that people want to use their databases. This task is unlikely to ever be finished, given the seemingly infinite size of the task. You are welcome and encouraged to submit more tests to be included in this suite at any time, as holes in coverage are discovered.
CONSTRUCTOR FUNCTIONS AND METHODS
This function/method is stateless and can be invoked off of either this module's name or an existing module object, with the same result.
new()
my $validator = Rosetta::Validator->new();
my $validator2 = $validator->new();This "getter" function/method will create and return a single Rosetta::Validator (or subclass) object. All of this object's properties are set to default undefined values; you will at the very least have to set engine_name() afterwards.
STATIC CONFIGURATION PROPERTY ACCESSOR METHODS
These methods are stateful and can only be invoked from this module's objects. This set of properties are generally set once at the start of a Validator object's life and aren't changed later (though they can be), since they are generally static configuration data.
get_trace_fh()
my $fh = $validator->get_trace_fh();This "getter" method returns by reference the writeable Perl trace file handle property of of this object, if it is set. When set, details of what Rosetta::Validator and the Engines that it tests are doing will be written to the file handle; to turn off the tracing, just clear the property. This class does not open or close the file; your external code must do that.
clear_trace_fh()
$validator->clear_trace_fh();This "setter" method clears the trace file handle property of this object, if it was set, thereby turning off any tracing output.
set_trace_fh( NEW_FH )
$validator->set_trace_fh( \*STDOUT );This "setter" method sets or replaces the trace file handle property of this object to a new writeable Perl file handle, provided in NEW_FH, so any subsequent tracing output is sent there.
get_engine_name()
my $engine_name = $validator->get_engine_name();This "getter" method returns this object's "engine name" character string property. This property defines the name of the Rosetta Engine module that you want this Validator object to test.
clear_engine_name()
$validator->clear_engine_name();This "setter" method clears the "engine name" property.
set_engine_name( ENGINE_NAME )
$validator->set_engine_name( 'Rosetta::Engine::Generic' );This "setter" method sets or replaces the "engine name" property.
get_engine_config_option( ECO_NAME )
This "getter" method will return one value for this object's "engine configuration options" property, which matches the ECO_NAME argument. This property sets up operational parameters for the Rosetta Engine module, where it takes options; examples being the name of the data source name or the authorization identifier to use.
get_engine_config_options()
This "getter" method will fetch all of this object's "engine configuration options", returning them in a Hash ref.
clear_engine_config_option( ECO_NAME )
This "setter" method will clear one "engine configuration options" value, which matches the ECO_NAME argument.
clear_engine_config_options()
This "setter" method will clear all of the "engine configuration options".
set_engine_config_option( ECO_NAME, ECO_VALUE )
$validator->set_engine_config_option( 'local_dsn', 'test' );
$validator->set_engine_config_option( 'dbi_driver', 'mysql' );This "setter" method will set or replace one "engine configuration options" value, which matches the ECO_NAME argument, giving it the new value specified in ECO_VALUE.
set_engine_config_options( EC_OPTS )
$validator->set_engine_config_options( { 'login_name' => 'jane', 'login_pass' => 'pawd' } );This "setter" method will set or replace multiple "engine configuration options" values, whose names and values are specified by keys and values of the EC_OPTS hash ref argument.
DYNAMIC STATE MAINTENANCE PROPERTY ACCESSOR METHODS
These methods are stateful and can only be invoked from this module's objects. Each of these contains the results of a test performing method call.
get_test_results()
This "getter" method returns a new array ref having a copy of this object's "test results" array property. This property is explicitely emptied by external code, by invoking the clear_test_results() method, prior to that code requesting that we perform one or more tests. As soon as said tests are performed, external code reads this array's values using the get_test_results() method. Each element of the "test results" array is a hash ref containing these 5 elements: 1. 'FEATURE_KEY'. 2. 'FEATURE_STATUS'; one of 'SKIP' (test was not run at all), 'PASS' (test was run and passed), 'FAIL' (test was run and failed); for the first 2, the Engine said it had support for the feature, and for the third it said that it did not. 3. 'FEATURE_DESC_MSG'; a Locale::KeyedText::Message (LKT) object that is the Validator module's description of what DBMS/Engine feature is being tested. 4. 'VAL_ERROR_MSG'; a LKT object that is set when 'FEATURE_STATUS' is 'FAIL'; this is the Validator module's own Error Message, if a test failed; this is made for a failure regardless of whether the Engine threw its own exception. 5. 'ENG_ERROR_MSG'; a LKT object that is the Error Message that the Rosetta Interface or Engine threw, if any.
clear_test_results()
This "setter" method empties this object's "positional host param map array" array property. See the previous method's documentation for when to use it.
TEST PERFORMING METHODS
These methods are stateful and can only be invoked from this module's objects. These methods comprise the core of the Rosetta::Validator module, and are what actually perform the tests on the Rosetta Engines.
perform_tests()
This method will instantiate a new Rosetta Interface tree, and a SQL::Routine Container, populate the latter, invoke the former, saying to use the Engine specified in the "engine name" property, try all sorts of database actions, and record the results in the "test results" property, and then destroy the Rosetta and SQL::Routine objects.
INFORMATION FUNCTIONS AND METHODS
These "getter" functions/methods return Rosetta::Validator constant information which is useful when interpreting or using other aspects of the class.
total_possible_tests()
This method returns an integer that says how many elements are supposed to be in the "test results" array after perform_tests() is run; it should be equal to the number of skips + passes + fails. You can use this number at the start of your test script when declaring the 1..N total number of tests that will be considered, and you can compare that to the actual results.
BUGS
This module is currently in pre-alpha development status, meaning that some parts of it will be changed in the near future, perhaps in incompatible ways.
SEE ALSO
perl(1), Rosetta, SQL::Routine, Locale::KeyedText, Rosetta::Engine::Generic.