NAME

Log::Munger::RulesTest - Runs a rule file's own tests and lints it for the usual mistakes.

VERSION

Version 0.0.1

SYNOPSIS

use Log::Munger::RulesTest;

my $results = Log::Munger::RulesTest->test( 'file' => 'postfix' );

if ( defined( $results->{'fatal'} ) ) {
    print 'File could not be loaded... ' . $results->{'fatal'} . "\n";
} else {
    print 'Errors:'   . "\n" . join( "\n", @{ $results->{'errors'} } )   . "\n\n";
    print 'Warnings:' . "\n" . join( "\n", @{ $results->{'warnings'} } ) . "\n\n";
}

A rule file carries its own tests. Each primitive under vars has positive and negative cases under vars_tests, each rule has a tests block naming strings that should and should not match, and each decompose entry has its own tests. This module is what runs all of that, along with a lint pass over every resolved var.

A rule's test case may also name the program the line arrived under, which is what puts the rule's gate under test alongside its patterns:

tests:
  positive:
    # the gate has to let dnsmasq-dhcp through before the patterns are tried
    - string: 'DHCPACK(eth0) 192.0.2.50 aa:bb:cc:dd:ee:ff myhost'
      program: 'dnsmasq-dhcp'
      result:
        dnsmasq_dhcp_type: 'DHCPACK'
        dnsmasq_dhcp_iface: 'eth0'
        dnsmasq_dhcp_ip: '192.0.2.50'
        dnsmasq_dhcp_mac: 'aa:bb:cc:dd:ee:ff'
        dnsmasq_dhcp_hostname: 'myhost'
  negative:
    # a plain string is a line no pattern may match, whatever the program
    - 'this is not a dnsmasq line at all'
    # naming a program instead says the gate must not let this daemon in,
    # even though the line itself is one the patterns handle
    - string: 'DHCPACK(eth0) 192.0.2.50 aa:bb:cc:dd:ee:ff myhost'
      program: 'dhcpd'

Naming a program is worth the line it costs. A wrong pattern loses one message type; a wrong gate loses the daemon, and every pattern test still passes while it does. Gates are checked against a record holding nothing but PROGRAM, which is what every gate any rule file has needed keys on.

A case may also carry a numeric list, naming the fields the file's convert has to have turned into numbers:

- string: 'Maximum number of concurrent DNS queries reached (max: 150)'
  program: 'dnsmasq'
  numeric:
    - dnsmasq_max_queries
  result:
    # captures are compared as the strings they are, before any conversion
    dnsmasq_max_queries: '150'

result and numeric are asking different questions of the same case, which is why both can name the same field. result compares the raw capture, which is the string '150'. numeric runs decompose and convert over a copy of those captures and asks what '150' became -- so a field a decompose produced can be listed too, which is the case worth covering most: a kv blob split into fields, one of which a convert then coerces.

What numeric checks is how perl is holding the value, not what it looks like. Asking looks_like_number would pass on the captured string whether the convert ran or not, which is the one thing the case was written to find out.

An enriched map says what the whole field set looks like at the same point, which is what puts a decompose under test as the rule really uses it:

- string: 'neti : TTY=pts/0 ; PWD=/home/neti ; USER=root ; COMMAND=/bin/ls -la'
  program: 'sudo'
  result:
    # the raw captures: one blob, not yet split
    sudo_user: 'neti'
    sudo_kv: 'TTY=pts/0 ; PWD=/home/neti ; USER=root ; COMMAND=/bin/ls -la'
  enriched:
    # and what a consumer receives
    sudo_user: 'neti'
    sudo_TTY: 'pts/0'
    sudo_PWD: '/home/neti'
    sudo_USER: 'root'
    sudo_COMMAND: '/bin/ls -la'

A decompose entry's own tests feed it a hand-written input, which says the entry works but not that anything is wired to it: rename the capture it reads and those tests still pass while the rule quietly stops splitting. enriched is compared exactly in both directions, so the missing fields are caught.

Nothing here needs live log data or a running system, so it is cheap enough to wire into CI. That is what log_munger test_all does, and it exits non-zero if any file reports an error. log_munger test_rule_file -f <file> does the same thing for one file and dumps the whole result as YAML.

METHODS

test

Runs a rule file's tests and lints it, gathering everything that went wrong rather than stopping at the first problem.

Give it either file or hash, not both and not neither. Beyond that, it reports rather than dies: a rule file bad enough that it will not even load comes back as a fatal, not an exception.

The checks are:

• Every vars_tests entry. The var is spliced into the entry's test_template, then each positive case has to match with TEST capturing the expected result, and no negative case may match with TEST captured. A negative that matches without capturing TEST is a warning, not an error.

• Every resolved var, linted for leftover grok %{...}, named captures Perl will not accept, embedded newlines, and anything that will not compile.

• Every rule, compiled exactly as Log::Munger::LogProcessor compiles it, then run against its tests. Positive strings have to match with the expected captures and negative strings have to match nothing. A pattern that looks like a var reference but names no existing var is flagged, since the engine would silently treat it as an inline regexp.

• Every rule's gate, for the test cases that name a program. A positive case naming one requires the gate to accept it before the patterns are tried; a negative case naming one passes if the gate refuses it or no pattern matches. A gated rule no case names a program for is a warning.

• Every rule's convert, for the test cases carrying a numeric list. The case's captures are run through decompose and convert on a copy, and each field named has to come out held as a number rather than as the string the pattern captured. A rule that converts a field to a number and lists none is a warning.

• Every rule's decompose, for the test cases carrying an enriched map. The case's captures are run through decompose and convert on a copy, and the whole field set is compared exactly. A rule whose decompose fires for one of its own tests and says nothing about the result is a warning.

• Every decompose entry, rule level and file level, applied on its own to its tests input and checked against the expected output.

• A file level convert map, for types that are not recognised.

Anything that would produce wrong output is an error. Anything merely worth knowing, such as a var or rule with no tests at all, is a warning.

- file :: The file to load and test. Either a bare name resolved through the
    search path, such as "postfix", or a path.
    Default :: undef

- hash :: An already loaded rules hash ref to test instead of a file.
    Default :: undef

Returns a hash ref:

- fatal :: The reason the file could not be loaded, or undef if it loaded.
    When this is set the other two are empty, since nothing could be tested.

- errors :: Array ref of strings, each naming a problem that would produce
    wrong output. Empty if there were none.

- warnings :: Array ref of strings, each naming something worth knowing that
    is not itself a failure. Empty if there were none.

Every message is prefixed with where the problem is, written as a path into the rule file, so .rules.3.tests.positive.0 is the first positive test of the fourth rule.

Dies only if neither file nor hash was given, if both were, or if either is of the wrong type.

my $results = Log::Munger::RulesTest->test( 'file' => 'postfix' );
my $results = Log::Munger::RulesTest->test( 'hash' => $rules_hash );

cpanm Log::Munger

perl -MCPAN -e shell
install Log::Munger