NAME

Log::Munger::Degrok - Rewrites grok templating into the form Log::Munger uses.

VERSION

Version 0.0.1

SYNOPSIS

use Log::Munger::Degrok;

print Log::Munger::Degrok->string( 'string' => 'client=%{IP:client_ip}' ) . "\n";
# client=(?<client_ip>[% IP %])

print Log::Munger::Degrok->file( 'file' => '/path/to/patterns.grok' );

print Log::Munger::Degrok->grok2rules( 'file' => '/path/to/patterns.grok', 'includes' => ['base'] );

Grok references a named pattern as %{TEMPLATE} and captures one as %{TEMPLATE:VAR}. Log::Munger uses Template for the first and Perl's own named capture syntax for the second, so those become [% TEMPLATE %] and (?<VAR>[% TEMPLATE %]). This module does that rewrite.

Only the delimiters change. The primitive names in base.yaml deliberately track grok's, so a pattern that referenced %{IP} before goes on referencing the same thing afterwards, and most patterns work straight after conversion. What the rewrite cannot fix is a capture name Perl will not accept, since grok is happy with names like src-ip and Perl is not. Those come across unchanged and are caught later, either at load time or by Log::Munger::RulesTest.

METHODS

string

Rewrites the grok templating in a string and returns the result.

Every %{...} reference is rewritten, however many there are, and anything that is not one is left exactly as it was.

- string :: The string to convert. Required.
    Default :: undef

Returns the rewritten string. Dies only if string is undef.

my $results;
my $some_string = 'client=%{IP:client_ip} user=%{USERNAME:user}';
eval { $results = Log::Munger::Degrok->string( 'string' => $some_string ); };
if ($@) {
    die( 'Failed to process the string "' . $some_string . '"... ' . $@ );
}
print $results . "\n";
# client=(?<client_ip>[% IP %]) user=(?<user>[% USERNAME %])

file

"string" applied to a whole file, a line at a time.

The file's structure is untouched. Only the %{...} references change, so a grok patterns file comes back as the same file with Log::Munger templating in it. To turn one into an actual rule file, use "grok2rules" instead.

- file :: The file to convert. Required.
    Default :: undef

Returns the whole rewritten file as a single string, line endings and all. Dies if file is undef or the file cannot be read.

my $results;
my $file = '/tmp/patterns.grok';
eval { $results = Log::Munger::Degrok->file( 'file' => $file ); };
if ($@) {
    die( 'Failed to process "' . $file . '"... ' . $@ );
}
print $results;

grok2rules

Turns a grok patterns file into the skeleton of a Log::Munger rule file.

A grok patterns file is a list of NAME regexp lines. Each one becomes a var: plain lines go under vars, and lines that reference other patterns go under vars_templated once they have been through "string". Comments and blank lines are dropped.

Naming what you already have as includes is worth doing. Most grok patterns files start with their own copies of IP, WORD, HOSTNAME and the rest, and without the includes those all come across and shadow the ones in base.yaml. With them, the duplicates are recognised and handled per overwrite instead.

What comes back is a skeleton, not a finished rule file. There is no rules section, no gates, no tests and no enrichment, because none of that exists in a grok patterns file to convert. Writing those is the part still left to do.

- file :: The grok patterns file to convert. Required.
    Default :: undef

- includes :: Rule files to treat as includes. Their vars are loaded so
    names already provided by one can be recognised, and they are written
    into the skeleton's own includes list. The taken value is an array ref.
    Default :: []

- overwrite :: What to do about a name an include already defines.
    Accepted values are:
        - yes :: Take the grok file's version.
        - no_silent :: Keep the include's version, saying nothing.
        - no_warn :: Keep the include's version and warn about it.
        - no_die :: Die on the first one.
    Default :: no_warn

Returns the skeleton as a YAML string, ready to write to a file. Dies if file is undef or unreadable, if overwrite is not one of the four values above, if an include cannot be loaded, or if a line in the patterns file has a name but no regexp after it.

my $results;
my $file = '/tmp/patterns.grok';
eval { $results = Log::Munger::Degrok->grok2rules( 'file' => $file, 'includes' => ['base'] ); };
if ($@) {
    die( 'Failed to process "' . $file . '"... ' . $@ );
}
print $results;