/ 
Regexp-Debugger-0.002007
River stage one • 5 direct dependents • 8 total dependents
/ Regexp::Debugger

NAME

Regexp::Debugger - Visually debug regexes in-place

VERSION

This document describes Regexp::Debugger version 0.002007

SYNOPSIS

use Regexp::Debugger;

DESCRIPTION

When you load this module, any regex in the same lexical scope will be visually (and interactively) debugged as it matches.

INTERFACE

The module itself provides no API. You load it and the debugger is automatically activated in that lexical scope.

The debugger offers the following commands:

?

: Print a help message listing these commands

s

: Step forward (stepping into any named subpattern calls)

n

: Step forward (stepping over any named subpattern calls)

-

: Step backward (stepping into any named subpattern calls)

p

: Step backward (stepping over any named subpattern calls)

r

: Continue forward until the end of the current (sub)pattern

m

: Continue forward to the next regex component that matches

M

: Continue forward to the next regex component that matches in the current (sub)pattern (i.e. silently stepping over any named subpattern calls)

f

: Continue forward to the next regex component that fails to match something

F

: Continue forward to the next regex component that fails to match something in the current (sub)pattern (i.e. silently stepping over any named subpattern calls)

c

: Continue forward until the entire regex matches or completely backtracks

C

: Continue forward until the entire regex matches or completely backtracks, silently stepping over any named subroutine calls

R

: Rewind to the start of the entire match

<RETURN>/<ENTER>

: Repeat the previous command

v

: Switch to regex/string visualization mode

h

: Switch to heatmapped visualization mode

e

: Switch to the event log

j

: Switch to the underlying JSON data

d

: Describe each component of the regex in detail

V
H
E
J
D

: Take a snapshot of the corresponding display mode.

When prompted for a filename:

<RET>

...prints the snapshot to the terminal

<TAB>

...prints the snapshot to a file named "./rxrx_DISPLAY_MODE_TIMESTAMP"

Anything else

...prints the snapshot to that file

q

: Quit the debugger and finish matching this regex without any further visualization. The program continues to execute and other regexes may still be debugged.

x

: Exit the debugger and the entire program immediately.

CONFIGURATION

You can configure the debugger by setting up a .rxrx file in in the current directory or in your home directory. This configuration consists of single-line key:value pairs (everything else in the file is silently ignored).

Display mode configuration

If the 'display' key is specified, the debugger starts in that mode. The four available modes are:

# Show dynamic visualization of matching (the default)...
display : visual

# Show dynamic heatmap visualization of matching...
display : heatmap

# Show multi-line matching event log...
display : events

# Show JSON encoding of matching process...
display : JSON

Whitespace display configuration

Normally, the debugger compacts whitespaces in the regex down to a single space character, but you can configure that with the show_ws key:

# Compact whitespace and comments to a single space (the default)...
show_ws : compact

# Compact whitespace, but show comments, newlines (\n), and tabs (\t)...
show_ws : visible

# Don't compact whitespace, and show newlines and tabs as \n and \t...
show_ws : original

Colour configuration

The following keys reconfigure the colours with which the debugger displays various information:

Colours for debugging information

  • try_col

    The colour in which attempts to match part of the regex are reported

  • match_col

    The colour in which successful matches of part of the regex are reported

  • fail_col

    The colour in which unsuccessful matches of part of the regex are reported

  • ws_col

    The colour in which special characters (such as "\n", "\t", "\e", etc.) are reported (as single letters: 'n', 't', 'e', etc.)

  • info_col

    The colour in which other information is reported

Colours for regex descriptions

  • desc_regex_col

    The colour in which components of the regex are displayed

  • desc_text_col

    The colour in which descriptions of regex components are displayed

  • desc_sep_col

    The colour in which separators between component descriptions are displayed.

Colours for heatmaps

Any key that starts with heatmap... is treated as a specifier for an equal part of the total range of each heatmap.

These names are sorted (numerically, if possible; otherwise alphabetically) and the corresponding values are then used to display equal percentiles from the heatmap.

For example (using numeric sorting):

heatmap_0_colour      : cyan   on_black   #  0-33rd  percentile
heatmap_50_colour     : yellow on_black   # 34-66th  percentile
heatmap_100_colour    : red    on_black   # 67-100th percentile

Or, equivalently (using alphabetic sorting):

heatmap_infrequent    : cyan   on_black   #  0-33rd  percentile
heatmap_more_frequent : yellow on_black   # 34-66th  percentile
heatmap_very_frequent : red    on_black   # 67-100th percentile

Colour specifications

The colour values that may be used in any of the above colour specifications are any combination of the following (i.e. the colour specifiers supported by the Term::ANSIColor module):

clear           reset             bold            dark
faint           underline         underscore      blink
reverse         concealed

black           red               green           yellow
blue            magenta           cyan            white
bright_black    bright_red        bright_green    bright_yellow
bright_blue     bright_magenta    bright_cyan     bright_white

on_black        on_red            on_green        on_yellow
on_blue         on_magenta        on_cyan         on_white
on_bright_black on_bright_red     on_bright_green on_bright_yellow
on_bright_blue  on_bright_magenta on_bright_cyan  on_bright_white

The default colour configurations are:

try_col    :  bold magenta  on_black
match_col  :  bold cyan     on_black
fail_col   :       yellow   on_red
ws_col     :  bold blue     underline
info_col   :       white    on_black

desc_regex_col  :  white    on_black
desc_text_col   :  cyan     on_black
desc_sep_col    :  blue     on_black underline

heatmap__20th_percentile  :  white   on_black
heatmap__40th_percentile  :  cyan    on_blue
heatmap__60th_percentile  :  blue    on_cyan
heatmap__80th_percentile  :  red     on_yellow
heatmap_100th_percentile  :  yellow  on_red

Output configuration

Normally Regexp::Debugger sends its visualizations to the terminal and expects input from the same device.

However, you can configure the module to output its information (in standard JSON format) to a nominated file instead, using the 'save_to' option:

save_to : filename_to_save_data_to.json

Data saved in this way may be re-animated using the rxrx utility, or by calling Regexp::Debugger::rxrx() directly. (See: "COMMAND-LINE DEBUGGING" for details).

Configuration API

You can also configure the debugger on a program-by-program basis, by passing any of the above key/value pairs when the module is loaded.

For example:

use Regexp::Debugger  fail => 'bold red',  whitespace => 'compact';

Note that any configuration specified in the user's .rxrx file is overridden by an explicit specification of this type.

The commonest use of this mechanism is to dump regex debugging information from an non-interactive program:

use Regexp::Debugger  save_to => 'regex_debugged.json';

Note that, when 'save_to' is specified within a program, the value supplied does not have to be a string specifying the filename. You can also provide an actual filehandle (or equivalent). For example:

use Regexp::Debugger save_to => IO::Socket::INET->new(
                                    Proto     => "tcp",
                                    PeerAddr  => 'localhost:666',
                                );

COMMAND-LINE DEBUGGING

The module provides a non-exported subroutine (rxrx()) that implements a useful command-line regex debugging utility.

The utility can be invoked with:

perl -MRegexp::Debugger -E 'Regexp::Debugger::rxrx\(@ARGV\)'

which is usually aliased in the shell to rxrx (and will be referred to by that name hereafter).

Regex debugging REPL

When called without any arguments, rxrx initiates a simple REPL that allows the user to type in regexes and strings and debug matches between them:

  • Any line starting with a / is treated as a new regex to match with. The closing / may be omitted. If the closing / is supplied, any one or more of the following flags may be specified immediately after it: x, i, m, s, a, u, d, l.

  • Any line starting with a +/ is treated as the first line of a new multi- line regex to match with. Subsequent lines are added to the regex until the closing / is encountered. Any one or more of the following flags may be specified immediately after the closing /: x, i, m, s, a, u, d, l.

  • Any line starting with a ' or " is treated as a new string to match against. The corresponding closing delimiter may be omitted.

  • Any line beginning with m causes the REPL to match the current regex against the current string, visualizing the match in the usual way.

  • Any line beginning with g causes the REPL to exhaustively match the current regex against the current string (i.e. as if the regex had a /g flag), visualizing all the matches in the usual way.

  • Any line beginning with d causes the REPL to display a detailed decomposition and explanation of the current regex.

  • Any line beginning with q or x causes the REPL to quit and exit.

  • Any line beginning with ? invokes the help listing for the REPL.

If the IO::Prompter module (version 0.004 or later) is available, the input process remembers its history, which you can recall by typing CTRL-R. Repeated CTRL-R's step successively backwards through earlier inputs. CTRL-N steps successfully forward again. You can then use CTRL-B/CTRL-F/CTRL-A/CTRL-E to move the cursor around the line of recalled input, to delete or insert characters. This is useful for modifying and retrying a recently entered regex or string.

Debugging regexes from a dumped session

When called with a filename, rxrx first checks whether the file contains a JSON dump of a previous debugging, in which case it replays the visualization of that regex match interactively.

This is useful for debugging non-interactive programs where the 'save_to' option was used (see "Output configuration" and "Configuration API").

In this mode, all the features of the interactive debugger (as listed under "INTERFACE") are fully available: you can step forwards and backwards through the match, skip to the successful submatch or a breakpoint, swap visualization modes, and take snapshots.

Wrap-around regex debugging

When called with the name of a file that does not contain a JSON dump, rxrx attempts to execute the file as a Perl program, with Regexp::Debugger enabled at the top level. In other words:

rxrx prog.pl

is a convenient shorthand for:

perl -MRegexp::Debugger prog.pl

LIMITATIONS

/x-mode comments

Due to limitations in the Perl overload::constant() mechanism, the current implementation cannot always distinguish whether a regex has an external /x modifier (and hence, what whitespace and comment characters mean). Whitespace is handled correctly in almost all cases, but comments are sometimes not.

When processing a # comment to end of line within a regex, the module currently assumes a /x is in effect at start of the regex (unless that assumption causes the regex to fail to compile). This will sometimes cause erroneous behaviour if an unescaped # is used in a non-/x regex.

Unfortunately, this limitation is unlikely to be fully removed in a future release, unless an additional flag-detection mechanism is added to overload::constant().

Note, however, that this limitation does not affect the handling of comments in (?x:...) blocks or of literal # in (?-x:...) blocks within a regex. These are always correctly handled, which means that explicitly using either of these blocks is a reliable workaround. Alternatively, there is no problem if you always use the /x modifier on every debugged regex (for example, via use re '/x'). Nor if you explicitly escape every literal # (i.e. write it as \#).

As regards whitespace, the one case where the current implementation does not always correctly infer behaviour is where whitespace is used to separate a repetition qualifier from the atom it qualifies in a non-/x regex, such as:

/ x + /

Because the module defaults to assuming that regexes always have /x applied, this is always interpreted as:

/\ x+\ /x

rather than what it really is, namely:

/\ x\ +\ /

The most reliable workaround for the time being is either to always use /x on any regex, or never to separate repetition qualifiers from their preceding atoms.

Multiple 'save_to' with the same target

At present, making the same file the target of two successive save_to requests causes the second JSON data structure to overwrite the first.

This limitation will be removed in a subsequent release (but this will certainly involve a small change to the structure of the JSON data that is written, even when only one save_to is specified).

Variable interpolations

The module handles the interpolation of strings correctly, expanding them in-place before debugging begins.

However, it currently does not correctly handle the interpolation of qr'd regexes. That is, this:

use Regexp::Debugger;

my $ident = qr{ [^\W\d]\w* }x;      # a qr'd regex...

$str =~ m{ ($ident) : (.*) }xms;    # ...interpolated into another regex

does not work correctly...and usually will not even compile.

It is expected that this limitation will be removed in a future release, however it may only be possible to fix the problem for more recent versions of Perl (i.e. 5.14 and later) in which the regex engine is re-entrant.

DIAGNOSTICS

Odd number of configuration args after "use Regexp::Debugger"

The module expects configuration arguments (see "Configuration API") to be passed as key => value pairs. You passed something else.

Unknown 'show_ws' option: %s

The only valid options for the 'show_ws' configuration option are 'compact', 'visible', or 'original'. You specified something else (or misspelled one of the above).

Unknown 'display' option: %s

The only valid options for the 'display' configuration option are 'visual' or 'heatmap' or 'events' or 'JSON'. You specified something else (or misspelled one of the above).

Invalid 'save_to' option: %s (%s)

The value associated with the 'save_to' option is expected to be a filehandle opened for writing, or else a string containing the name of a file that can be opened for writing. You either passed an unopened filehandle, an unwritable filename, or something that wasn't a plausible file. Alternatively, if you passed a filepath, was the directory not accessible to, or writeable by, you?

Possible typo in %s

Prior to executing each regex, the module checks for common regex problems that can be detected statically. For example, it looks for the two most common typos made when defining and calling independent subpatterns. Namely: writing (<NAME>...) instead of (?<NAME>...) and (&SUBPAT) instead of (?&SUBPAT)

To silence these warnings, just fix the typos.

Or, if the constructs are intentional, change (<NAME>...) to (\<NAME>...) and (&SUBPAT) to (\&SUBPAT)

DEPENDENCIES

This module only works with Perl 5.10.1 and later.

The following modules are used when available:

Term::ANSIColor

Text colouring only works if this module can be loaded. Otherwise, all output will be monochromatic.

Win32::Console::ANSI

Under Windows, text colouring also requires that this module can be loaded. Otherwise, all output will be monochromatic.

File::HomeDir

If it can't find a useful value for $ENV{HOME}, Regexp::Debugger attempts to use this module to determine the user's home directory, in order to search for a .rxrx config file.

JSON::XS
JSON
JSON::DWIW
JSON::Syck

JSON output (i.e. for the 'save_to' option) is only possible if one of these modules can be loaded. Otherwise, all JSON output will default to an empty {}.

Term::ReadKey

Single-character interactions only work if this module can be loaded. Otherwise, all command interactions will require a <RETURN> after them.

Time::HiRes

Autogenerated timestamps (e.g. for snapshots) will only be sub-second accurate if this module can be loaded. Otherwise, all timestamps will only be to the nearest second.

INCOMPATIBILITIES

None reported, but this module will almost certainly not play nicely with any other that modifies regexes using overload::constant.

BUGS AND LIMITATIONS

No bugs have been reported.

Please report any bugs or feature requests to bug-regexp-debugger@rt.cpan.org, or through the web interface at http://rt.cpan.org.

AUTHOR

Damian Conway <DCONWAY@CPAN.org>

LICENCE AND COPYRIGHT

Copyright (c) 2011-2012, Damian Conway <DCONWAY@CPAN.org>. All rights reserved.

This module is free software; you can redistribute it and/or modify it under the same terms as Perl itself. See perlartistic.

DISCLAIMER OF WARRANTY

BECAUSE THIS SOFTWARE IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY FOR THE SOFTWARE, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES PROVIDE THE SOFTWARE "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE SOFTWARE IS WITH YOU. SHOULD THE SOFTWARE PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING, REPAIR, OR CORRECTION.

IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR REDISTRIBUTE THE SOFTWARE AS PERMITTED BY THE ABOVE LICENCE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE USE OR INABILITY TO USE THE SOFTWARE (INCLUDING BUT NOT LIMITED TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD PARTIES OR A FAILURE OF THE SOFTWARE TO OPERATE WITH ANY OTHER SOFTWARE), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.