/ 
Parse-Marpa-0.205_006
/ Parse::Marpa::Doc::Diagnostics

NAME

Parse::Marpa::Doc::Diagnostics - Diagnostics

DESCRIPTION

This document covers Marpa methods and method options primarily useful for debugging grammars and parses. Many diagnostics require no knowledge of Marpa's internals to use and interpret. Others require a good deal of internals know-how. The descriptions of individual methods and named arguments will indicate which is which. There's a guide to Marpa internals in the internals document.

Basic techniques

In debugging a grammar, first look at the place where the parse was exhausted. That and inspection of the input and the grammar are oftehn enough to spot the problem.

If that fails, and you don't already have Marpa's warnings option turned on (they're on by default), you probably should turn them on.

If that doesn't help, it is often useful to turn on trace_lex. This tells you which tokens the lexer is looking for and which ones it thinks it found. If the problem is in lexing, trace_lex tells you the whole story. Even if the problem is in the grammar, which tokens the lexer is looking for is a clue to what the recognizer is doing. That is because Marpa uses predictive lexing and only looks for tokens that will result in a successful parse according to the grammar.

If that still doesn't help, but you are able to return values from the parse using the Parse::Marpa::Evaluator::next method, you can run Parse::Marpa::Evaluator::show method after calling next. The show method returns the parse derivation. Next to each rule in the derivation will be the SDFA state involved, for use if you go on into the internals.

Advanced Techniques

Before going deep into the internals, you should look carefully at the output of show_rules and show_symbols to see if anything is clearly not right or not what you expected. Depending on where in the process you're having problems, you might want to turn on some of the more helpful traces. trace_actions will show you the actions as they are being finalized. In an ambiguous parse, trace_evaluation_choices shows the choices Marpa is making. trace_iteration_changes and trace_rules traces the initialization of, and changes in, node values.

For the real "into the Earley sets with gun and camera" stuff, run show_SDFA on the precomputed grammar and show_status on the recognizer. The internals document has example outputs from these methods and outlines how to read them.

OPTIONS

These are Marpa options, valid with the Parse::Marpa::Grammar::new, Parse::Marpa::Grammar::set, or Parse::Marpa::Recognizer::new() methods. Unless otherwise stated, a named argument is valid and useful for all these methods at every point in the parse. Trace output goes to the trace file handle.

academic

The academic option turns off all grammar rewriting. The resulting grammar is useless for parseless. The purpose of the academic argument is testing that Marpa can accurately duplicate examples from textbooks. This is handy for testing Marpa's deepest internals. A recognizer cannot be created from a grammar marked academic. The academic option cannot be set in the recognizer or after the grammar is precomputed.

trace_actions

Traces the actions as they are compiled. Little or no knowledge of Marpa internals required. This option is useless once the recognizer has been created. Setting it after that point will result in a warning.

trace_evaluation_choices

This option traces the choices Marpa make. when Marpa has a choice among more than one rule, link or token. Choices only occur if the grammar is ambiguous. Knowledge of Marpa internals probably needed.

trace_iteration_changes

Traces setting of, and changes in node values. Knowledge of Marpa internals very useful.

trace_iteration_searches

Traces Marpa's exploration of the Earley sets as it is evaluating nodes. Requires knowledge of Marpa internals. Probably not useful except in combination with trace_iteration_changes. trace_iterations turns on both.

trace_iterations

A short hand for setting both trace_iteration_changes and trace_iteration_searches.

trace_lex

A short hand for setting both trace_lex_matches and trace_lex_tries. Very useful, and can be interpreted with limited knowledge of Marpa internals. Because Marpa uses predictive lexing, this can give you an idea of how lexing is working, but also of what the parse engine is looking for. Often the first thing I turn on when I'm debugging a grammar.

trace_lex_matches

Traces every successful match in lexing. Can be interpreted with little knowledge of Marpa internals.

trace_lex_tries

Traces every attempted match in lexing. Can be interpreted with little knowledge of Marpa internals. Usually not useful without trace_lex_matches. trace_lex turns on both.

trace_priorities

Traces the priority setting of each SDFA state. Requires knowledge of Marpa internals.

trace_rules

Traces rules as they are added to the grammar. Useful, but you may prefer the show_rules() method. Doesn't require knowledge of Marpa internals.

Remember, if you are adding rules via the source method option, the other method options take effect after the processing of the source option. As a practical matter, that means that if you must don't set trace_rules in a method call prior to the one with the source option, you will miss the addition of all but a few internally added rules.

trace_values

As each node value is set, prints a trace of the rule and the value. Very helpful and does not require knowledge of Marpa internals.

METHODS

Grammar Methods

Parse::Marpa::inaccessible_symbols(grammar)

Given a precomputed grammar, returns the raw interface names of the inaccessible symbols. The same information is more easily obtained by turning on the warnings option.

Parse::Marpa::show_NFA(grammar)

Given a grammar object, returns a multi-line string listing the states of the NFA with the LR(0) items and transitions for each. Not really helpful for debugging grammars and requires very deep knowledge of Marpa internals.

Parse::Marpa::show_SDFA(grammar)

Given a gramar object, returns a multi-line string listing the states of the SDFA with the LR(0) items, NFA states, and transitions for each. Very useful, but requires knowledge of Marpa internals.

Parse::Marpa::show_accessible_symbols(grammar)

Given a grammar object, returns a one-line string with the raw interface names of the accessible symbols of the grammar, space-separated. Handy for quick comparison tests, but otherwise not very useful.

Parse::Marpa::show_location(message, text, offset)

message must be a string, text a reference to a string, and offset, a character offset within that string. show_location() returns a multi-line string with a header line containing message, the line from text containing offset, and a "pointer" line. The pointer line uses the ASCII "caret" symbol to point to the exact offset.

Parse::Marpa::show_nullable_symbols(grammar)

Given a grammar object, returns a one-line string with the raw interface names of the nullable symbols of the grammar, space-separated. The format is handy for quick comparison tests, but otherwise not very useful.

Parse::Marpa::show_nulling_symbols(grammar)

Given a grammar object, returns a one-line string with the raw interface names of the nulling symbols of the grammar, space-separated. The format is handy for quick comparison tests, but otherwise not very useful.

Parse::Marpa::show_problems(grammar)

Returns a string describing the problems a grammar had in the precomputation phase. Marpa does not immediately throw an exception for many of the precomputation problems because the user usually will want to fix several at a time. If there were no problems, returns a string saying so.

This returned string is the same that Marpa includes in exceptions thrown when the user attempts to compile, or to create a parse from, a grammar with problems.

Parse::Marpa::show_productive_symbols(grammar)

Given a grammar object, returns a one-line string with the raw interface names of the productive symbols of the grammar, space-separated. The format is handy for quick comparison tests, but otherwise not very useful.

Parse::Marpa::show_rules(grammar)

Returns a string listing the rules, each commented as to whether it was nullable, nulling, unproductive, inaccessible, empty or not useful. If a rule had a non-zero priority, that is also shown. Often useful and much of the information requires no knowledge of the Marpa internals to interpret.

Parse::Marpa::show_symbols(grammar)

Returns a string listing the symbols, along with whether they were nulling, nullable, unproductive or inaccessible. Also shown is a list of rules with that symbol on the left hand side, and a list of rules which have that symbol anywhere on the right hand side. Often useful and much of the information requires no knowledge of the Marpa internals to interpret.

Parse::Marpa::unproductive_symbols(grammar)

Given a precomputed grammar, returns the raw interface names of the unproductive symbols. The same information is more easily obtained by turning on the warnings option.

show_status(parse)

This is the central tool for debugging a parse using Marpa internals. Takes a parse object as its argument and returns a multi-line string listing every Earley item in every Earley sets. For each Earley item, any current successor, predecessor, effect, cause, pointer or value is shown. Also shown are lists of all the links and rules in each Earley item, indicating which link or rule is the current choice.

For detailed investigation of a parse, this, the output of trace_lex and listings of the symbols, the rules, and the SDFA states (see show_SDFA()), will usually be everything you need.

Evaluator Method

Parse::Marpa::Evaluator::show

Takes an evaluator object as its argument which has had a value returned by next method, and shows the parse derivation used to produce that value. Very useful. Basic use requires no Marpa internals. It also reports the Earley item and SDFA state at each line of the derivation.