NAME

XML::XQL - A perl module for querying XML tree structures with XQL

SYNOPSIS

use XML::XQL;
use XML::XQL::DOM;

$parser = new XML::DOM::Parser;
$doc = $parser->parsefile ("file.xml");

# Return all elements with tagName='title' under the root element 'book'
$query = new XML::XQL::Query (Expr => "book/title");
@result = $query->solve ($doc);

# Or (to save some typing)
@result = XML::XQL::solve ("book/title", $doc);

DESCRIPTION

The XML::XQL module implements the XQL (XML Query Language) proposal submitted to the XSL Working Group in September 1998. The spec can be found at: http://www.w3.org/TandS/QL/QL98/pp/xql.html Most of the contents related to the XQL syntax can also be found in the XML::XQL::Tutorial that comes with this distribution. Note that XQL is not the same as XML-QL!

The current implementation only works with the XML::DOM module, but once the design is stable and the major bugs are flushed out, other extensions might follow, e.g. for XML::Grove.

XQL was designed to be extensible and this implementation tries to stick to that. Users can add their own functions, methods, comparison operators and data types. Plugging in a new XML tree structure (like XML::Grove) should be a piece of cake.

To use the XQL module, either

use XML::XQL;

use XML::XQL::Strict;

The Strict module only provides the core XQL functionality as found in the XQL spec. By default (i.e. by using XML::XQL) you get 'XQL+', which has some additional features.

See the section 'Additional Features in XQL+' for the differences.

This module is still in development. See the To-do list in XQL.pm for what still needs to be done. Any suggestions are welcome, the sooner these implementation issues are resolved, the faster we can all use this module.

If you find a bug, you would do me great favor by sending it to me in the form of a test case. See the file t/template.t that comes with this distribution.

If you have written a cool comparison operator, function, method or XQL data type that you would like to share, send it to enno@att.com and I will add it to this module.

XML::XQL global functions

solve (QUERY_STRING, INPUT_LIST...)

@result = XML::XQL::solve ("doc//book", $doc);

This is provided as a shortcut for:

$query = new XML::XQL::Query (Expr => "doc//book");
@result = $query->solve ($doc);

defineFunction (NAME, FUNCREF, ARGCOUNT [, ALLOWED_OUTSIDE [, CONST, [QUERY_ARG]]])

Defines the XQL function (at the global level, i.e. for all newly created queries) with the specified NAME. The ARGCOUNT parameter can either be a single number or a reference to a list with numbers. A single number expands to [ARGCOUNT, ARGCOUNT]. The list contains pairs of numbers, indicating the number of arguments that the function allows. The value -1 means infinity. E.g. [2, 5, 7, 9, 12, -1] means that the function can have 2, 3, 4, 5, 7, 8, 9, 12 or more arguments. The number of arguments is checked when parsing the XQL query string.

The second parameter must be a reference to a Perl function or an anonymous sub. E.g. '\&my_func' or 'sub { ... code ... }'

If ALLOWED_OUTSIDE (default is 0) is set to 1, the function or method may also be used outside subqueries in node queries. (See NodeQuery parameter in Query constructor)

If CONST (default is 0) is set to 1, the function is considered to be "constant". See "Constant Function Invocations" for details.

If QUERY_ARG (default is 0) is not -1, the argument with that index is considered to be a 'query parameter'. If the query parameter is a subquery, that returns multiple values, the result list of the function invocation will contain one result value for each value of the subquery. E.g. 'length(book/author)' will return a list of Numbers, denoting the string lengths of all the author elements returned by 'book/author'.

Note that only methods (not functions) may appear after a Bang "!" operator. This is checked when parsing the XQL query string.

XML::XQL::Query methods

The following functions are also available at the query level, i.e. when called on a Query object they only affect this Query and no others:

defineFunction, defineMethod, defineComparisonOperators, 
defineTokenQ, defineTokenQQ

See 'XML::XQL Global functions' for details. Another way to define these features for a particular Query is by passing the appropriate values to the XML::XQL::Query constructor.

solve (INPUT_LIST...)

Note that solve takes a list of nodes which are assumed to be in document order and must belong to the same document. E.g:

$query = new XML::XQL::Query (Expr => "doc//book");
@result = $query->solve ($doc);
@result2 = $query->solve ($node1, $node2, $node3);

XML::XQL::Query constructor

Usage, e.g:

 $query = new XML::XQL::Query(
	Expr => "book/author",
	Func => [ myfunc => \&my_func,		# define 2 functions
		  myfunc2 => \&my_func2 ],
	FuncArgCount => [ myfunc2 => [2, -1] ], # myfunc2 has 2 or more args
	AllowedOutSideSubquery => [ myfunc => 1 ],
	ConstFunc => [ myfunc2 => 1],
	CompareOper => [ mycmp => \&mycmp ],	# define comparison operator
	q => "str");				# use str// as string delim

Expr => STRING: The query expression to be evaluated.
NodeQuery => BOOLEAN: If set to 1, the query is a Node Query as opposed to a Full Query (which is the default.) A node query is a query that is only capable of returning Nodes. A full query is capable of returning Node values and non-Node values. Non-Node values include XML Primitives, element type names, namespace URI's, concatenated text nodes, and node type names. The distinction is significant because node queries may appear as XSL match and select patterns, while full queries have use in other applications. The difference between the two forms of queries is trivial and exists only as constraints on the syntax of node queries. Node queries may contain nested full queries.
Func => [ FUNCNAME => FUNCREF, ...]: Defines one or more functions. FUNCNAME is the name as used in the query expression. FUNCREF can be either a function reference like \&my_func or an anonymous sub. See also: defineFunction
Method => [ FUNCNAME => FUNCREF, ...]: Defines one or more methods. FUNCNAME is the name as used in the query expression. FUNCREF can be either a function reference like \&my_func or an anonymous sub. See also: defineMethod
FuncArgCount => [ FUNCNAME => ARGCOUNT, ...]: Defines the number of arguments for one or more functions or methods. FUNCNAME is the name as used in the query expression. See also: defineFunction and defineMethod
AllowedOutsideSubquery => [ FUNCNAME => BOOLEAN, ...]: Defines whether the specified function or method is allowed outside subqueries. FUNCNAME is the name as used in the query expression. See also: defineFunction and defineMethod
ConstFunc => [ FUNCNAME => BOOLEAN, ...]: Defines whether the function (not method!) is a "constant" function. FUNCNAME is the name as used in the query expression. See "Constant Function Invocations" for a definition of "constant" See also: defineFunction and defineMethod
CompareOper => [ OPERNAME => FUNCREF, ...]: Defines the comparison operator with the specified OPERNAME, e.g. if OPERNAME is "contains", you can use "$contains$" in the query. See also: defineComparisonOperators
q => TOKEN: Defines the q// token. See also: defineTokenQ
qq => TOKEN: Defines the qq// token. See also: defineTokenQQ
Error => FUNCREF: Defines the function that is called when errors occur during parsing the query expression. The default function prints an error message to STDERR.
Debug => FLAGS: Sets the debug level for the Yapp parser that parses the query expression. Default value is 0 (don't print anything). The maximum value is 0x17, which prints a lot of stuff. See the Parse::Yapp manpage for the meaning of the individual bits.
Reserved hash keys: Users may add their own (key, value) pairs to the Query constructor. Beware that the key 'Tree' is used internally.

Additional Features in XQL+

Sequence operators ';' and ';;'

The sequence operators ';' (precedes) and ';;' (immediately precedes) are not in the XQL spec, but are described in 'The Design of XQL' by Jonathan Robie who is one of the designers of XQL. It can be found at http://www.texcel.no/whitepapers/xql-design.html See also the XQL Tutorial for a description of what they mean.

q// and qq// String Tokens

String tokens a la q// and qq// are allowed. q// evaluates like Perl's single quotes and qq// like Perl's double quotes. Note that the default XQL strings do not allow escaping etc., so it's not possible to define a string with both single and double quotes. If 'q' and 'qq' are not to your liking, you may redefine them to something else or undefine them altogether, by assigning undef to them. E.g:

# at a global level - shared by all queries (that don't (re)define 'q')
XML::XQL::defineTokenQ ('k');
XML::XQL::defineTokenQQ (undef);

# at a query level - only defined for this query
$query = new XML::XQL::Query (Expr => "book/title", q => 'k', qq => undef);

From now on k// works like q// did and qq// doesn't work at all anymore.

Query strings can have embedded Comments

For example:

 $queryExpr = "book/title          # this comment is inside the query string
	       [. = 'Moby Dick']"; # this comment is outside

Optional dollar delimiters and case-insensitive XQL keywords

The following XQL keywords are case-insensitive and the dollar sign delimiters may be omitted: $and$, $or$, $not$, $union$, $intersect$, $to$, $any$, $all$, $eq$, $ne$, $lt$, $gt$, $ge$, $le$, $ieq$, $ine$, $ilt$, $igt$, $ige$, $ile$.

E.g. $AND$, $And$, $aNd$, and, And, aNd are all valid replacements for $and$.

Note that XQL+ comparison operators ($match$, $no_match$, $isa$, $can$) still require dollar delimiters and are case-sensitive.

Comparison operator: $match$ or '=~'

E.g. "book/title =~ '/(Moby|Dick)/']" will return all book titles containing Moby or Dick. Note that the match expression needs to be quoted and should contain the // or m// delimiters for Perl.

When casting the values to be matched, both are converted to Text.

Comparison operator: $no_match$ or '!~'

E.g. "book/title !~ '/(Moby|Dick)/']" will return all book titles that don't contain Moby or Dick. Note that the match expression needs to be quoted and should contain the // or m// delimiters for Perl.

When casting the values to be matched, both are converted to Text.

Comparison operator: $isa$

E.g. '//. $isa$ "XML::XQL::Date"' returns all elements for which the value() function returns an XML::XQL::Date object. (Note that the value() function can be overridden to return a specific object type for certain elements and attributes.) It uses expandType to expand XQL primitive type names.

Comparison operator: $can$

E.g. '//. $can$ "swim"' returns all elements for which the value() function returns an object that implements the (Perl) swim() method. (Note that the value() function can be overridden to return a specific object type for certain elements and attributes.)

Function: once (QUERY)

E.g. 'once(id("foo"))' will evaluate the QUERY expression only once per query. Certain query results (like the above example) will always return the same value within a query. Using once() will cache the QUERY result for the rest of the query.

Note that "constant" function invocations are always cached. See also "Constant Function Invocations"

Function: subst (QUERY, EXPR, EXPR [,MODIFIERS, [MODE]])

E.g. 'subst(book/title, "[M|m]oby", "Dick", "g")' will replace Moby or moby with Dick globally ("g") in all book title elements. Underneath it uses Perl's substitute operator s///. Don't worry about which delimiters are used underneath. The function returns all the book/titles for which a substitution occurred. The default MODIFIERS string is "" (empty.) The function name may be abbreviated to "s".

For most Node types, it converts the value() to a string (with xql_toString) to match the string and xql_setValue to set the new value in case it matched. For XQL primitives (Boolean, Number, Text) and other data types (e.g. Date) it uses xql_toString to match the String and xql_setValue to set the result. Beware that performing a substitution on a primitive that was found in the original XQL query expression, changes the value of that constant.

If MODE is 0 (default), it treats Element nodes differently by matching and replacing text blocks occurring in the Element node. A text block is defined as the concatenation of the raw text of subsequent Text, CDATASection and EntityReference nodes. In this mode it skips embedded Element nodes. If a text block matches, it is replaced by a single Text node, regardless of the original node type(s).

If MODE is 1, it treats Element nodes like the other nodes, i.e. it converts the value() to a string etc. Note that the default implementation of value() calls text(), which normalizes whitespace and includes embedded Element descendants (recursively.) This is probably not what you want to use in most cases, but since I'm not a professional psychic... :-)

Function: map (QUERY, CODE)

E.g. 'map(book/title, "s/[M|m]oby/Dick/g; $_")' will replace Moby or moby with Dick globally ("g") in all book title elements. Underneath it uses Perl's map operator. The function returns all the book/titles for which a change occurred.

??? add more specifics

Function: eval (EXPR [,TYPE])

Evaluates the Perl expression EXPR and returns an object of the specified TYPE. It uses expandType to expand XQL primitive type names. If the result of the eval was undef, the empty list [] is returned.

E.g. 'eval("2 + 5", "Number")' returns a Number object with the value 7, and 'eval("%ENV{USER}")' returns a Text object with the user name.

Consider using once() to cache the return value, when the invocation will return the same result for each invocation within a query.

??? add more specifics

Function: new (TYPE [, QUERY [, PAR] *])

Creates a new object of the specified object TYPE. The constructor may have any number of arguments. The first argument of the constructor (the 2nd argument of the new() function) is considered to be a 'query parameter'. See defineFunction for a definition of query parameter. It uses expandType to expand XQL primitive type names.

Method: DOM_nodeType ()

Returns the DOM node type. Note that these are mostly the same as nodeType(), except for CDATASection and EntityReference nodes. DOM_nodeType() returns 4 and 5 respectively, whereas nodeType() returns 3, because they are considered text nodes.

Function wrappers for Perl builtin functions

XQL function wrappers have been provided for most Perl builtin functions. When using a Perl builtin function like "substr" in an XQL+ querry, an XQL function wrapper will be generated on the fly. The arguments to these functions may be regular XQL+ subqueries (that return one or more values) for a query parameter (see generateFunction for a definition.) Most wrappers of Perl builtin functions have argument 0 for a query parameter, except for: chmod (parameter 1 is the query parameter), chown (2) and utime (2). The following funcitons have no query parameter, which means that all parameters should be a single value: atan2, rand, srand, sprintf, rename, unlink, system.

The function result is casted to the appropriate XQL primitive type (Number, Text or Boolean), or to an empty list if the result was undef.

Implementation Details

XQL Builtin Data Types

The XQL engine uses the following object classes internally. Only Number, Boolean and Text are considered primitive XQL types:

XML::XQL::Number

For integers and floating point numbers.
XML::XQL::Boolean

For booleans, e.g returned by true() and false().
XML::XQL::Text

For string values.
XML::XQL::Date

For date, time and date/time values. E.g. returned by the date() function.
XML::XQL::Node

Superclass of all XML node types. E.g. all subclasses of XML::DOM::Node subclass from this.
Perl list reference

Lists of values are passed by reference (i.e. using [] delimiters). The empty list [] has a double meaning. It also means 'undef' in certain situations, e.g. when a function invocation or comparison failed.

Type casting in comparisons

When two values are compared in an XML comparison (e.g. $eq$) the values are first casted to the same data type. Node values are first replaced by their value() (i.e. the XQL value() function is used, which returns a Text value by default, but may return any data type if the user so chooses.) The resulting values are then casted to the type of the object with the highest xql_primType() value. They are as follows: Node (0), Text (1), Number (2), Boolean (3), Date (4), other data types (4 by default, but this may be overriden by the user.)

E.g. if one value is a Text value and the other is a Number, the Text value is cast to a Number and the resulting low-level (Perl) comparison is (for $eq$):

$number->xql_toString == $text->xql_toString

If both were Text values, it would have been

$text1->xql_toString eq $text2->xql_toString

Note that the XQL spec is vague and even conflicting where it concerns type casting. This implementation resulted after talking to Joe Lapp, one of the spec writers.

Adding Data Types

If you want to add your own data type, make sure it derives from XML::XQL::PrimitiveType and implements the necessary methods.

I will add more stuff here to explain it all, but for now, look at the code for the primitive XQL types or the Date class (in Date.pm.)

Document Order

The XQL spec states that query results always return their values in document order, which means the order in which they appeared in the original XML document. Values extracted from Nodes (e.g. with value(), text(), rawText(), nodeName(), etc.) always have a pointer to the reference node (i.e. the Node from which the value was extracted.) These pointers are acknowledged when (intermediate) result lists are sorted. Currently, the only place where a result list is sorted is in a $union$ expression, which is the only place where the result list can be unordered. (If you find that this is not true, let me know.)

Non-node values that have no associated reference node, always end up at the end of the result list in the order that they were added. The XQL spec states that the reference node for an XML Attribute is the Element to which it belongs, and that the order of values with the same reference node is undefined. This means that the order of an Element and its attributes would be undefined. But since the XML::DOM module keeps track of the order of the attributes, the XQL engine does the same, and therefore, the attributes of an Element are sorted and appear after their parent Element in a sorted result list.

Constant Function Invocations

If a function always returns the same value when given "constant" arguments, the function is considered to be "constant". A "constant" argument can be either an XQL primitive (Number, Boolean, Text) or a "constant" function invocation. E.g.

date("12-03-1998")
true()
sin(0.3)
length("abc")
date(substr("12-03-1998 is the date", 0, 10))

are constant, but not:

length(book[2])

Results of constant function invocations are cached and calculated only once for each query. See also the CONST parameter in defineFunction. It is not necessary to wrap constant function invocations in a once() call.

Constant XQL functions are: date, true, false and a lot of the XQL+ wrappers for Perl builtin functions. Function wrappers for certain builtins are not made constant on purpose to force the invocation to be evaluated every time, e.g. 'mkdir("/user/enno/my_dir", "0644")' (although constant in appearance) may return different results for multiple invocations. See %PerlFunc in Plus.pm for details.

Function: count ([QUERY])

The count() function has no parameters in the XQL spec. In this implementation it will return the number of QUERY results when passed a QUERY parameter.

Method: text ([RECURSE])

When expanding an Element node, the text() method adds the expanded text() value of sub-Elements. When RECURSE is set to 0 (default is 1), it will not include sub-elements. This is useful e.g. when using the $match$ operator in a recursive context (using the // operator), so it won't return parent Elements when one of the children matches.

Method: rawText ([RECURSE])

See text().

AUTHOR

Please send bugs, comments and suggestions to Enno Derksen <enno@att.com>

To install XML::XQL, copy and paste the appropriate command in to your terminal.

cpanm

cpanm XML::XQL

CPAN shell

perl -MCPAN -e shell
install XML::XQL

For more information on module installation, please visit the detailed CPAN module installation guide.

	Global
`s`	Focus search bar
`?`	Bring up this help dialog

	GitHub
`g` `p`	Go to pull requests
`g` `i`	go to github issues (only if github is preferred repository)

	POD
`g` `a`	Go to author
`g` `c`	Go to changes
`g` `i`	Go to issues
`g` `d`	Go to dist
`g` `r`	Go to repository/SCM
`g` `s`	Go to source
`g` `b`	Go to file browse

	Search terms
module: (e.g. module:Plugin)
distribution: (e.g. distribution:Dancer auth)
author: (e.g. author:SONGMU Redis)
version: (e.g. version:1.00)