NAME
Text::Nunjucks::Lexer - 1:1 port of Nunjucks src/lexer.js -- the template tokenizer
VERSION
version 0.001
DESCRIPTION
Engine-internal: this module is a stage of the template pipeline, used by Text::Nunjucks::Environment, and not an interface for applications.
A 1:1 port of Nunjucks src/lexer.js. Turns a template string into a stream of tokens.
my $tokens = Text::Nunjucks::Lexer->lex( '{{ a }}', \%opts );
while ( my $tok = $tokens->next_token ) {
# $tok->type, $tok->value, $tok->lineno, $tok->colno
}
lex returns a Text::Nunjucks::Lexer::Tokenizer; each next_token yields a Text::Nunjucks::Lexer::Token (with type, value, lineno, colno) until the input is exhausted, when it returns undef. lineno and colno are 0-based, as in the JavaScript original.
The token type strings are exposed as class methods mirroring the upstream module.exports (e.g. Text::Nunjucks::Lexer->TOKEN_SYMBOL).
SUPPORT
Issues
Please report bugs and feature requests at https://github.com/Getty/p5-text-nunjucks/issues.
CONTRIBUTING
Contributions are welcome! Please fork the repository and submit a pull request.
AUTHOR
Torsten Raudssus <getty@cpan.org>
COPYRIGHT AND LICENSE
This software is copyright (c) 2026 by Torsten Raudssus <torsten@raudssus.de> https://raudssus.de/.
This is free software; you can redistribute it and/or modify it under the same terms as the Perl 5 programming language system itself.