NAME

Text::Nunjucks::Lexer - 1:1 port of Nunjucks src/lexer.js -- the template tokenizer

VERSION

version 0.001

DESCRIPTION

Engine-internal: this module is a stage of the template pipeline, used by Text::Nunjucks::Environment, and not an interface for applications.

A 1:1 port of Nunjucks src/lexer.js. Turns a template string into a stream of tokens.

my $tokens = Text::Nunjucks::Lexer->lex( '{{ a }}', \%opts );
while ( my $tok = $tokens->next_token ) {
  # $tok->type, $tok->value, $tok->lineno, $tok->colno
}

lex returns a Text::Nunjucks::Lexer::Tokenizer; each next_token yields a Text::Nunjucks::Lexer::Token (with type, value, lineno, colno) until the input is exhausted, when it returns undef. lineno and colno are 0-based, as in the JavaScript original.

The token type strings are exposed as class methods mirroring the upstream module.exports (e.g. Text::Nunjucks::Lexer->TOKEN_SYMBOL).

SUPPORT

Issues

Please report bugs and feature requests at https://github.com/Getty/p5-text-nunjucks/issues.

CONTRIBUTING

Contributions are welcome! Please fork the repository and submit a pull request.

AUTHOR

Torsten Raudssus <getty@cpan.org>

COPYRIGHT AND LICENSE

This software is copyright (c) 2026 by Torsten Raudssus <torsten@raudssus.de> https://raudssus.de/.

This is free software; you can redistribute it and/or modify it under the same terms as the Perl 5 programming language system itself.