/ 
Langertha-Knarr-1.102
/ Langertha::Knarr::PassthroughUsage

NAME

Langertha::Knarr::PassthroughUsage - Token usage and model read off a copy of a raw passthrough answer

VERSION

version 1.102

SYNOPSIS

my $reader = Langertha::Knarr::PassthroughUsage->new;

# A stream, chunk by chunk, as the bytes go to the client
$reader->add_chunk($_) for @chunks;

# or a buffered answer at once
$reader->read_body($body);

$tracing->end_trace( $trace, output => '[stream]', $reader->trace_args );

DESCRIPTION

The raw passthrough ("raw_passthrough" in Langertha::Knarr) pipes the upstream's bytes to the client untouched. For its Langfuse trace, this reader looks at a copy of those bytes and keeps the token usage and the model the upstream reported -- it never changes, holds back or re-frames what the client gets.

It reads what the three passthrough upstreams send, by the shape of each JSON object, buffered or streamed:

  • OpenAI: model and usage of a completion; in a stream the chunk carrying usage (sent with stream_options.include_usage).

  • Anthropic: model and usage of a message; in a stream the usage of message_start, with the counts of every message_delta laid over it.

  • Ollama: model, prompt_eval_count and eval_count of an answer; in a stream those of the done frame.

The usage is kept in the provider's own keys, which "end_trace" in Langertha::Knarr::Tracing maps. A stream is read line by line (SSE data: lines, NDJSON lines), a line split across chunks included; only the unfinished last line is held, at most "max_line" bytes of it, and only lines that can carry usage or a still unknown model are decoded. Anything that does not parse is skipped.

max_line

Bytes of an unfinished line held between chunks. Default 65536; a longer line is dropped whole -- a usage frame is far shorter.

usage

The provider's usage hash seen last (merged for Anthropic), or undef.

model

The model the upstream named last, or undef.

add_chunk

$reader->add_chunk($bytes);

Reads the next piece of a stream. Returns nothing.

read_body

$reader->read_body($bytes);

Reads a whole buffered answer: one JSON object, or a buffered stream.

finish

$reader->finish;

Reads a last line the stream did not end with a newline. "trace_args" calls it.

trace_args

$tracing->end_trace( $trace, output => '[stream]', $reader->trace_args );

usage and model as far as the answer had them, for "end_trace" in Langertha::Knarr::Tracing.

SUPPORT

Issues

Please report bugs and feature requests on GitHub at https://github.com/Getty/langertha-knarr/issues.

IRC

Join #langertha on irc.perl.org or message Getty directly.

CONTRIBUTING

Contributions are welcome! Please fork the repository and submit a pull request.

AUTHOR

Torsten Raudssus <torsten@raudssus.de> https://raudssus.de/

COPYRIGHT AND LICENSE

This software is copyright (c) 2026 by Torsten Raudssus.

This is free software; you can redistribute it and/or modify it under the same terms as the Perl 5 programming language system itself.