/ 
Langertha-Knarr-1.102
/ Langertha::Knarr::Tracing

NAME

Langertha::Knarr::Tracing - Automatic Langfuse tracing per proxy request

VERSION

version 1.102

SYNOPSIS

use Langertha::Knarr::Tracing;

my $tracing = Langertha::Knarr::Tracing->new(config => $config);

my $trace_id = $tracing->start_trace(
  model    => 'gpt-5.6-terra',
  engine   => 'Langertha::Engine::OpenAI',
  messages => \@messages,
  params   => \%params,
  format   => 'openai',
);

# ... handle request ...

$tracing->end_trace($trace_id,
  output => $response_text,
  model  => 'gpt-5.6-terra',
  usage  => { input => 100, output => 50, total => 150 },
);

DESCRIPTION

Records every proxy request as a Langfuse trace with a nested generation. When tracing is not configured (no public and secret key), all methods are no-ops.

Two transports carry the trace to Langfuse, selected by "transport": ingestion (the default) posts events to Langfuse's /api/public/ingestion API, otel exports OpenTelemetry spans (see "OpenTelemetry transport"). Both record the same content through the same calls, in one fire-and-forget POST per request that gives up after "langfuse_timeout" in Langertha::Knarr::Config seconds (default 15).

Langfuse credentials are read from the config file's langfuse: section or from the LANGFUSE_PUBLIC_KEY, LANGFUSE_SECRET_KEY, and LANGFUSE_URL environment variables. The module strips surrounding quotes from environment variable values, which Docker --env-file sometimes adds literally.

Timing sources

Knarr has two request paths and they do not measure latency the same way. The generation's startTime always marks the moment "start_trace" ran; what differs is where endTime and completionStartTime come from.

  • Routed, non-streaming — a Langertha engine produced a Langertha::Response, so Langertha::Knarr::Handler::Tracing hands the engine-measured timing hash to "end_trace". endTime becomes startTime + total_seconds and completionStartTime becomes startTime + ttft_seconds, both anchored to the high-resolution timestamp "start_trace" recorded. This is the only path with a real time-to-first-token, and the durations exclude the proxy's own formatting overhead.

  • Routed, streaming — the decorator accumulates deltas and never sees a response object, so it measures time-to-first-token itself in the proxy: Langertha::Knarr::Handler::Tracing starts its clock before opening the upstream stream and stops at the first non-undef delta, then passes that ttft_seconds to "end_trace" as timing. completionStartTime is therefore emitted, but — unlike the non-streaming path, where the engine measures — this figure is the proxy's own view and includes its dispatch overhead. endTime stays the wall-clock moment the stream was exhausted; a stream that yields no delta (empty or failed) produces no timing and no completionStartTime.

  • Raw passthrough — bytes are piped 1:1 and never parsed, so no Langertha::Response exists at all. endTime is again the proxy's own wall clock at "end_trace", which includes network time to the upstream provider.

Callers that pass no timing therefore keep exactly the previous behaviour: proxy-measured endTime, no completionStartTime.

OpenTelemetry transport

With "transport" otel, each request goes out as one OTLP/HTTP request, JSON encoded, to /api/public/otel/v1/traces under "config"'s Langfuse URL, with the same Basic authentication and the x-langfuse-ingestion-version: 4 header (Langfuse's real-time ingestion path). A span must not be exported twice, so "start_trace" only records the start and "end_trace" builds and sends the finished trace, two spans under one random trace id:

  • a root span named after "trace_name" (kind SERVER) with langfuse.trace.name, langfuse.trace.tags (knarr), langfuse.trace.metadata.* (format, engine, model, params, passthrough_fallback), and the request messages and the output (or the error) as langfuse.observation.input / langfuse.observation.output, which Langfuse shows as the trace's input and output;

  • its child proxy-request (kind CLIENT), a langfuse.observation.type generation with langfuse.observation.model.name, input and output, langfuse.observation.usage_details (the same buckets the ingestion path sends as usageDetails, cache reads and writes apart from the uncached input, see "end_trace"), langfuse.observation.completion_start_time and langfuse.observation.metadata.* (timing, response_id, configured_model, thinking, rate_limit, tool_calls). A start_hires given to "start_trace" is the start of both spans.

An error sets the status of both spans to ERROR with the message, plus langfuse.observation.level ERROR and langfuse.observation.status_message, so the trace itself shows as failed, not only its generation.

Span start and end are the instants the ingestion path reports (see "Timing sources"), in Unix nanoseconds. Structured values (messages, params, usage, metadata hashes) are JSON-encoded into string attributes; plain strings go as they are, as Langfuse's own SDKs send them.

config

The Langertha::Knarr::Config object. Required. Provides Langfuse credentials and trace_name.

trace_name

The Langfuse trace name applied to all traces. Resolved in priority order from: langfuse.trace_name in config, LANGFUSE_TRACE_NAME env var, KNARR_TRACE_NAME env var, or the default knarr-proxy.

transport

ingestion (default) or otel, from "langfuse_transport" in Langertha::Knarr::Config (langfuse.transport, else KNARR_LANGFUSE_TRANSPORT). An enabled tracer reads it, and "langfuse_timeout" in Langertha::Knarr::Config, when it is built, so an invalid value croaks at startup, not on the first request.

start_trace

my $trace_info = $tracing->start_trace(
  model    => $model_name,
  engine   => $engine_class,
  messages => \@messages,
  params   => \%params,
  format   => 'openai',
);

Creates a new Langfuse trace and generation. Returns a $trace_info hashref that must be passed to "end_trace". Returns undef when tracing is disabled.

The returned hashref carries start_hires, the gettimeofday pair behind start_time. "end_trace" anchors engine-measured durations to it; see "Timing sources".

Two optional arguments:

  • start_hires -- a gettimeofday pair the trace starts at instead of now, for a trace opened after the request went out. The raw passthrough of a model that may still fall back to its engine opens its trace only once the upstream's status rules that out (see "raw_passthrough" in Langertha::Knarr).

  • passthrough_fallback -- the upstream status that sent a raw passthrough request to its engine instead (401), recorded in the trace's metadata. Langertha::Knarr::Handler::Tracing passes it from the request.

end_trace

$tracing->end_trace($trace_info,
  output => $response_text,
  model  => $model,
  usage  => { input => 100, output => 50, total => 150 },
  timing => { ttft_seconds => 0.25, total_seconds => 1.5 },
  response_id => 'chatcmpl-123',
);

# On error:
$tracing->end_trace($trace_info, error => "Something went wrong");

Closes the generation and trace started by "start_trace", then flushes the batch to Langfuse. Pass error to record a failed generation at level ERROR. Does nothing when $trace_info is undef (tracing was disabled at start).

Optional metadata carried off a Langertha::Knarr::Response, all skipped when absent:

  • timing — HashRef with ttft_seconds / total_seconds. Drives the generation's endTime and completionStartTime (the Langfuse field for time-to-first-token) and is recorded verbatim in the metadata, so provider-native stage durations survive too. See "Timing sources".

  • response_id — the provider's own response id, for correlating a Langfuse generation with the provider's logs.

  • configured_model — the model name Knarr answers the client under when it differs from the model the backend reported (a routed model's configured name). The generation's model is the reported one; this keeps the configured one next to it.

  • thinking — reasoning text the engine split off content. It is model output that output no longer contains, so the trace is the only place it survives.

  • rate_limit — a Langertha::RateLimit (or equivalent hashref). Flattened to its quota scalars; the raw header hash is not recorded.

  • usage — a Langertha::Usage (the shape every routed response carries), a hashref in Langfuse's own keys (input / output / total), or a provider's usage hash (OpenAI prompt_tokens, Anthropic input_tokens, Ollama prompt_eval_count, ...), read through "from_hash" in Langertha::Usage. Whatever the input, the generation gets the counts in the only keys Langfuse stores: usageDetails for v3, which lets it override usage, and usage as { input, output, total, unit => 'TOKENS' } for Langfuse v2, which drops the key it does not know.

    usageDetails holds Langfuse's exclusive buckets, each priced on its own and total their sum: input is the uncached input, prompt-cache reads go to input_cached_tokens and cache writes to input_cache_creation (each only when not zero), and output. Every token lands in exactly one bucket, whichever way the provider counts: Anthropic reports cache_read_input_tokens / cache_creation_input_tokens beside input_tokens, OpenAI's prompt_tokens_details.cached_tokens is part of prompt_tokens and is taken out of input. v2's usage has no cache buckets, so its input is the whole input, cache included. A hashref in Langfuse's keys may carry the two cache buckets itself; its missing total is the sum of its buckets; a provider's total is always the sum. Without usage, with an empty hash, or with one that counts no tokens (an error body, a hash without any count key), neither key is sent, so Langfuse never records a zeroed usage. The raw passthrough passes the provider's usage hash it read off a copy of the upstream's answer (see "tracing" in Langertha::Knarr).

  • tool_calls — the response's Langertha::ToolCall list. The trace is the detailed view, so it records the full tool calls — id, name, the complete arguments and synthetic — flattened to plain hashes. The JSONL request log keeps only a trimmed form (see "end_request" in Langertha::Knarr::RequestLog).

flush

$tracing->flush;

Sends all pending trace events to Langfuse in one POST and clears the internal buffer: an ingestion batch to /api/public/ingestion, or, with "transport" otel, an OTLP ExportTraceServiceRequest to /api/public/otel/v1/traces. Called automatically by "end_trace". Does nothing when tracing is disabled or the batch is empty.

Never throws: a batch that cannot be JSON-encoded is logged at error level and dropped, the same way an ingestion HTTP failure is. "end_trace" runs on the request's response path, so a tracing problem must not take the client's response down with it.

The POST is sent without waiting for it and gives up after "langfuse_timeout" in Langertha::Knarr::Config seconds (langfuse.timeout or KNARR_LANGFUSE_TIMEOUT, default 15, 0 for none): a Langfuse that accepts the connection and never answers is logged as a Langfuse flush error warning, and never holds a request.

SEE ALSO

SUPPORT

Issues

Please report bugs and feature requests on GitHub at https://github.com/Getty/langertha-knarr/issues.

IRC

Join #langertha on irc.perl.org or message Getty directly.

CONTRIBUTING

Contributions are welcome! Please fork the repository and submit a pull request.

AUTHOR

Torsten Raudssus <torsten@raudssus.de> https://raudssus.de/

COPYRIGHT AND LICENSE

This software is copyright (c) 2026 by Torsten Raudssus.

This is free software; you can redistribute it and/or modify it under the same terms as the Perl 5 programming language system itself.