/ 
Langertha-Knarr-1.102
/ Langertha::Knarr

NAME

Langertha::Knarr - Universal LLM hub — proxy, server, and translator across OpenAI/Anthropic/Ollama/A2A/ACP/AG-UI

VERSION

version 1.102

SYNOPSIS

The fastest way to use Knarr is the Docker image:

docker run -e ANTHROPIC_API_KEY -p 8080:8080 raudssus/langertha-knarr
ANTHROPIC_BASE_URL=http://localhost:8080 claude

The Perl API behind it:

use IO::Async::Loop;
use Langertha::Knarr;
use Langertha::Knarr::Config;
use Langertha::Knarr::Router;
use Langertha::Knarr::Handler::Router;

my $loop   = IO::Async::Loop->new;
my $config = Langertha::Knarr::Config->new(file => 'knarr.yaml');
my $router = Langertha::Knarr::Router->new(config => $config);

my $knarr = Langertha::Knarr->new(
    handler => Langertha::Knarr::Handler::Router->new(router => $router),
    router  => $router,
    loop    => $loop,
    listen  => $config->listen,
);
$knarr->run;   # blocks; OpenWebUI etc. can now connect

knarr start (see knarr) builds exactly this from a config file, plus the raw passthrough, the tracing and request-log decorators, the proxy key and the other settings of Langertha::Knarr::Config.

DESCRIPTION

Langertha::Knarr is a universal LLM hub that exposes any backend — a Langertha::Raider, a raw Langertha::Engine, a remote A2A or ACP agent, or any custom Langertha::Knarr::Handler — over the standard LLM HTTP wire protocols spoken by OpenWebUI, the OpenAI / Anthropic / Ollama SDKs, and the agent ecosystems around A2A, ACP, and AG-UI.

By default a single running Knarr answers all of these simultaneously on every listening port, driven by the same handler implementation:

  • OpenAI: POST /v1/chat/completions, GET /v1/models

  • Anthropic: POST /v1/messages

  • Ollama: POST /api/chat, POST /api/generate, GET /api/tags, GET /api/version, POST /api/show

  • A2A: GET /.well-known/agent.json and JSON-RPC POST /

  • ACP: GET /agents, POST /runs

  • AG-UI: POST /awp

  • Knarr's own GET /.well-known/langertha.json (see "MANIFEST")

Knarr is built on IO::Async and Net::Async::HTTP::Server with native Future::AsyncAwait integration into Langertha engines, so streaming works end-to-end token-by-token without any thread or event-loop bridges.

MANIFEST

GET /.well-known/langertha.json serves a Langertha provider manifest (schema v1, see Langertha::Manifest) so a client such as raider --provider HOST can configure itself. It is built by Langertha::Knarr::Manifest from what this Knarr exposes: one endpoint per protocol with a manifest dialect (OpenAI /v1 as openai-chat, Anthropic as anthropic-compat, Ollama as ollama), every listed model (configured aliases and auto-discovered ids) on each of them, with the serving engine's model-scoped capabilities narrowed to what that protocol forwards, and an api_key auth entry when "auth_token" is set. The route is protected by "auth_token" like GET /v1/models. It never carries configuration: no upstream URL, key, key variable name, engine class or passthrough target.

The manifest needs a Langertha that ships Langertha::Manifest::Builder; with an older one the route answers 404 with a JSON error.

ARCHITECTURE

Three pluggable layers:

Protocols

Wire formats live in Langertha::Knarr::Protocol::*. Each consumes Langertha::Knarr::Protocol and is loaded by default. See Langertha::Knarr::Protocol::OpenAI, Langertha::Knarr::Protocol::Anthropic, Langertha::Knarr::Protocol::Ollama, Langertha::Knarr::Protocol::A2A, Langertha::Knarr::Protocol::ACP, Langertha::Knarr::Protocol::AGUI.

Handlers

Backend logic — what answers the request. Knarr ships with Langertha::Knarr::Handler::Router (the default, model→engine via Langertha::Knarr::Router), Langertha::Knarr::Handler::Engine (single engine), Langertha::Knarr::Handler::Raider (per-session agent), Langertha::Knarr::Handler::Passthrough (raw HTTP forward), Langertha::Knarr::Handler::A2AClient / Langertha::Knarr::Handler::ACPClient (consume remote agents), and Langertha::Knarr::Handler::Code (coderef-backed for tests). Implement Langertha::Knarr::Handler to write your own. Decorators (Langertha::Knarr::Handler::Tracing, Langertha::Knarr::Handler::RequestLog) wrap any inner handler and add behavior on top — they themselves consume the Handler role and compose freely.

Transport

Default is Net::Async::HTTP::Server with chunked SSE / NDJSON streaming on one or more listen sockets. For Plack deployments, Langertha::Knarr::PSGI wraps the same Knarr instance into a PSGI app (buffered — see its docs for the streaming caveat).

handler

Required. An object consuming Langertha::Knarr::Handler.

listen

ArrayRef of host:port strings or { host => ..., port => ... } hashes. Defaults to a single entry composed from "host" and "port".

host

Default 127.0.0.1. Used when "listen" is not given.

port

Default 8088. Used when "listen" is not given.

loop

Optional IO::Async::Loop instance. Defaults to a fresh one.

workers

Number of processes that serve requests. Default 1: "run" serves from the calling process, nothing is forked. With more, "run" binds the listen sockets, runs auto-discovery and the capability probe once (see "run"), then forks that many workers that all accept on the same sockets, and stays behind as their supervisor, serving no request itself. Must be 1 or more; anything else croaks at construction. knarr start sets it from -w N, else from "workers" in Langertha::Knarr::Config (workers: or KNARR_WORKERS).

Every worker has its own state: Langertha::Knarr::Sessions (so a Langertha::Knarr::Handler::Raider conversation is only continued when its next request reaches the same worker -- nothing routes it there), the engine cache, and its own Langfuse batches, flushed by that worker. Request log lines from all workers go to the same logging.file, one whole line per write. Langertha::Knarr::PSGI never calls "run"; there the PSGI server decides about processes.

protocols

ArrayRef of protocol class basenames to load. Defaults to all six shipped protocols.

protocol_args

Optional HashRef of constructor arguments per protocol, keyed by the name as it appears in "protocols". The A2A agent card name and description:

protocol_args => { A2A => {
  agent_name        => 'Support Agent',
  agent_description => 'Answers support questions',
} },

knarr start passes "protocol_args" in Langertha::Knarr::Config.

auth_token

Optional shared secret. When set, every incoming request must present it as Authorization: Bearer or x-api-key. Discovery routes (/.well-known/agent.json) stay anonymous.

A header sent twice counts as one value of comma-separated elements (that is how a PSGI server and HTTP::Headers->header merge it), and the key may be any one of them.

The key never leaves Knarr: it is taken out of a request to a passthrough upstream (the "raw_passthrough" and a Langertha::Knarr::Handler::Passthrough in the handler chain alike). From Authorization and x-api-key, every element that is the key -- bare or as Bearer key, in either header -- is removed, the others are kept, and a header left with nothing is dropped; a header without the key goes through unchanged. Every other header is forwarded, so a passthrough client sends its own provider key in the other header -- the proxy key in x-api-key and the OpenAI key as Authorization: Bearer, or the proxy key as Authorization: Bearer and the Anthropic key in x-api-key.

ollama_compat_version

The Ollama version reported at GET /api/version as {"version":"..."}. Default 0.34.4, the current Ollama release when it was chosen (2026-09-23). It is a compatibility claim, not Knarr's own version: Ollama clients read it as the server's Ollama version. Two clients set its limits:

  • VS Code Copilot (BYOK Ollama) refuses a server below 0.6.4.

  • Open WebUI parses every dotted part with int(), so anything but digits and dots (0.34.4-knarr, v0.34) breaks its connection check.

The value must therefore be three dot-separated numbers (/^\d+\.\d+\.\d+$/); anything else croaks at construction. No surveyed client gates think, tools or format on the version -- they read /api/show capabilities per model -- so raising it enables nothing Knarr lacks. Configured as ollama_compat_version in the config file or KNARR_OLLAMA_COMPAT_VERSION.

public_url

Optional public base URL (e.g. https://knarr.example) used in the provider manifest. When unset, the base URL is taken from the request (Host, and X-Forwarded-Proto when it is http or https).

raw_passthrough

Optional Langertha::Knarr::Handler::Passthrough. Together with a "router", a chat request for a model the router does not configure -- or knows only from auto-discovery on that same upstream ("discovered_url" in Langertha::Knarr::Router, "is_upstream_for" in Langertha::Knarr::Handler::Passthrough), when the request carries the client's own provider key -- is piped byte for byte to the upstream for the client's protocol (an Ollama /api/generate to the upstream's /api/generate, any other Ollama chat to /api/chat), bypassing the handler chain -- but only when that protocol has an upstream ("serves_protocol" in Langertha::Knarr::Handler::Passthrough). A request in any other protocol (Ollama without an ollama upstream, A2A, ACP, AG-UI) goes through the handler, where Langertha::Knarr::Handler::Router hands it to the default engine, or answers 404 in the protocol's error shape when there is none.

Every client header line goes to the upstream as a line of its own, in its order -- a header sent twice arrives twice -- with the proxy key taken out of each line (see "auth_token") and without Host, Content-Length, Connection and Transfer-Encoding. The upstream's response headers come back the same way, repeats included (Set-Cookie twice), except the connection-level ones and the framing Knarr sets itself (Transfer-Encoding, Content-Length). A body compressed with an encoding Knarr's HTTP client decodes (gzip, deflate) goes back decoded, without its Content-Encoding; a body in any other encoding (br, zstd, several stacked), buffered or streamed, goes back as the upstream sent it, with it. A stream gets Cache-Control: no-cache when the upstream sent none.

The provider key is looked for once the proxy key is taken out (see "auth_token"): Authorization for OpenAI, x-api-key or Authorization for Anthropic; Ollama needs none. A discovered model requested without one goes through the handler instead, to the engine that listed it, with that engine's key -- the upstream would only answer 401. A model nobody configured or discovered passes through either way.

A discovered model's request that does carry a key the upstream refuses with 401 -- the placeholder key an SDK sends when it has none -- is answered the same way: through the handler, by the engine that listed the model, with that engine's key. The client never sees the 401, streaming or not: the upstream's status is known before any byte goes to the client. The upstream is asked once, its refusal is dropped, and the request is traced once, by Langertha::Knarr::Handler::Tracing, with passthrough_fallback: 401 in the trace's metadata. Any other status (403, 429, 5xx) goes back unchanged, and so does the 401 for a model nobody configured or discovered, which has no engine to fall back to.

router

Optional Langertha::Knarr::Router, usually the one the Langertha::Knarr::Handler::Router handler wraps. Knarr itself uses it to decide which requests go to the "raw_passthrough" (models it does not configure, and models it only discovered on that upstream when the client sends its own key), to answer POST /api/show from the routed engine's capabilities (without a router it falls back to the handler's list_models and claims no vision), and to start the capability probe in "start".

tracing

Optional Langertha::Knarr::Tracing. Only the "raw_passthrough" uses it: a request that bypasses the handler chain gets a lightweight Langfuse trace from here. Its generation carries the token usage and the model the upstream reported, read off a copy of the answer by Langertha::Knarr::PassthroughUsage -- the JSON body, or the frames of a stream that carry them (OpenAI's final usage chunk, which the client asks for with stream_options.include_usage; Anthropic's message_start and message_delta; Ollama's done frame). The bytes the client gets are never touched; nothing is read without an active trace, nor from bytes that still carry a Content-Encoding. Requests through the handler chain are traced by the Langertha::Knarr::Handler::Tracing decorator instead -- so is a raw passthrough that falls back to its engine after a 401 (see "raw_passthrough"); the trace of a request that may fall back is therefore opened only once the upstream's status is known, dated back to when the request went out. Raw passthrough requests are not written to the request log; a fallback is, by the handler chain.

start

$knarr->start;

Binds all listen sockets and registers the dispatcher. Returns $self. Does not enter the event loop. Each socket queues up to SOMAXCONN pending connections (128 where Socket does not know it; the kernel caps it at its own limit, net.core.somaxconn on Linux).

Once the sockets listen, it starts "probe_capabilities_f" in Langertha::Knarr::Router on the "loop" when a "router" that has it is set (see "probe_capabilities" in Langertha::Knarr::Config). The probe runs in the background; until it has finished, POST /api/show and the manifest answer from Langertha's static capability tables. Langertha::Knarr::PSGI does not call "start", so under PSGI the probe runs only when you call it.

run

$knarr->run;   # blocks

With one "workers" (the default): calls "start" if needed, then enters the "loop" and blocks.

With more, it prefork-serves and returns only once all workers are gone:

1. The listen sockets are bound here ("start" may have done it), and this process stops accepting on them.
2. Auto-discovery and the capability probe run here, once, to the end, before any worker exists -- each worker inherits the discovered models and the learned capabilities instead of asking the upstreams again. Until the probe is done (each probe gives up after "probe_timeout" in Langertha::Knarr::Config), connections wait in the sockets' backlog. Upstream connections the probe kept open are closed, and the helper processes of the loop's resolver end, so no two workers share one: each worker starts its own resolver helpers.
3. "workers" processes are forked; each rebuilds the loop's kernel state ("post_fork" in IO::Async::Loop) and accepts on the inherited sockets.
4. This process supervises them: a worker that exits is replaced -- one that dies within five seconds of its start after a pause of 1, 2, 4 ... up to 30 seconds, so a worker that cannot start does not spin. A replacement that cannot be forked (out of processes or memory) is tried again the same way while the other workers serve on; only a failed fork of the first workers makes "run" die, after stopping those already started. SIGTERM or SIGINT is passed on to the workers as SIGTERM; once they have exited, "run" returns.

session

my $session = $knarr->session($id);

Returns the Langertha::Knarr::Session for the given id, creating one on demand. Used internally by the dispatcher.

manifest_response

my ($status, $json_body) = $knarr->manifest_response(
    scheme => 'https', host => 'knarr.example', prefix => '' );

Builds the provider manifest (see "MANIFEST") and returns the HTTP status and JSON body. The base URL is "public_url", else scheme://host plus prefix from the request. 404 when core has no manifest builder, 400 when no base URL can be derived, 500 when the manifest cannot be built. Used by the native server and Langertha::Knarr::PSGI.

SUPPORT

Issues

Please report bugs and feature requests on GitHub at https://github.com/Getty/langertha-knarr/issues.

IRC

Join #langertha on irc.perl.org or message Getty directly.

CONTRIBUTING

Contributions are welcome! Please fork the repository and submit a pull request.

AUTHOR

Torsten Raudssus <torsten@raudssus.de> https://raudssus.de/

COPYRIGHT AND LICENSE

This software is copyright (c) 2026 by Torsten Raudssus.

This is free software; you can redistribute it and/or modify it under the same terms as the Perl 5 programming language system itself.