/ 
Langertha-Skeid-0.003
/ Langertha::Skeid::Proxy

NAME

Langertha::Skeid::Proxy - Multi-format LLM proxy (OpenAI, Anthropic, Ollama) powered by Langertha::Skeid routing

VERSION

version 0.003

SYNOPSIS

use Langertha::Skeid::Proxy;
use Mojo::Server::Daemon;

my $app = Langertha::Skeid::Proxy->build_app(config_file => '/etc/skeid/skeid.yaml');
Mojo::Server::Daemon->new(app => $app, listen => ['http://127.0.0.1:8090'])->run;

# or simply
skeid serve --config /etc/skeid/skeid.yaml --listen 127.0.0.1:8090

DESCRIPTION

The Mojolicious application in front of Langertha::Skeid. It speaks three client formats -- OpenAI, Anthropic and Ollama -- and makes one kind of upstream call, an OpenAI-shaped POST to the node routing picked (ADR 0001); the translation lives in Langertha::Skeid::Protocol and its per-format modules. Everything else -- nodes, routing, admission, pricing, usage -- is the control plane's, driven through "call_function" in Langertha::Skeid, and is configured there (see "CONFIGURATION" in Langertha::Skeid). skeid serve runs it.

The request path is asynchronous throughout (ADR 0005): waiting for capacity is a timer, key resolution goes through "key_async" in Langertha::Skeid::KeyBroker, and the upstream call never blocks the loop.

Client routes

No Skeid credential is needed on these; see "Customer identity".

GET  /health                     {status: ok, proxy: skeid, config_reload: {...}}
GET  /.well-known/langertha.json provider manifest for the presented key
GET  /v1/models                  OpenAI: node models and alias names, once each
POST /v1/chat/completions        OpenAI chat; a stream is relayed byte for byte
POST /v1/embeddings              OpenAI embeddings
POST /v1/messages                Anthropic Messages, streamed or not
POST /api/chat                   Ollama chat; streams unless "stream": false
POST /api/generate               Ollama generate; streams unless "stream": false
GET  /api/tags                   Ollama: the same names as /v1/models
GET  /api/ps                     Ollama: always an empty list

/health stays ok while a config reload is failing -- the proxy serves under the config it kept -- and shows the reload state without its message. /v1/models and /api/tags list the node models and the alias names, each once, so a client can discover what it may put in model. The list follows the policy of the key presented (no key: the default policy): a name the key's models do not grant, an alias with every tier denied and a model only denied nodes serve are left out. Unhealthy nodes are included. A node without a model is not listed -- it matches any requested name, so no name reaches it in particular. The manifest route answers 404 unless the config enables it, 401 without a key and 403 for a key without a grant; see "Provider Manifest" in Langertha::Skeid.

Registry route

GET  /skeid/registry/snapshot    signed capacity snapshot, for a fronting Skeid

Bearer token: the admin API key or the registry read key (registry.read_key_env), and nothing else accepts the read key. 404 when neither is configured or the registry is not enabled, 401 for a wrong token, 503 while the signing secret is missing. The body is signed in X-Skeid-Registry-Signature and sent Cache-Control: no-store. See "registry_enabled" in Langertha::Skeid and ADR 0017.

Admin routes

Bearer token: the admin API key ("admin" in Langertha::Skeid). Without one configured every /skeid/* route answers 404; a missing or wrong token answers 401.

GET  /skeid/nodes                {nodes}
POST /skeid/nodes                body: a node, as a config nodes entry -> {ok, nodes}, or 400
POST /skeid/nodes/:id/health     body: {"healthy": true|false} -> {ok}
GET  /skeid/config               {reload}: the config reload status, with its message
GET  /skeid/metrics/nodes        {metrics}: per-node counters, never billed
GET  /skeid/usage                ?since=&api_key_id=&model=&limit= (default 50) -> the report

Changes made here live in this process only: a changed nodes section in the config replaces them, and under --workers each write reaches one worker (ADR 0010).

Customer identity

Skeid does not authenticate customers. The key a client presents (Authorization: Bearer, else x-api-key) derives the customer key id ("key_id_for_key" in Langertha::Skeid; no key is anonymous), which selects the routing policy and is recorded on the usage event. With routing.trust_key_id_header a x-skeid-key-id (or x-api-key-id) header names the key id instead.

The upstream call

The node URL gets /v1 added unless it ends in it, then /chat/completions or /embeddings; the body carries the served model (an alias tier's model), everything else as the client sent it or as translated. The client's headers go upstream except the hop-by-hop ones, Host, Content-Length and Accept-Encoding. When the node has a key of its own -- api_key_ref through the key broker, else api_key_env -- it replaces Authorization and the client's Authorization and x-api-key are dropped, however the client spelled them. A node that names neither forwards the client's own key. A node that names one and gets no key from it -- the broker fails or is not running, the variable is unset or empty -- is not called at all, and neither is one that left the inventory after it was selected: the request is refused with 503 (see "Errors"), so the client's key never stands in for the node's. An answer that came from a node carries x-skeid-node with the node id. Rate-limit headers and 429s on every response feed "observe_response_headers" in Langertha::Skeid.

Each admitted request gets its request.finish on every path and one usage event, failures included.

A client that hangs up

A client that closes its connection before the answer is complete ends its request at that moment. While it waits for capacity it stops waiting and takes no slot. Once a node was called, the upstream connection is closed -- which is how the node learns to stop generating -- and is not returned to the pool; the slot is given back with a request.finish marked aborted -- counted apart, not as a node error, so the node's error counter and the registry snapshot's errors_in_window stay untouched -- and the one usage event is written with ok = 0, status_code 499 and error_type client_abort, priced from the usage the stream had reported until then (nothing, for a request that was not streamed). A client that leaves while the node's key is still being resolved gives its slot back too, but nothing was forwarded, so no usage event is written. When the node had already finished and only the rest of the answer was still being written out, the request stays what it was: finished and metered by the node's answer.

Errors

A request no node may serve for this key is 403 permission_error; a model no healthy node serves is 503 model_not_found; eligible nodes that stay full past the wait are 429 rate_limit_error; an upstream failure is its status (or 502) with type upstream_error; a node whose own key cannot be resolved is 503 upstream_key_unavailable, logged with the key reference and recorded as a failed usage event. The body is shaped for the face that was called: OpenAI's {error: {message, type}}, Anthropic's envelope ("error_body" in Langertha::Skeid::Protocol::Anthropic) on /v1/messages, and Ollama's {error: "..."} on /api/*. A stream that fails after it opened ends with the face's in-band error event where it has one.

METHODS

build_app

my $app = Langertha::Skeid::Proxy->build_app(%options);

Builds the Mojolicious application. Options:

The Skeid's "on_usage_lost" in Langertha::Skeid is set to this app's usage event lost log line, so a usage event a write-behind store could not write is logged like one whose synchronous write failed. An embedding application that wants its own hook sets it after build_app.

With both OPENBAO_ROLE_ID and OPENBAO_SECRET_ID set, a Langertha::Skeid::KeyBroker::OpenBao at OPENBAO_ADDR (default http://127.0.0.1:8200) becomes the key broker; if its login fails the proxy warns and runs without one. A broker that can renew its token starts renewing on a timer. The capacity probes of every node are started and restarted whenever the probed part of the inventory changes.

Upstream connections time out after 10s to connect, and at most SKEID_UPSTREAM_POOL (default 100) are kept. An upstream request may take SKEID_UPSTREAM_TIMEOUT seconds (default 300; a positive integer, anything else counts as unset) and may be silent for all of them -- the time to the first token is silence on the wire. The client's connection is given the same time on top of the server's own inactivity timeout, on the routes that call an upstream and for that request only: the proxy never closes a request its upstream is still working on, and is still there to answer when the upstream timed out. Every other route stays under the server's timeout. The client's side is read from the user agent when a request arrives, so whoever changes $app->ua->request_timeout afterwards changes both sides, and should set $app->ua->inactivity_timeout to match.

The app has a skeid helper returning the control plane.

SEE ALSO

SUPPORT

Issues

Please report bugs and feature requests on GitHub at https://github.com/Getty/langertha-skeid/issues.

IRC

Join #langertha on irc.perl.org or message Getty directly.

CONTRIBUTING

Contributions are welcome! Please fork the repository and submit a pull request.

AUTHOR

Torsten Raudssus <torsten@raudssus.de> https://raudssus.de/

COPYRIGHT AND LICENSE

This software is copyright (c) 2026 by Torsten Raudssus.

This is free software; you can redistribute it and/or modify it under the same terms as the Perl 5 programming language system itself.