/ 
Langertha-Knarr-1.102
/ Langertha::Knarr::Router

NAME

Langertha::Knarr::Router - Model name to Langertha engine routing with caching

VERSION

version 1.102

SYNOPSIS

use Langertha::Knarr::Router;

my $router = Langertha::Knarr::Router->new(config => $config);

my ($engine, $model) = $router->resolve('gpt-5.6-terra');
my $result = $engine->simple_chat(@messages);

my $models = $router->list_models;

DESCRIPTION

Resolves a model name to a Langertha engine instance and canonical model identifier. Engine instances are cached per engine, URL, API key variable, model and context_size, and reused across requests to avoid repeated construction overhead. Two models on the same endpoint get two instances, each carrying its own model.

Engine classes are resolved from both Langertha::Engine::* and LangerthaX::Engine::*.

When auto_discover is enabled in the config, the router queries each configured endpoint (engine, URL and API key variable) for its full model list on first use, making all discovered models available as routing targets.

A model id several endpoints list belongs to one of them, the same one on every run: an endpoint that is the passthrough upstream of some protocol ("passthrough" in Langertha::Knarr::Config, compared with "same_upstream" in Langertha::Knarr::Handler::Passthrough) before any other, then by model config name. The other endpoints listing it are logged at debug level.

A discovered model is routed with the config of the model entry it was discovered through, minus the keys that describe that one model. The keys split into two groups:

  • Endpoint-level, inherited: engine, url, api_key_env, api_key (where and how to reach the upstream), and system_prompt, temperature, response_size (the operator's request policy for that endpoint; a response_size is a cap on the answer, not a property of the model).

  • Model-specific, not inherited: model (replaced by the discovered id) and context_size (a context window belongs to one model; another model on the same endpoint may hold less, and on Ollama it sizes the loaded model through num_ctx). A discovered model gets no context_size, so the engine and upstream defaults apply.

user_agent_timeout is endpoint-level too: the seconds Langertha waits for that upstream. A model config without one gets "upstream_timeout" in Langertha::Knarr::Config (default 300); 0 in either leaves the engine without a timeout.

Any other key is inherited. A new per-model key belongs in the model-specific group.

config

The Langertha::Knarr::Config object. Required.

capabilities_generation

A counter that grows every time "probe_capabilities_f" finishes probing an engine instance. What an engine reports through supports() may have changed then, so anything that caches capabilities (the manifest) keys on it.

resolve

my ($engine, $model) = $router->resolve($model_name, %opts);
my ($engine, $model) = $router->resolve($model_name, skip_default => 1);

Resolves $model_name to a Langertha engine instance and the canonical model string to use with that engine. The resolution order is:

1. Explicit model config in "models" in Langertha::Knarr::Config
2. Auto-discovered models (if auto_discover is enabled)
3. The default engine from "default_engine" in Langertha::Knarr::Config (skipped when skip_default => 1 is passed)

On the default engine a named model replaces the model of the default: section, so the client's model reaches the provider as asked. Without a model name (undef or empty -- an A2A request never names one) only the default engine can answer, and it answers as configured: with its own model, or with the provider's default when default: names none (the third return value is then true and the returned model undef).

The third return value is true when the matched model config has no model key: the engine is then built without a model, the provider's default answers, and the returned model string is just the requested alias. Langertha::Knarr::Handler::Router uses it to report the model that answered instead of the alias.

my ($engine, $model, $alias_only) = $router->resolve($model_name);

Croaks if the model cannot be resolved, and croaks No model specified when no model is named and there is no default engine. Pass skip_default => 1 to allow the caller to try passthrough before falling back to the default engine; with it, an unresolved or missing model name returns an empty list.

list_models

my $models = $router->list_models;

Returns an ArrayRef of model hashrefs, each with keys id, engine, model, and source (either configured or discovered). Triggers auto-discovery if not already done. Used to build the model list responses for GET /v1/models and GET /api/tags.

probe_capabilities_f

$router->probe_capabilities_f( loop => $loop )->get;

Asks each routed engine instance once which capabilities its model has, from the provider's own model metadata (Langertha core's probe_model_capabilities_f, ADR 0032; today that is image_input). The facts are stored on the very instance "resolve" hands out, so POST /api/show (vision) and the provider manifest (image_input) pick them up without further work.

Every model "list_models" shows is resolved (running auto-discovery first when it is enabled and has not run yet), and every distinct engine instance behind them is covered. An instance is skipped when the installed core has no probe (Langertha 0.503), when its engine implements none (model_metadata_format is undefined: only OpenRouter, Mistral, LM Studio, T-Systems, Ollama and llama.cpp read model metadata; the others would answer {} without a request anyway), when it has no model to ask about, or when it was covered before. Calling the method again therefore covers only engine instances that are new since the last call, such as the ones a later discovery added.

How an instance is covered depends on its metadata document:

  • A catalogue (OpenRouter, Mistral, LM Studio, T-Systems: one document names every model; Langertha::ModelProbe->is_catalogue) is fetched once per endpoint (engine, URL and API key variable, the same key discovery uses) with models => 'all', and what it taught is imported into every other instance of that endpoint with import_learned_capabilities, without a request. A gateway with hundreds of discovered models costs one request. Facts never cross endpoints: two OpenRouter entries with different URLs are two catalogues. An instance that appears on an endpoint whose catalogue was already learned imports it; after a failed catalogue probe the next call asks again for the new instances only. A core without import_learned_capabilities probes every instance on its own, as below.

  • A per-model document (Ollama /api/show, llama.cpp) is asked once per instance, for that instance's upstream model.

The probes run concurrently, at most four at a time. A probe that fails or takes longer than "probe_timeout" in Langertha::Knarr::Config seconds (only when loop is given) is logged as a warning and not retried; the engines it would have taught keep the capabilities they had. The returned Future never fails and resolves to the number of engine instances covered (probed or imported into). With "probe_capabilities" in Langertha::Knarr::Config off it resolves to 0 at once.

"start" in Langertha::Knarr calls this once the server listens.

is_passthrough_model

next if $router->is_passthrough_model($model);

True when the router cannot resolve $model without the default engine: neither configured nor auto-discovered.

discovered_url

my $url = $router->discovered_url($model);

For a model known only through auto-discovery (not in "models" in Langertha::Knarr::Config), the base URL of the engine that listed it; undef for a configured or unknown model, or when the engine has no url. Langertha::Knarr sends such a model to the raw passthrough when it was listed by the passthrough upstream of the client's protocol and the client sends its own provider key.

SEE ALSO

SUPPORT

Issues

Please report bugs and feature requests on GitHub at https://github.com/Getty/langertha-knarr/issues.

IRC

Join #langertha on irc.perl.org or message Getty directly.

CONTRIBUTING

Contributions are welcome! Please fork the repository and submit a pull request.

AUTHOR

Torsten Raudssus <torsten@raudssus.de> https://raudssus.de/

COPYRIGHT AND LICENSE

This software is copyright (c) 2026 by Torsten Raudssus.

This is free software; you can redistribute it and/or modify it under the same terms as the Perl 5 programming language system itself.