NAME

Langertha::Engine::LlamaCpp - llama.cpp server

VERSION

version 0.302

SYNOPSIS

use Langertha::Engine::LlamaCpp;

my $llama = Langertha::Engine::LlamaCpp->new(
    url           => 'http://localhost:8080/v1',
    system_prompt => 'You are a helpful assistant',
);

print $llama->simple_chat('Hello!');

my $embedding = $llama->simple_embedding('Some text');

DESCRIPTION

Provides access to llama.cpp's built-in HTTP server, which exposes an OpenAI-compatible API. Composes Langertha::Role::OpenAICompatible.

Only url is required. The URL must include the /v1 path prefix (e.g., http://localhost:8080/v1). Since llama.cpp serves exactly one model (loaded at server startup), no model name or API key is needed.

Supports chat, streaming, embeddings, and MCP tool calling.

See https://github.com/ggml-org/llama.cpp/blob/master/examples/server/README.md for server setup.

THIS API IS WORK IN PROGRESS

SEE ALSO

• https://github.com/ggml-org/llama.cpp - llama.cpp project

• Langertha::Engine::vLLM - Another self-hosted OpenAI-compatible engine

• Langertha::Engine::OllamaOpenAI - Ollama's OpenAI-compatible API

SUPPORT

Issues

Please report bugs and feature requests on GitHub at https://github.com/Getty/langertha/issues.

CONTRIBUTING

Contributions are welcome! Please fork the repository and submit a pull request.

AUTHOR

Torsten Raudssus <torsten@raudssus.de> https://raudss.us/

COPYRIGHT AND LICENSE

This software is copyright (c) 2026 by Torsten Raudssus.

This is free software; you can redistribute it and/or modify it under the same terms as the Perl 5 programming language system itself.