# llmyard

> An OpenAI-compatible LLM egress gateway. Applications send one base URL and one per-project
> virtual key. llmyard holds the real provider credentials server-side, screens outbound content,
> enforces durable budgets, routes across providers with failover and circuit breaking, and records
> the real cost of every call read from the provider's own answer.

llmyard is the product. The repository, the container image and the environment-variable prefix are
all called axon; they are the same thing. The source is not public.

## What changes in your application

Two lines. Point the OpenAI SDK at https://llmyard.cc/v1 and send a per-project virtual key instead
of a provider key. Ask for a tier and llmyard resolves it to a concrete provider and model, or send
a concrete provider/model and it passes straight through.

Tier names are yours to choose: a tier is any name you put in the routing table, and the
conventional set is trivial, standard and heavy. Two names are reserved by llmyard —
auto and classifier — and mean "decide the model per request". A new key starts on the
entry tier, trial.

## The request loop, in order

1. Authenticate. The bearer virtual key resolves the project. Identity is never a client header.
2. Screen. 6 guardrail classes, redact or block, per project. Off by default.
3. Look up the cache. Exact, then semantic. A hit returns here and costs nothing.
4. Reserve the budget. Durable, and it survives a restart.
5. Route. Sender override, then project default, then passthrough.
6. Prove the endpoint. Circuit breaker closed, price inside the 3.0x guard.
7. Inject the real upstream credential. Server-side only.
8. Record the cost, read from the provider's answer.

Step 2 runs before 3, 4 and 7 so that a redacted value never reaches an embedding, a stored body or
an upstream request.

## Endpoints

POST /v1/chat/completions, /v1/messages (Anthropic wire, translated), /v1/embeddings,
/v1/responses, /v1/rerank, /v1/moderations, /v1/images/generations, /v1/audio/speech,
/video/generations. GET /v1/models, /status/peek, /usage/me.

Speech to text is NOT served. That caller goes to its vendor directly.

## Numbers

- 9 vendor presets, one credential each
- Reserved tier names: auto, classifier. Entry tier: trial. All other tier names are operator policy
- 6 guardrail classes: credit_card, iban, credential, email, phone, prompt_injection
- Semantic cache match at cosine 0.95, TTL 3600s, 1024 dimensions
- Failover price guard 3.0x; soft cap downroutes at 90% to the trivial tier
- Request body cap 32 MB; caller deadline ceiling 120000 ms
- New-key defaults: 1000 calls/hour, $5.00/day

## What it stores, and what it does not

Request and response bodies are written to axon_request_bodies, 32 KB per side. Two
dials decide how much is kept and the narrower wins: the project's own setting, and the operator
ceiling AXON_CAPTURE_BODIES. A project created today keeps only failed calls; rows created before
migration 0019 kept the previous default of all, and that migration deliberately did not rewrite
them. An operator routing other people's traffic should set AXON_CAPTURE_BODIES=errors.

Nothing is deleted automatically. The retention windows are unset by default and the only archive
implementation refuses to produce the receipt a partition drop requires, so no partition is ever
dropped as shipped.

The guardrail audit table records the class, the JSON path and a count. It does not record what
matched, and there is no column that could.

llmyard has no telemetry. A self-hosted deployment sends nothing to the operator of the hosted
service.

## Limits worth knowing before you rely on it

The prompt-injection screen catches known shapes only. It does not catch a novel semantic attack,
other languages, encodings other than base64, homoglyphs, indirect injection through a fetched
document, or a payload assembled across several requests. It is a signal, not a boundary.

/metrics is unauthenticated. The binary does not guard it; keeping it off the public internet is a
deployment choice.

## Licence

Elastic License 2.0. Source-available, not OSI open source. You may read the code, run it, change
it, and use it inside your own company at any scale. You may not offer llmyard to third parties as
a hosted or managed service, and you may not remove the licence-key checks. One project needs no
key; a second project and memberships need an offline key (AX1.<payload>.<signature>, Ed25519,
verified against a public key compiled into the binary — there is no licence server).

## Pages

- https://llmyard.cc/ — overview
- https://llmyard.cc/how-it-works — the request loop, routing, the cache
- https://llmyard.cc/reference — endpoints, ratings, defaults
- https://llmyard.cc/contact