Architecture
Canonical format, request lifecycle, and routing inside the LM0 gateway.
LM0 is a self-hosted style LLM gateway: it exposes OpenAI- and Anthropic-compatible HTTP surfaces, aggregates upstream provider keys, translates wire formats, and routes/load-balances requests.
One canonical format in the middle
The gateway does not “pass through” OpenAI-to-OpenAI or Anthropic-to-Anthropic as a special case for every path. Instead:
- Inbound translate client JSON →
CanonicalRequest - Route to a provider adapter
- Outbound translate provider result →
CanonicalResponse/ stream events - Serialize back to the client’s expected wire format
With M inbound translators and N outbound adapters you support M×N combinations with M+N code paths.
client ──► inbound translate ──► CanonicalRequest
│
▼
Router
│
▼
Provider adapter
│
▼
client ◄── outbound serialize ◄── CanonicalResponse
Canonical types include requests, responses, stream events, and content blocks such as text, image, tool_use, tool_result, and thinking.
Request lifecycle
- Middleware — request id, CORS, registry attachment
- Auth — resolve API key; enforce ban / balance / RPM in DB mode
- Route handler — parse body, inbound translate to canonical
- Access check — model and pool authorization
- Router — ordered targets, retry and fallback
- Provider adapter — native fetch + translate to canonical
- Outbound serializer — client wire format
- Usage recording (DB mode) — buffer then flush metering events
Endpoints (public)
| Method | Path | Surface |
|---|---|---|
POST |
/v1/chat/completions |
OpenAI |
POST |
/v1/messages |
Anthropic |
POST |
/v1/messages/count_tokens |
Anthropic-style |
POST |
/v1/responses |
OpenAI Responses subset |
GET |
/v1/models |
Dual shape |
GET |
/healthz |
Health |
Auth modes
DB-backed
Postgres users, API keys, package tokens, RPM tiers, dashboard and admin mounts when configured.
Virtual keys
Config-defined keys for local/dev; optional open gateway when no keys are set (never for public prod).
Why this design
- Keep customer SDKs stable while swapping upstreams
- Centralize policy (auth, RPM, pools, maintenance)
- Meter usage once in the middle of the stack
- Grow providers without rewriting every client integration