Streaming
Server-sent events for OpenAI and Anthropic clients through LM0.
LM0 supports streaming generation for the main OpenAI and Anthropic surfaces. Upstream streams are normalized into a canonical event stream, then rewritten into the wire format your client expects.
OpenAI Chat Completions
{
"model": "your-pool/your-model",
"stream": true,
"messages": [{ "role": "user", "content": "Stream a short poem." }]
}
Clients receive standard OpenAI SSE chunks ending with a [DONE] sentinel (as implemented by the gateway for the chat surface).
curl
curl -N https://api.17.wtf/v1/chat/completions \
-H "Authorization: Bearer $LM0_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "your-pool/your-model",
"stream": true,
"messages": [{ "role": "user", "content": "Count from 1 to 5." }]
}'
Anthropic Messages
{
"model": "your-pool/your-model",
"max_tokens": 256,
"stream": true,
"messages": [{ "role": "user", "content": "Stream a short poem." }]
}
Event types follow Anthropic’s streaming protocol after outbound translation.
Cross-provider streams
If an OpenAI client streams against an Anthropic upstream (or the reverse), LM0 maps stream events through the canonical layer so your client still sees its native chunk shapes.
Some advanced event types or partial tool-call deltas may be approximated or omitted depending on upstream capabilities. Prefer non-streaming when debugging tool-heavy workflows.
Operational tips
- Use
-N/ disable buffering with reverse proxies when testing SSE - Keep timeouts long enough for slow models
- Retry logic should treat mid-stream disconnects carefully — partial output may already have been emitted