These pages describe the intended behaviour of the gateway. The open-source release and the enterprise edition are being split out; details will be updated when the first release ships.
Streaming
How server-sent event streams pass through, and why chunk boundaries survive.
Streaming responses are forwarded incrementally. The gateway does not buffer a stream in order to inspect it.
What is forwarded
Each frame is written to the client as it arrives from the provider. Frame boundaries and
their contents are unchanged, so a client that parses data: lines sees exactly what the
provider emitted.
This matters for anything that depends on framing:
- Incremental rendering in a UI
- Client-side accumulation of tool-call arguments across frames
- Providers that encode usage or finish reasons in dedicated frames
What is observed
Metrics and logs are produced from a copy of the stream, taken as it passes. The copy is bounded:
| Bound | Purpose |
|---|---|
| A fixed capture size | A copy never grows without limit |
| Operator-configurable caps | How much of a body may be recorded in a span |
| Skipped entirely when disabled | No copy is taken when no exporter is configured |
When a stream exceeds a capture bound, the copy is truncated and the truncation is recorded. The client’s stream is unaffected.
Early errors inside a successful response
Some providers return 200 and then report an error as the first stream event. The
gateway recognises startup events for the dialects it understands, so a failure reported
that way can still trigger a retry or a fallback rather than being handed to the client as
a terminal stream error.
Client disconnects
If the client disconnects, the upstream request is cancelled. No partially consumed response is recorded as a success.