These pages describe the intended behaviour of the gateway. The open-source release and the enterprise edition are being split out; details will be updated when the first release ships.
Transparent forwarding
Precisely what the gateway preserves, and what it never touches.
The forward path has one job: move bytes from the client to a provider and back, without changing them.
What is preserved
The gateway does not parse the body for the purpose of rewriting it. The following reach the provider exactly as the client sent them:
- Request body, byte for byte
- Header order and casing
- Query string, including parameter order
- Streaming frame boundaries
- Prompt prefix, so provider-side prompt caching stays valid
What the gateway adds or changes
A short, closed list:
| Change | Reason |
|---|---|
| One credential header (or a signature) | Authenticating to the provider |
Host |
The request is addressed to the provider, not to the gateway |
| Hop-by-hop headers are dropped | Required by HTTP/1.1 and HTTP/2 |
A traceparent header, when tracing is on |
W3C trace context propagation |
Nothing else is added, removed or reordered.
What the gateway reads without changing
Routing needs a few facts before it can pick a target. These are read from a copy of the request and never written back:
- The matched route, to learn which provider may serve this path
- The model name, when a route declares where it appears
- The authenticated identity, for rate limits and attribution
Reading for accounting is not the same as rewriting. The bytes forwarded are the bytes received.
Choosing a route
A request is matched against the routes declared on each provider. A route is a method and a path pattern, optionally with a regular expression when a provider puts something variable in the path.
A request that matches no route is rejected with 404 before any upstream call is made.