These pages describe the intended behaviour of the gateway. The open-source release and the enterprise edition are being split out; details will be updated when the first release ships.
Management console
The gateway is configured from its console. The configuration file only bootstraps the process.
Everything about traffic is managed from the console: providers, credentials, routes, routing policy, and alerts. The configuration file is a bootstrap file that tells the process where to listen and which domain it serves.
The split
| Lives in the configuration file | Lives in the console |
|---|---|
| Listen address and port | Providers and their base URLs |
| Public domain | Credentials and key pools |
| Console bind address | Routes |
| State directory | Fallback chains and weights |
| Telemetry exporters | Circuit breaking policy |
| Alert rules and channels |
The rule of thumb: anything you might change on a Tuesday afternoon belongs in the console. Restarting a gateway to reorder a fallback chain is a deployment, and a deployment is the wrong unit for a routing decision.
Reaching the console
The console is served on its own address, configured separately from the traffic port:
[console]
listen = "127.0.0.1:8081"Binding it to a private interface and reaching it over a tunnel or a bastion is the simplest way to keep it off the public internet. See Configuration for the full bootstrap file.
What changes take effect immediately
Adding a provider, rotating a credential, reordering a fallback chain, adjusting a weight and editing an alert rule all apply to in-flight traffic without a restart. There is no reload step and no signal to send.
Where state lives
Everything the console manages is written under the configured state directory. That directory is the source of truth for routing; the process reads it at startup and writes to it on change.
Back it up the way you back up any other piece of production configuration. In Kubernetes
it belongs on a PersistentVolume, not in a ConfigMap — see
Kubernetes.
Credentials
Provider credentials entered in the console are encrypted at rest with a key derived from your deployment’s secret. Losing that secret means re-entering the credentials; it does not mean they leak.
A credential is never shown again after it is saved. The console displays a fingerprint so you can confirm which value is in use.
Access
The console has its own authentication, separate from the credentials used for inference traffic. Callers never need console access, and console users never need an inference key.