Gateway troubleshooting and observability

Organized as symptom → checks → deeper docs. Tenant keys, model allowlists, upstream routes, etc. follow your deployment README and service source.

HTTP status quick reference

StatusCommon causes
401Missing/wrong Bearer, disabled or deleted key.
403Model not on key allowlist (TenantModelNotAllowedError); no usable upstream key after binding (TenantUpstreamKeysExhaustedError); IP allowlist (ip_not_allowed).
400Zod body validation; invalid input_file; unknown MIME, etc.
502Upstream APICallError (body often includes upstream_error).
501Unsupported capability (ChatCompletionsNotSupportedError).

Streaming 200 with mid-stream failure: Tool calls and streaming.

Models and routing

  • model spelling exactly matches custom_models.code?
  • Does GET /v1/models list the model?
  • Does the tenant API key allowlist omit the target model?
  • Upstream missing key / wrong binding? Cross-check README and routing code.

Multimodal and PDF

  • File parts use input_file three-way rule? Chat Completions.
  • PDF preprocess failures: error.code (pdf_password_required, pdf_expand_too_large, …) and GATEWAY_PDFIUM_WASM_PATH.

Proxies and SSE

  • Nginx / Envoy buffering delaying first bytes or breaking streams.
  • Timeouts killing long answers or tool chains.
  • HTTP/2 vs client SSE quirks (sometimes force HTTP/1.1 for this path).

Logs and tracing

  • Pass x-trace-id through this service to upstream logs for correlation with tenant usage and trace systems (SDK: SDK quickstart).

See also

Back to docs home