Tool-Calling, JSON-Schema Mode, and Audio Endpoints Land in the AI Gateway
The AI Gateway now handles tool-calling and structured JSON output consistently across providers, and adds speech and transcription endpoints.
Three related gaps in the AI Gateway's OpenAI-compatible API are closed with this change, all found while auditing the gateway against what a serious multi-provider API needs to support.
Tool-calling, made consistent
Before this, whether your tools and tool_choice parameters actually reached the model depended on which provider you'd routed to. Anthropic silently dropped tool_choice. Google and Cohere dropped tools entirely. Azure OpenAI dropped both. Now each provider gets its own correct translation — Anthropic's tool_choice shape, Gemini's functionDeclarations/toolConfig, Cohere's parameter_definitions — and the model's tool-call response is translated back into the standard OpenAI shape regardless of which provider actually answered.
Structured output via json_schema
response_format now supports OpenAI's json_schema structured-output mode. For OpenAI-compatible providers it's passed straight through; for Gemini it's mapped to that provider's native responseSchema. For providers that have no equivalent — Anthropic, Cohere, Workers AI, Azure Foundry, Cloudflare — a request asking for it now gets a clear, named error instead of being silently ignored and answered as free-form text.
A real routing bug, found in the process
While consolidating this, we found that three providers — xAI, MiniMax, and OpenRouter — had no handling at all on the gateway's direct request path (the one taken whenever no Cloudflare AI account is configured, which is the default in testing and in some deployments). Every request to one of those three was failing outright with "Unsupported provider." That's fixed as part of this same change, along with several other providers that were silently dropping tools/tool_choice/response_format on that same path.
Audio endpoints, newly wired up
POST /v1/audio/speech and POST /v1/audio/transcriptions were defined in the gateway's types but never actually mounted as routes. They are now — OpenAI only for the moment, with exact per-character pricing for speech and duration-based settlement for transcription. Multi-provider and BYOK support for audio is a follow-up, not part of this change.