Chat Completions

Create a chat completion using any configured provider through a single OpenAI-compatible endpoint.

POST /v1/chat/completions

Required capability: chat

Request Body

Parameter	Type	Required	Description
`model`	`string`	Yes	Model identifier (e.g. `gpt-4`, `claude-3-opus`, `gemini-pro`). The gateway routes to the appropriate provider.
`messages`	`array`	Yes	Array of message objects (minimum 1). See Message Format.
`temperature`	`number`	No	Sampling temperature between 0 and 2. Higher values produce more random output.
`top_p`	`number`	No	Nucleus sampling parameter between 0 and 1.
`max_tokens`	`integer`	No	Maximum number of tokens to generate.
`max_completion_tokens`	`integer`	No	Alternative to `max_tokens`. Maximum completion tokens to generate.
`stream`	`boolean`	No	If `true`, responses are streamed as server-sent events. Defaults to `false`.
`stop`	`string \| string[]`	No	Sequences where the model stops generating.
`n`	`integer`	No	Number of completions to generate (1-10), returned as `choices[0..n-1]`. Refused with `stream: true` — a streamed response carries one completion. Providers that do not implement `n` return one and are billed for one.
`tools`	`array`	No	List of tool definitions the model may call. Each entry is either a function tool (`type: "function"`) or a gateway built-in tool (`type: "built_in"`). See Built-in Tools.
`tool_choice`	`string \| object`	No	Controls tool calling: `"auto"`, `"none"`, `"required"`, or a specific function.
`response_format`	`object`	No	Output format: `{"type": "text"}`, `{"type": "json_object"}`, or `{"type": "json_schema", "json_schema": {...}}`.
`reasoning_effort`	`string`	No	Reasoning budget for reasoning-capable models: `"minimal"`, `"low"`, `"medium"`, or `"high"`. Passed through to the provider’s native reasoning/thinking parameter.
`reasoning`	`object`	No	Fine-grained reasoning controls: `{ "effort": "minimal\|low\|medium\|high", "budget_tokens": <int>, "exclude_from_output": <bool> }`. The adapter maps these to the provider’s native thinking-budget mechanism.
`cache_control`	`object`	No	Prompt-caching directives: `{ "cache_prefix": <bool>, "ttl_seconds": <int> }`. The adapter selects the provider’s native caching mechanism.
`seed`	`integer`	No	Seed for deterministic sampling (best-effort).
`user`	`string`	No	End-user identifier for abuse tracking.
`frequency_penalty`	`number`	No	Penalty for token frequency, between `-2.0` and `2.0`. Positive values discourage repetition.
`presence_penalty`	`number`	No	Penalty for token presence, between `-2.0` and `2.0`. Positive values encourage new topics.
`logprobs`	`boolean`	No	If `true`, return per-token log probabilities in the response.
`top_logprobs`	`integer`	No	Number of top alternative tokens to include in `logprobs`, between `0` and `20`. Requires `logprobs: true`.
`logit_bias`	`object`	No	Map of token-id (string) to bias value between `-100` and `100`. Adjusts the likelihood of specific tokens appearing.
`rag`	`boolean \| object`	No	Ask for retrieval from the Documents corpus. `true` uses the defaults; the object form takes `{ "document_ids": ["<id>"], "top_k": <int 0-100>, "threshold": <0-1> }`. Gateway extension — not sent to the provider. See Retrieval.

Message Format

Each message object has the following structure:

Field	Type	Required	Description
`role`	`string`	Yes	One of `system`, `user`, `assistant`, or `tool`.
`content`	`string \| array \| null`	Yes	The message content. Can be a string, an array of content parts (text or image_url), or `null` for assistant messages with tool calls.
`name`	`string`	No	Name of the message author.
`tool_calls`	`array`	No	Tool calls made by the assistant. Each has `id`, `type: "function"`, and `function: {name, arguments}`.
`tool_call_id`	`string`	No	For `tool` role messages, the ID of the tool call being responded to.

Multimodal Content

The content field can be an array of content parts for vision requests:

[
  { "type": "text", "text": "What is in this image?" },
  { "type": "image_url", "image_url": { "url": "https://example.com/image.png", "detail": "auto" } }
]

The detail field accepts "auto", "low", or "high".

Response

{
  "id": "chatcmpl-abc123",
  "object": "chat.completion",
  "created": 1709000000,
  "model": "gpt-4",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "Hello! How can I help you today?"
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 12,
    "completion_tokens": 9,
    "total_tokens": 21
  }
}

Streaming

Set "stream": true to receive responses as server-sent events (SSE). The response uses Content-Type: text/event-stream.

Each event is a JSON object prefixed with data: :

data: {"id":"chatcmpl-abc123","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"content":"Hello"},"finish_reason":null}]}

data: {"id":"chatcmpl-abc123","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"content":"!"},"finish_reason":null}]}

data: [DONE]

The stream terminates with data: [DONE].

Headers

The response includes:

Header	Description
`X-Request-ID`	Unique identifier for the request, useful for debugging and audit trails.
`X-Provider`	The provider the request was routed to (e.g. `openai`, `anthropic`).
`X-Model`	The resolved model that served the request.

Example

curl https://your-gateway.example.com/v1/chat/completions \
  -H "Authorization: Bearer aigw_sk_your_api_key" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4",
    "messages": [
      {"role": "system", "content": "You are a helpful assistant."},
      {"role": "user", "content": "Explain quantum computing in one sentence."}
    ],
    "temperature": 0.7,
    "max_tokens": 100
  }'

Streaming Example

curl https://your-gateway.example.com/v1/chat/completions \
  -H "Authorization: Bearer aigw_sk_your_api_key" \
  -H "Content-Type: application/json" \
  -N \
  -d '{
    "model": "gpt-4",
    "messages": [{"role": "user", "content": "Write a haiku about APIs."}],
    "stream": true
  }'

Tool Calling Example

curl https://your-gateway.example.com/v1/chat/completions \
  -H "Authorization: Bearer aigw_sk_your_api_key" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4",
    "messages": [{"role": "user", "content": "What is the weather in London?"}],
    "tools": [
      {
        "type": "function",
        "function": {
          "name": "get_weather",
          "description": "Get the current weather for a location",
          "parameters": {
            "type": "object",
            "properties": {
              "location": {"type": "string"}
            },
            "required": ["location"]
          }
        }
      }
    ],
    "tool_choice": "auto"
  }'

Built-in Tools

In addition to function tools, the gateway accepts built-in tools that map to provider-native capabilities. Each built-in tool uses type: "built_in" with a built_in name and an optional config object:

`built_in`	Description
`file_search`	Gateway-side retrieval over your vector stores. The gateway retrieves matching chunks and injects them as system context, then strips the tool before the request reaches the provider. Pass `config: { "vectorStoreIds": [...], "topK": <int> }`.
`web_search`	Maps to the provider’s native web-search tool (where supported, e.g. Anthropic, Gemini).
`code_execution`	Maps to the provider’s native code-execution tool.
`computer_use`	Maps to the provider’s native computer-use tool.
`search_grounding`	Maps to Gemini’s search-grounding capability.

{
  "model": "gpt-4o",
  "messages": [{ "role": "user", "content": "Summarize our onboarding policy." }],
  "tools": [
    {
      "type": "built_in",
      "built_in": "file_search",
      "config": { "vectorStoreIds": ["vs_abc123"], "topK": 5 }
    }
  ]
}

The file_search built-in is handled entirely by the gateway and works with any chat-capable provider. The remaining built-ins are forwarded to providers that support the corresponding native tool.

Retrieval

Set rag to have the gateway search the Documents corpus with the last user message and paste the best-matching chunks into the system prompt before the request reaches a provider.

{
  "model": "gpt-4o",
  "messages": [{ "role": "user", "content": "What is our notice period?" }],
  "rag": true
}

The object form narrows it:

{
  "rag": {
    "document_ids": ["665f1c2a9b4e5d6f7a8b9c0d"],
    "top_k": 3,
    "threshold": 0.8
  }
}

Field	Type	Description
`document_ids`	`string[]`	Restrict retrieval to these documents. Ids, not filenames. Narrows within the caller’s scope; it cannot reach a document the caller could not otherwise read.
`top_k`	`integer`	How many chunks to paste, `0`–`100`. Defaults to 5. `0` means do not augment this request — it is an instruction, not an omission.
`threshold`	`number`	Minimum cosine similarity, `0`–`1`. Defaults to 0.7. `0` accepts anything the corpus returns.

Unrecognised keys are refused rather than ignored, so a misspelt option is a 400 and never a silently unretrieved request.

Retrieval is included at every licence tier. Two things can stop it:

The serving routing rule’s enableRag switch. It is a veto, not a trigger — a request that does not ask is never augmented, whatever the rule says, and a rule with retrieval switched off refuses one that does ask.
The embedding service being unreachable. The request is still served, answered from the model’s own knowledge without the corpus. That is a deliberate policy — retrieval does not block a completion — and it is counted as aigw_retrieval_embedding_failures_total{surface="rag"} so it can be alerted on rather than silently changing answers.

rag is a gateway extension. It is consumed here and never forwarded to a provider. /v1/completions and /v1/messages accept it on the same terms.

Gateway Features

The chat completions endpoint passes through the full gateway middleware pipeline:

Request normalization — requests are translated to a unified internal format, enabling routing to any provider.
Retrieval — a request that sets rag has the best-matching chunks of the Documents corpus pasted into its system prompt. See Retrieval.
Prompt guards — configurable content filters for injection detection, PII, and toxicity.
Budget checks — requests are blocked if any budget in the caller’s scope, or the API key budget, has been exceeded.
Semantic cache — identical or semantically similar requests may be served from cache.
Usage tracking — token usage is recorded for billing and analytics.
Audit logging — requests are logged for compliance.