All pages

Chat completions

POST /v1/chat/completions is the main conversation endpoint and matches the OpenAI contract exactly: the same model + messages body, the same response. SSE streaming, tool calling, image (vision) input, and audio input are supported. Every chat model listed by GET /v1/models works with this endpoint.

POST/v1/chat/completionsBearer auth

Generate a chat completion

Send a conversation to an LLM and get a response. Supports all chat models: GPT-4o, Claude, Llama, Gemini, and 350+ more.

Two modes:

  • Non-streaming (default): Returns the full response as JSON. Compatible with openai-python, openai-node, and any OpenAI SDK.
  • Streaming ("stream": true): Returns Server-Sent Events (SSE). Tokens arrive as they're generated, perfect for real-time chat UIs.

Response headers:

  • X-Hormouz-Admission-Latency-Ms: Time spent on auth + billing hold
  • X-Hormouz-Upstream-Latency-Ms: Time waiting for the AI provider
  • X-Hormouz-Proxy-Latency-Ms: Total proxy overhead

Parameters

FieldinTypeDescription
Idempotency-Keyheaderstring

Optional deduplication token (any string up to 200 characters).

How it works:

  • Same key + same request body → cached response replayed (safe retry)
  • Same key + different body → 409 CONFLICT error
  • Keys expire after 24 hours

When to use: Add this header when you want to safely retry a request after a network timeout without risking a duplicate operation.

Request bodyrequired

FieldTypeDescription
modelrequiredstring

Model identifier from the catalogue. Use GET /v1/models to browse available models.

Examples: openai/gpt-4o, anthropic/claude-sonnet-4, meta-llama/llama-3.1-70b-instruct, google/gemini-2.5-flash.

Example: openai/gpt-4o

messagesrequiredarray of object

Conversation history as an array of message objects. Each message has a role and content.

Roles:

  • system: Sets the AI's behavior and personality
  • user: Your message / question
  • assistant: The AI's previous response (for multi-turn)
  • tool: Result of a tool/function call
rolerequiredstring

Who sent this message.

Allowed values: system · user · assistant · tool

contentrequiredstring | array of object

Message content. For text-only messages, a simple string.

For multimodal inputs, an array of content parts. Three supported part types: text, image_url, input_audio.

Vision (image) example:

[
  {"type": "text", "text": "What is in this image?"},
  {"type": "image_url", "image_url": {"url": "https://..."}}
]

Audio (transcription / analysis) example:

[
  {"type": "text", "text": "Transcribe this audio."},
  {"type": "input_audio", "input_audio": {
    "data": "<base64-encoded bytes>",
    "format": "webm"
  }}
]

Audio is supported on POST /v1/chat/completions only; POST /v1/responses has no audio content type today and returns 400 AUDIO_NOT_SUPPORTED_ON_RESPONSES if input_audio is present. Per-model coverage varies; see the API guide §8b for the model matrix.

streamboolean

When true, the response is streamed as Server-Sent Events (SSE). Each data: … line is a JSON chunk carrying a partial completion; the stream ends with data: [DONE].

Recommended for chat interfaces: tokens appear as they're generated instead of waiting for the full response.

The chunk before [DONE] carries a usage object with the upstream's authoritative cost, prompt_tokens, and completion_tokens. Your hold is settled against this number within the same response, so balance accuracy is independent of /generation polling.

Measured timing (Iran → gateway → upstream, production):

  • gpt-* models: ~570ms TTFB, 4–8ms inter-chunk avg, 60–95 tok/s
  • gemini-2.5-flash: ~470ms TTFB, 18–80ms inter-chunk avg, 45–70 tok/s
  • Gateway admission overhead: ~3–5ms per request (negligible)
  • Concurrent streams scale near-linearly up to your per-key stream-concurrency cap (default 10).

Default: false

sorted_bystring

Which upstream provider to prefer when the model is served by more than one.

Most models are hosted by several providers whose price and speed differ substantially — on a single popular model the input price ranges more than 8x and the slowest provider takes several times longer than the fastest. By default the gateway lets the upstream router choose. Set this to steer that choice:

  • cheapest — lowest price per token
  • latency — lowest time to first token
  • throughput — highest tokens per second

Ranking is by median (p50) over a rolling 5-minute window of live traffic, so it follows current conditions including temporary provider discounts.

This is a preference, not a guarantee. If the preferred option is unavailable or rate-limited, the request is served by the next one rather than failing. It has no effect on models with a single provider.

Setting it changes how the request is billed. A sorted request goes only through the standard multi-provider route and bills at the model's standard list price: any discounted route the model normally takes (shown as a discount badge on its model page) is skipped for that request.

Any value other than the three above is rejected with 400 INVALID_SORTED_BY; the error message lists the supported values.

Models where this applies are marked supports_sorted_by in GET /v1/models.

Allowed values: cheapest · latency · throughput

Example: cheapest

max_tokensinteger

Maximum number of tokens to generate in the response. Higher values allow longer responses but cost more.

Default: 4096

temperaturenumber (double)

Controls randomness (0.0 to 2.0).

  • 0.0: Deterministic, always picks the most likely token
  • 0.7: Balanced (good default for chat)
  • 1.5+: Creative, more varied outputs
top_pnumber (double)

Nucleus sampling threshold (0.0 to 1.0). Alternative to temperature; usually set one or the other, not both.

toolsarray of object

Tool/function definitions for the model to call. Two flavours coexist freely in the same array:

  1. User-defined function tools (the OpenAI shape):

    {"type":"function","function":{"name":"get_weather","parameters":{…}}}
    

    Forwarded byte-for-byte to the upstream. Requires supports_tools: true in the model catalogue.

  2. Built-in server tools: work on every model, no supports_tools requirement:

    typeWhat it doesOptional parameters
    web_searchLive web search; citations on choices[0].message.annotations[]max_results (default 3), max_total_results, search_context_size, allowed_domains, excluded_domains
    web_fetchFetch a single URL's contentmax_content_tokens (default 50000), max_uses (default 10, hard cap 50), allowed_domains, blocked_domains
    datetimeCurrent date/time (free)timezone (IANA, default UTC)
    image_generationModel-driven image gennone documented

Not supported (returns 400 INVALID_TOOLS):

  • "model":"…:online" suffix
  • "plugins":[{"id":"web"}] array
  • "tools":[{"type":"<internal>:*"}] (internal namespace: drop the prefix)
tool_choicestring | object

Controls tool calling behavior:

  • "auto": Model decides when to call tools (default)
  • "none": Never call tools
  • {"type": "function", "function": {"name": "..."}}: Force specific tool
stopstring | array of string

Stop sequence(s). Generation stops when any of these strings appear in the output. Up to 4 sequences.

Example

Responses

200headers: X-Hormouz-Admission-Latency-Ms, X-Hormouz-Upstream-Latency-Ms, X-Hormouz-Proxy-Latency-Ms, X-Suggest-Stream, X-Client-Timeout-Hint

Success. Response format depends on the stream parameter:

  • stream: false (default) → application/json with full response
  • stream: true → text/event-stream with SSE chunks
400

The request body or query parameters are invalid. Check the error.code and error.message fields for details. Codes the gateway emits at 400:

  • BAD_REQUEST: generic malformed JSON / missing required field.
  • INVALID_TOOLS: the tools array, plugins field, or model's :online suffix uses an unsupported shape. The response names the offender and links to the supported form. See the dedicated InvalidTools response below for the three triggers and migration hints.
  • UNSUPPORTED_TOOL: a built-in tool (file_search, code_interpreter, computer_use, mcp) whose pricing dimension is not yet wired into the gateway hold estimator.
  • INVALID_AUDIO_FORMAT: input_audio.format value is not in the supported set. The envelope lists every supported codec name. The gateway accepts both short names (wav, webm) and full MIME aliases (audio/wav, audio/webm); anything else fails fast at admission. See API guide §8b.
  • AUDIO_NOT_SUPPORTED_ON_RESPONSES: input_audio content part sent to POST /v1/responses. The Responses API has no audio content type today; resend to POST /v1/chat/completions instead.
401

Authentication failed. Either the Authorization header is missing, the key format is invalid, or the key doesn't match any active account.

{
  "error": {
    "code": "UNAUTHORIZED",
    "message": "Authentication required",
    "status": 401
  }
}
402

Your credit balance is too low to cover the estimated cost of this request. Contact your admin for a top-up.

{
  "error": {
    "code": "INSUFFICIENT_CREDITS",
    "message": "Insufficient credit balance. Add credit at https://app.hormouz.net/billing to continue.",
    "status": 402
  }
}
409

Idempotency key conflict: you sent the same Idempotency-Key with a different request body.

429

Too many requests. Back off and retry after a short delay. Check the Retry-After header for guidance.

499

Your HTTP client cancelled the request before the gateway finished. No charge: the hold has been released. The most common cause is a 30 s default client timeout against a model whose p95 latency is 45-60 s; bump your client timeout to 120 s.

Retries with the same Idempotency-Key will replay this 499 forever (the prior outcome is "you cancelled"). For an actual re-attempt, mint a fresh Idempotency-Key. See docs/CUSTOMER_ERROR_HANDLING.md §3 for the full contract.

{
  "error": {
    "code": "CLIENT_CLOSED",
    "message": "Client disconnected before response",
    "status": 499
  }
}
500

Unexpected server error. Please retry with exponential backoff. If the issue persists, contact your admin.

502

The upstream AI provider is temporarily unavailable. Retry with exponential backoff (1s, 2s, 4s) using the same Idempotency-Key: if the upstream actually completed the work and only the response failed, the gateway replays the cached success without a second charge. Holds are released automatically on upstream failures.

503

The upstream AI provider is temporarily unavailable. Retry with exponential backoff (1s, 2s, 4s) using the same Idempotency-Key: if the upstream actually completed the work and only the response failed, the gateway replays the cached success without a second charge. Holds are released automatically on upstream failures.

504

The gateway's 120 s upstream deadline elapsed before the upstream provider produced a response. Retry with the same Idempotency-Key so a delayed-but-eventually-successful upstream call can be replayed instead of re-charged. Consider switching to a faster model or capping max_tokens if this recurs.

{
  "error": {
    "code": "UPSTREAM_TIMEOUT",
    "message": "Upstream did not respond before timeout",
    "status": 504
  }
}

FAQ

How do I use GPT-4o or Claude from Iran?

Through Hormouz: set base_url to https://api.hormouz.net/v1 and pick a model such as openai/gpt-4o or anthropic/claude-sonnet-4. Stable access from inside Iran with local billing.

Is tool / function calling supported?

Yes, with the exact OpenAI contract (tools, tool_choice). Server-side tools such as web_search additionally work on every model.

Updated 2026-10-13