# Track client-side analytics events Source: https://docs.verlon.ai/api-reference/analytics/track-client-side-analytics-events /api-reference/openapi.json post /v1/analytics/events **Use when:** caller is reporting batched client-side analytics events (page views, feature usage) — for Verlon's product analytics, not for billing or rate limiting Public endpoint (auth optional). Accepts batched client-side analytics events — page views, button clicks, feature usage. Auth is best-effort: if a valid Bearer token is provided, events are attributed to that user; otherwise the request body's `user_id` / `anonymous_id` is used. Used by Verlon's marketing site and dashboard for product analytics, not for billing or rate limiting. # Anthropic-format messages Source: https://docs.verlon.ai/api-reference/anthropic-format/anthropic-format-messages /api-reference/openapi.json post /v1/messages **Use when:** caller has an existing Anthropic SDK Messages-API flow and wants to route through Verlon with zero code changes via baseURL swap Drop-in replacement for `POST https://api.anthropic.com/v1/messages`. Accepts the upstream Anthropic Messages API request body verbatim and returns the upstream Anthropic response body verbatim — including streaming SSE chunks when `stream: true`. Route through a Verlon gate by passing `model` as a Verlon gate id. Existing Anthropic SDK clients work unmodified by changing the `baseURL` to `https://api.verlon.ai/v1/`. Errors are returned in the Anthropic wrapped shape (`{ type: 'error', error: { type, message } }`) so the Anthropic SDK's error parser handles them transparently. # Anthropic-format token counting Source: https://docs.verlon.ai/api-reference/anthropic-format/anthropic-format-token-counting /api-reference/openapi.json post /v1/messages/count_tokens **Use when:** caller uses the Anthropic SDK count_tokens flow (or an agentic client like Claude Code) against the Verlon drop-in endpoint Drop-in replacement for `POST https://api.anthropic.com/v1/messages/count_tokens`. Returns an estimated `input_tokens` for the given system prompt, messages, and tools. The count is an estimate: gates route to arbitrary providers whose tokenizers differ, so exact provider token counts are only known at completion time. # Anthropic Messages API Source: https://docs.verlon.ai/api-reference/anthropic-messages Anthropic-compatible messages endpoint This documentation page is under development and will be available soon. In the meantime, if you have questions about this topic: * Check our [GitHub](https://docs.verlon.ai) for updates * Reach out to our team at [support@verlon.ai](mailto:support@verlon.ai) ## What's Available Now While this specific documentation is being written, you can explore: > **[Getting Started](/getting-started/quickstart)** > > Quick start guide to get up and running > **[SDK Reference](/sdk-reference/overview)** > > Complete SDK method documentation > **[OpenAI SDK Integration](/integrations/openai-sdk)** > > Use Verlon with OpenAI SDK > **[Gates & Routing](/platform/gates)** > > How Verlon routes requests # Chat Completions API Source: https://docs.verlon.ai/api-reference/chat-completions OpenAI-compatible chat completions endpoint This documentation page is under development and will be available soon. In the meantime, if you have questions about this topic: * Check our [GitHub](https://docs.verlon.ai) for updates * Reach out to our team at [support@verlon.ai](mailto:support@verlon.ai) ## What's Available Now While this specific documentation is being written, you can explore: > **[Getting Started](/getting-started/quickstart)** > > Quick start guide to get up and running > **[SDK Reference](/sdk-reference/overview)** > > Complete SDK method documentation > **[OpenAI SDK Integration](/integrations/openai-sdk)** > > Use Verlon with OpenAI SDK > **[Gates & Routing](/platform/gates)** > > How Verlon routes requests # Check what depends on a gate (before deletion) Source: https://docs.verlon.ai/api-reference/gates/check-what-depends-on-a-gate-before-deletion /api-reference/openapi.json get /v1/gates/:id/dependencies **Use when:** caller is about to delete a gate and wants a preview of what depends on it (sub-gate attachments, agent gates that reference it) to avoid breaking other routes Returns the list of resources that depend on this gate — sub-gate attachments, in-flight sessions, scheduled experiments, etc. Use this as a pre-deletion check to surface impact to the user. # Create a new gate Source: https://docs.verlon.ai/api-reference/gates/create-a-new-gate /api-reference/openapi.json post /v1/gates **Use when:** caller wants to create a new model routing gate so requests can be sent to it via the SDK or compat routes Create a new Verlon gate with a primary model and optional fallback chain. Gates are the unit of routing — every chat/image/audio/etc. request names a gate, and Verlon decides which underlying model+provider serves it based on the gate config. # Declare a task on an agent gate Source: https://docs.verlon.ai/api-reference/gates/declare-a-task-on-an-agent-gate /api-reference/openapi.json post /v1/gates/:id/tasks **Use when:** caller wants to declare a named LLM call site on an agent gate, optionally pinning a model or delegating to an existing gate # Delete a task Source: https://docs.verlon.ai/api-reference/gates/delete-a-task /api-reference/openapi.json delete /v1/gates/:id/tasks/:taskId **Use when:** caller wants to remove a declared or discovered task from an agent gate; calls using its name fall back to pass-through # Duplicate an existing gate Source: https://docs.verlon.ai/api-reference/gates/duplicate-an-existing-gate /api-reference/openapi.json post /v1/gates/:id/duplicate **Use when:** caller wants to clone an existing gate with a new name (template-like reuse, A/B variants, etc.) Creates a copy of an existing gate. Useful for creating staging variants, experiment branches, or templates. Optional `name` in the body overrides the auto-generated copy name. Subject to the caller's tier gate limit. # Export a single gate config as a portable JSON document Source: https://docs.verlon.ai/api-reference/gates/export-a-single-gate-config-as-a-portable-json-document /api-reference/openapi.json get /v1/gates/:id/export **Use when:** caller wants a downloadable snapshot of one gate’s configuration by id Single-gate variant of `GET /export`: returns `{ schemaVersion, exportedAt, gate }` with the same portable-config shape, served as an attachment download. # Export every gate config as a portable JSON document Source: https://docs.verlon.ai/api-reference/gates/export-every-gate-config-as-a-portable-json-document /api-reference/openapi.json get /v1/gates/export **Use when:** caller wants a downloadable snapshot of all their gate/routing configuration — e.g. for backup or to migrate off the platform Returns `{ schemaVersion, exportedAt, gates }` where each gate is its portable configuration (routing strategy, fallback chain, params, agent-gate fields) without runtime state (spending counters, health) or account identifiers. Served with a `Content-Disposition: attachment` header so browser navigation downloads a file. # Get a gate by id Source: https://docs.verlon.ai/api-reference/gates/get-a-gate-by-id /api-reference/openapi.json get /v1/gates/:id **Use when:** caller has a gate id and needs the full gate configuration (most common single-gate read path) Returns a single gate's full configuration plus current health flags (deprecations, on-fallback, unrecoverable provider drift). Health detection failures are absorbed — the gate still returns; only the `health` field is null. # Get a gate by its name Source: https://docs.verlon.ai/api-reference/gates/get-a-gate-by-its-name /api-reference/openapi.json get /v1/gates/name/:name **Use when:** caller has a human-readable gate name (not id) and needs to resolve it to the full gate configuration Lookup by the user-facing gate name. Names are unique per user; returns 404 if not found. Use this when the caller has the name but not the id — e.g. SDK callers that reference gates by their declared name in code. # Get The Architect's model + config suggestions for a gate Source: https://docs.verlon.ai/api-reference/gates/get-the-architects-model-+-config-suggestions-for-a-gate /api-reference/openapi.json post /v1/gates/suggestions **Use when:** caller wants The Architect to analyze recent traffic for a gate and suggest model or configuration changes that would improve cost or quality Runs The Architect — Verlon's optimization agent — against a natural-language gate description and returns concrete suggestions (model, parameters, fallback chain, rationale). Caller controls the cost/latency/quality weighting via the optional `costWeight`/`latencyWeight`/`qualityWeight` fields (defaults to balanced 0.33/0.33/0.34). Optional `availableProviders` constrains the suggestion to a subset of providers. # List activity-log entries across every gate owned by the caller Source: https://docs.verlon.ai/api-reference/gates/list-activity-log-entries-across-every-gate-owned-by-the-caller /api-reference/openapi.json get /v1/gates/activity **Use when:** caller wants to inspect human/system activity across every gate (who changed what, including non-config events) Cross-gate activity-log feed — manual updates, automated optimizations, deletions, restores, rollbacks, etc. Activity rows are higher-level than history (which is field-by-field config diffs). `limit` defaults to 100, max 1000. # List activity-log entries for a specific gate Source: https://docs.verlon.ai/api-reference/gates/list-activity-log-entries-for-a-specific-gate /api-reference/openapi.json get /v1/gates/:id/activity **Use when:** caller wants to see the activity timeline (who edited, when, what changed) for one specific gate Returns the activity log for a single gate — higher-level than history (manual_update, automated_apply, rollback, deletion, etc.). Useful as a per-gate audit trail. # List an agent gate's tasks Source: https://docs.verlon.ai/api-reference/gates/list-an-agent-gates-tasks /api-reference/openapi.json get /v1/gates/:id/tasks **Use when:** caller wants the agent gate's declared and traffic-discovered tasks (named LLM call sites) — the dashboard Tasks table and the SDK's boot-time validation both read this Every task on the gate: declared ones (created here or by the SDK) and traffic-discovered ones (auto-registered when the gateway saw an unknown x-verlon-task name). A task's binding decides call-time config: inline model pin, delegation to another gate, or pass-through when unconfigured. # List configuration history across every gate owned by the caller Source: https://docs.verlon.ai/api-reference/gates/list-configuration-history-across-every-gate-owned-by-the-caller /api-reference/openapi.json get /v1/gates/history **Use when:** caller wants to inspect configuration change history across every gate (cross-gate audit trail, change log) Cross-gate config-history feed — every change to any of the caller's gates, ordered newest first. Each row captures the gate id, the changed fields, and the actor (user vs. The Architect). `limit` defaults to 100, max 1000. # List configuration history for a specific gate Source: https://docs.verlon.ai/api-reference/gates/list-configuration-history-for-a-specific-gate /api-reference/openapi.json get /v1/gates/:id/history **Use when:** caller wants to see the configuration change timeline for one specific gate Returns the config-change history for a single gate — every field that changed, when, and who/what made the change (user vs. The Architect). Useful as the source view for rollback decisions. # List gates owned by the authenticated user Source: https://docs.verlon.ai/api-reference/gates/list-gates-owned-by-the-authenticated-user /api-reference/openapi.json get /v1/gates **Use when:** caller wants to enumerate the gates the authenticated principal owns (typical first step when listing or picking one to update) Returns every gate visible to the caller, each annotated with current health (deprecations, on-fallback status, unrecoverable provider drift). Use this as the listing endpoint for dashboards and agent inventory calls. # List OpenClaw-tagged gates with analytics Source: https://docs.verlon.ai/api-reference/gates/list-openclaw-tagged-gates-with-analytics /api-reference/openapi.json get /v1/gates/openclaw **Use when:** caller is integrating with OpenClaw and needs to enumerate gates tagged for OpenClaw with their analytics summary attached Returns every gate the caller owns that is tagged with `"openclaw"`, annotated with request count, total cost, average latency, last-request timestamp, and current spending state. Used by the OpenClaw integration dashboard. # List soft-deleted (trashed) gates Source: https://docs.verlon.ai/api-reference/gates/list-soft-deleted-trashed-gates /api-reference/openapi.json get /v1/gates/trash/list **Use when:** caller wants to see soft-deleted gates that can still be restored (within the 30-day trash window) Returns every gate the caller has soft-deleted within the 30-day retention window. Restorable via `POST /v1/gates/:id/restore` until the retention window expires and the trash-cleanup worker hard-deletes them. # Make this gate its connector's default for new connections Source: https://docs.verlon.ai/api-reference/gates/make-this-gate-its-connectors-default-for-new-connections /api-reference/openapi.json post /v1/gates/:id/default **Use when:** caller wants `verlon connect` (and other connector setup flows) to resolve to this gate when no gate is named — the interactive picker pre-highlights it and non-interactive connects use it outright Marks this gate as the user's default for its connector (e.g. claude-code). At most one default exists per connector; setting a new one atomically unsets the previous. Rejected for `connector: 'sdk'` gates — nothing connects to an sdk gate, so a default is meaningless there. # Per-gate trace stats for every gate owned by the caller (batch) Source: https://docs.verlon.ai/api-reference/gates/per-gate-trace-stats-for-every-gate-owned-by-the-caller-batch /api-reference/openapi.json get /v1/gates/metrics/traces **Use when:** caller wants per-gate trace statistics (counts, activity, cost) for the whole inventory in one call — powers the agent-gates list without N+1 fetches # Pin or unpin a gate's BYOK key Source: https://docs.verlon.ai/api-reference/gates/pin-or-unpin-a-gates-byok-key /api-reference/openapi.json put /v1/gates/:id/provider-key **Use when:** caller wants this gate to serve on a specific BYOK key instead of the provider default (or to clear that pin) Pins the gate to a specific BYOK key (`providerKeyId`) or clears the pin (`null`). A pinned gate keeps serving on its key through provider-default rotations. A pin whose key is later deactivated, deleted, or belongs to a different provider than the serving model falls through to the provider default at request time — pinning never breaks a gate. # Restore a soft-deleted gate from the trash Source: https://docs.verlon.ai/api-reference/gates/restore-a-soft-deleted-gate-from-the-trash /api-reference/openapi.json post /v1/gates/:id/restore **Use when:** caller wants to restore a soft-deleted gate from the trash before its 30-day window expires Reverses a soft-delete — moves the gate from trash back to active. Only works within the 30-day retention window before the trash-cleanup worker hard-deletes the row. # Roll a gate back to a previous configuration Source: https://docs.verlon.ai/api-reference/gates/roll-a-gate-back-to-a-previous-configuration /api-reference/openapi.json post /v1/gates/:id/rollback **Use when:** caller wants to revert a gate to a previous configuration version (recovering from a bad change, etc.) Restores a gate to a specific historical config identified by `historyId` (from `/v1/gates/:id/history`). Optionally also invalidates the gate's memory entries (`rollbackMemory: true`) — useful when rolling back a config change The Architect's memory was advising against. # Soft-delete a gate (moves to trash for 30 days) Source: https://docs.verlon.ai/api-reference/gates/soft-delete-a-gate-moves-to-trash-for-30-days /api-reference/openapi.json delete /v1/gates/:id **Use when:** caller wants to delete a gate (soft-delete; stays in trash for 30 days before permanent removal) Soft-deletes a gate — it moves to the trash for a 30-day retention window before the trash-cleanup worker hard-deletes it. Rejected with 409 if the gate has active dependencies (sub-gate attachments, running sessions); pass `?force=true` to override (use with caution — the trash-cleanup worker handles related cleanup downstream). # Stop this gate being its connector's default Source: https://docs.verlon.ai/api-reference/gates/stop-this-gate-being-its-connectors-default /api-reference/openapi.json delete /v1/gates/:id/default **Use when:** caller wants new connections to go back to asking (or creating) instead of resolving to this gate Clears the default mark from this gate. Idempotent — clearing a gate that is not the default succeeds with `changed: false`. # Test a gate configuration with a sample request Source: https://docs.verlon.ai/api-reference/gates/test-a-gate-configuration-with-a-sample-request /api-reference/openapi.json post /v1/gates/test **Use when:** caller wants to send a sample request through a gate (production or draft) to verify routing, parameter handling, and output without leaving an audit trail Runs a quick test against a gate's primary model (and optionally every fallback model) with the supplied messages, returning per-model success/latency/content/error. Accepts either a stored `gateId` (looks up the saved config), an inline `gate` config override, or both (the override merges over the saved config). Set `quickTest: true` to skip the fallback walk and only test the primary. # Update a gate Source: https://docs.verlon.ai/api-reference/gates/update-a-gate /api-reference/openapi.json patch /v1/gates/:id **Use when:** caller wants to change a gate's configuration (model, fallbacks, parameters, system prompt, allowed overrides) Partial update of a gate's configuration. Validates model references at the API boundary; reject unknown models in `fallbackModels` and `modelPool`, but only when those fields are being explicitly set (so edits to other fields on a gate with pre-existing typo'd fallbacks aren't blocked). Triggers history + activity log entries when significant fields change. # Update a task's config Source: https://docs.verlon.ai/api-reference/gates/update-a-tasks-config /api-reference/openapi.json patch /v1/gates/:id/tasks/:taskId **Use when:** caller wants to change a task's model pin or gate binding — the remote-control action for one call site # Gemini-format content generation Source: https://docs.verlon.ai/api-reference/google-format/gemini-format-content-generation /api-reference/openapi.json post /v1beta/models/{model}:generateContent **Use when:** caller has an existing Google Gemini SDK generateContent flow and wants to route through Verlon with zero code changes via a baseUrl swap Drop-in replacement for `POST https://generativelanguage.googleapis.com/v1beta/models/{model}:generateContent`. Accepts the upstream Gemini request body verbatim and returns the upstream Gemini response body verbatim. Authenticate with `x-goog-api-key` (or `?key=`) carrying a Verlon API key. Errors are returned in the Google envelope (`{ error: { code, message, status } }`) so the `@google/genai` error parser handles them transparently. # Gemini-format streaming content generation Source: https://docs.verlon.ai/api-reference/google-format/gemini-format-streaming-content-generation /api-reference/openapi.json post /v1beta/models/{model}:streamGenerateContent **Use when:** caller wants a streamed Gemini generateContent response routed through a Verlon gate Streaming counterpart to `:generateContent`. Pass `?alt=sse` for Server-Sent Events (what every official Google SDK requests); without it the response is a streamed JSON array, matching the REST default. # Gemini-format token counting Source: https://docs.verlon.ai/api-reference/google-format/gemini-format-token-counting /api-reference/openapi.json post /v1beta/models/{model}:countTokens **Use when:** caller uses the Gemini SDK countTokens flow against the Verlon drop-in endpoint Drop-in replacement for `POST .../v1beta/models/{model}:countTokens`. Returns an estimated `totalTokens`. The count is an estimate: gates route to arbitrary providers whose tokenizers differ, so exact provider token counts are only known at completion time. # Mistral-format chat completions Source: https://docs.verlon.ai/api-reference/mistral-format/mistral-format-chat-completions /api-reference/openapi.json post /mistral/v1/chat/completions **Use when:** caller has an existing @mistralai/mistralai SDK flow and wants to route through Verlon with zero code changes via a serverURL swap Drop-in replacement for `POST https://api.mistral.ai/v1/chat/completions`. Point the Mistral SDK at `serverURL: "https://api.verlon.ai/mistral"`. Accepts the Mistral dialect (`random_seed`, `safe_prompt`, `tool_choice: "any"`) on top of the shared OpenAI-shape pipeline, and returns errors in the Mistral envelope (`{ object: "error", message, type, param, code }`) so the vendor SDK parses them. Streaming works unmodified — Mistral stream chunks are OpenAI-shaped. # Mistral-format FIM completions Source: https://docs.verlon.ai/api-reference/mistral-format/mistral-format-fim-completions /api-reference/openapi.json post /mistral/v1/fim/completions **Use when:** caller uses mistral.fim.complete / fim.stream (Codestral code completion with prompt + suffix) and wants to route through Verlon via a serverURL swap Drop-in replacement for `POST https://api.mistral.ai/v1/fim/completions` — Codestral fill-in-the-middle. Send `prompt` (code before the cursor) and `suffix` (code after it); the model completes the middle. Point the Mistral SDK at `serverURL: "https://api.verlon.ai/mistral"`. Streaming via `stream: true` returns SSE chunks. Errors use the Mistral envelope. `min_tokens` is accepted but not forwarded. # Mistral-format OCR Source: https://docs.verlon.ai/api-reference/mistral-format/mistral-format-ocr /api-reference/openapi.json post /mistral/v1/ocr **Use when:** caller uses mistral.ocr.process (document or image to markdown pages) and wants to route through Verlon via a serverURL swap Drop-in replacement for `POST https://api.mistral.ai/v1/ocr`. Send a `document` of type `document_url` or `image_url`; pages come back as markdown with optional extracted images, headers, and footers. Point the Mistral SDK at `serverURL: "https://api.verlon.ai/mistral"`. Errors use the Mistral envelope. `pages`, `image_limit`, `image_min_size`, and the annotation formats are accepted but not forwarded. # Exchange a refresh token for a fresh access + refresh pair Source: https://docs.verlon.ai/api-reference/oauth/exchange-a-refresh-token-for-a-fresh-access-+-refresh-pair /api-reference/openapi.json post /v1/oauth/token/refresh **Use when:** caller's access token has expired (or is about to) and they need a new pair — supplying their refresh_token in exchange Single-use rotation: each refresh token can be exchanged exactly once. Presenting a token that's already been consumed is treated as a leak signal — the entire refresh-token chain (every token derived from the same root, in either direction) is revoked and the caller is required to re-authenticate via the device flow. # Poll a device-flow for completion (RFC 8628 §3.4) Source: https://docs.verlon.ai/api-reference/oauth/poll-a-device-flow-for-completion-rfc-8628-§34 /api-reference/openapi.json post /v1/oauth/device/token **Use when:** caller (typically a CLI) is polling for the user to approve the device-code flow it started — returns the access_token + refresh_token once approved, or a structured error code (authorization_pending / slow_down / expired_token / access_denied) while still pending Per RFC 8628, errors during the polling phase use a 400 status with a stable `error` code the CLI branches on: `authorization_pending` (keep polling), `slow_down` (back off), `expired_token` (start a new flow), `access_denied` (user clicked Deny). Successful approval returns 200 with the issued token pair. # Start an OAuth device authorization flow (RFC 8628 §3.1) Source: https://docs.verlon.ai/api-reference/oauth/start-an-oauth-device-authorization-flow-rfc-8628-§31 /api-reference/openapi.json post /v1/oauth/device/authorize **Use when:** caller is a CLI / IDE plugin / MCP server starting a device-flow login — server returns a device_code (long, opaque) + user_code (short, human-typeable) plus the verification URL where the user lands to approve No auth required — this is how a client without credentials initiates one. Returns the device_code (the polling credential), the user_code (the human-typed identifier), the verification_uri, the expires_in (TTL in seconds), and the interval (polling cadence). # Extract text from an image or PDF via a Verlon gate Source: https://docs.verlon.ai/api-reference/ocr/extract-text-from-an-image-or-pdf-via-a-verlon-gate /api-reference/openapi.json post /v3/ocr **Use when:** caller has an image or PDF and wants the extracted text (structured OCR) routed through a Verlon gate Routes an OCR request through the named Verlon gate. The gate decides which underlying OCR model+provider serves the request based on its current configuration. Accepts images and PDFs; returns structured text extraction. # List available models (drop-in) Source: https://docs.verlon.ai/api-reference/openai-compat/list-available-models-drop-in /api-reference/openapi.json get /v1/models **Use when:** caller wants to discover which model ids serve, via an existing OpenAI or Anthropic SDK `models.list()` call Drop-in replacement for `GET https://api.openai.com/v1/models`. Lists the generally-available catalog — every id returned is routable as `model` on the completion routes. A request carrying `anthropic-version` or `x-api-key` instead receives the Anthropic paginated shape (`limit`, `before_id`, `after_id`) documented on the Anthropic drop-in page. # Retrieve a model (drop-in) Source: https://docs.verlon.ai/api-reference/openai-compat/retrieve-a-model-drop-in /api-reference/openapi.json get /v1/models/:model **Use when:** caller wants to confirm a single model id serves, via an existing OpenAI or Anthropic SDK `models.retrieve()` call Drop-in replacement for `GET https://api.openai.com/v1/models/{model}`. Unknown or retired ids return 404 in OpenAI's error envelope. # OpenAI-format chat completions Source: https://docs.verlon.ai/api-reference/openai-format/openai-format-chat-completions /api-reference/openapi.json post /v1/chat/completions **Use when:** caller is migrating an existing OpenAI SDK integration to Verlon with zero code changes — only the `baseURL` swaps to `https://api.verlon.ai/v1/` and `model` becomes a Verlon gate id Drop-in replacement for `POST https://api.openai.com/v1/chat/completions`. Accepts the upstream OpenAI request body verbatim and returns the upstream OpenAI response body verbatim — including streaming SSE chunks when `stream: true`. Route the request through a Verlon gate by passing `model` as a Verlon gate id. Existing OpenAI SDK clients work unmodified by changing only the `baseURL` to `https://api.verlon.ai/v1/`. Errors are returned in the OpenAI nested shape (`{ error: { message, type, code, param? } }`) so the OpenAI SDK's error parser handles them transparently. # OpenAI-format embeddings Source: https://docs.verlon.ai/api-reference/openai-format/openai-format-embeddings /api-reference/openapi.json post /v1/embeddings **Use when:** caller has an existing OpenAI SDK embeddings flow and wants to route through Verlon with zero code changes via baseURL swap Drop-in replacement for `POST https://api.openai.com/v1/embeddings`. Accepts the upstream OpenAI embeddings request body verbatim and returns the upstream OpenAI response body verbatim. Route through a Verlon gate by passing `model` as a Verlon gate id. Existing OpenAI SDK clients work unmodified via baseURL swap. Errors are returned in the OpenAI nested shape. # OpenAI-format image generations Source: https://docs.verlon.ai/api-reference/openai-format/openai-format-image-generations /api-reference/openapi.json post /v1/images/generations **Use when:** caller has an existing OpenAI SDK image-generation flow and wants to route through Verlon with zero code changes via baseURL swap Drop-in replacement for `POST https://api.openai.com/v1/images/generations`. Accepts the upstream OpenAI image-generation request body verbatim and returns the upstream OpenAI response body verbatim. Route the request through a Verlon gate by passing `model` as a Verlon gate id. Existing OpenAI SDK clients work unmodified via baseURL swap. Errors are returned in the OpenAI nested shape. # OpenAI-format Responses API Source: https://docs.verlon.ai/api-reference/openai-format/openai-format-responses-api /api-reference/openapi.json post /v1/responses **Use when:** caller uses the OpenAI Responses API (client.responses.create) — including the Agents SDK and multi-turn tool loops with encrypted reasoning replay — and wants to route through a Verlon gate by swapping only the baseURL Drop-in for `POST https://api.openai.com/v1/responses`. Accepts the Responses request body verbatim and returns the upstream Responses body verbatim, including `reasoning` items with `encrypted_content` so multi-turn tool loops replay correctly. Stateless in v1: `store: true` and `previous_response_id` are rejected — echo prior `reasoning` / `function_call` items back in `input` each turn. Only OpenAI-backed gates are supported; a gate resolving to another provider returns 400. # OpenAI-format text-to-speech Source: https://docs.verlon.ai/api-reference/openai-format/openai-format-text-to-speech /api-reference/openapi.json post /v1/audio/speech **Use when:** caller has an existing OpenAI SDK TTS flow and wants to route through Verlon with zero code changes via baseURL swap Drop-in replacement for `POST https://api.openai.com/v1/audio/speech`. Accepts the upstream OpenAI TTS request body verbatim and returns the raw audio bytes the upstream API returns. Route through a Verlon gate by passing `model` as a Verlon gate id. Existing OpenAI SDK clients work unmodified via baseURL swap. Errors are returned in the OpenAI nested shape. # OpenAI-format video content download Source: https://docs.verlon.ai/api-reference/openai-format/openai-format-video-content-download /api-reference/openapi.json get /v1/videos/:videoId/content **Use when:** caller uses openai.videos.downloadContent to fetch the finished video bytes Drop-in for `GET https://api.openai.com/v1/videos/{video_id}/content`. Streams the video (`video/mp4`) by proxying the provider-hosted result with Verlon-held credentials. Provider URLs expire upstream — an expired result returns an `invalid_request_error` telling the caller to regenerate. # OpenAI-format video generation job Source: https://docs.verlon.ai/api-reference/openai-format/openai-format-video-generation-job /api-reference/openapi.json post /v1/videos **Use when:** caller uses openai.videos.create (async video generation: create a job, poll it, download content) and wants to route through Verlon via a baseURL swap Drop-in for `POST https://api.openai.com/v1/videos`. Creates an async generation job and returns immediately with `status: "queued"`; poll `GET /v1/videos/{id}` until `completed`, then fetch bytes from `GET /v1/videos/{id}/content`. Address your gate the usual three ways (gate UUID in `model`, `gateId` body field, or `X-Verlon-Gate-Id` header); the gate may route to any video model — Veo, Grok Imagine, or Sora. The completed object also carries a non-standard `url` extension with the provider-hosted link, which expires upstream. # OpenAI-format video job deletion Source: https://docs.verlon.ai/api-reference/openai-format/openai-format-video-job-deletion /api-reference/openapi.json delete /v1/videos/:videoId **Use when:** caller uses openai.videos.delete to remove a video job Drop-in for `DELETE https://api.openai.com/v1/videos/{video_id}`. Soft-deletes the job (billing history is retained internally). # OpenAI-format video job list Source: https://docs.verlon.ai/api-reference/openai-format/openai-format-video-job-list /api-reference/openapi.json get /v1/videos **Use when:** caller uses openai.videos.list to enumerate their video generation jobs Drop-in for `GET https://api.openai.com/v1/videos`. Newest first; `limit` up to 100. # OpenAI-format video job poll Source: https://docs.verlon.ai/api-reference/openai-format/openai-format-video-job-poll /api-reference/openapi.json get /v1/videos/:videoId **Use when:** caller uses openai.videos.retrieve to poll an async video generation job until it completes Drop-in for `GET https://api.openai.com/v1/videos/{video_id}`. Returns the job object; `status` moves `queued` → `in_progress` → `completed` or `failed`. # API Reference Overview Source: https://docs.verlon.ai/api-reference/overview The Verlon API is OpenAI-format — call it with the SDK you already have Verlon's API speaks the formats the ecosystem already standardized on. There is no proprietary request shape to learn: point the official OpenAI SDK (or Anthropic, Mistral, Gemini) at Verlon and you have the full API. ## Base URL ``` https://api.verlon.ai ``` All routes authenticate with your Verlon API key as a Bearer token (`Authorization: Bearer sk-vrln-...`), except where noted. ## Inference | Route | Format | Official SDK | | - | - | - | | `POST /v1/chat/completions` | OpenAI Chat Completions | `openai` (`baseURL: .../v1`) | | `POST /v1/responses` | OpenAI Responses | `openai` | | `POST /v1/embeddings` | OpenAI Embeddings | `openai` | | `POST /v1/images/generations` | OpenAI Images | `openai` | | `POST /v1/audio/speech` | OpenAI Speech | `openai` | | `POST /v1/messages` | Anthropic Messages | `@anthropic-ai/sdk` (`baseURL: ...`, no `/v1`) | | `POST /mistral/v1/chat/completions` | Mistral Chat | `@mistralai/mistralai` (`serverURL: .../mistral`) | | `POST /mistral/v1/fim/completions` | Mistral FIM (Codestral) | `@mistralai/mistralai` | | `POST /v1beta/models/{model}:generateContent` | Gemini | `@google/genai` | | `POST /v1/videos` (+ retrieve/content/list/delete) | OpenAI Videos (async jobs) | `openai` | | `POST /mistral/v1/ocr` | Mistral OCR | `@mistralai/mistralai` (`serverURL: .../mistral`) | Address your gate on any inference route in one of three ways: a `gateId` body field, an `X-Verlon-Gate-Id` header, or the gate UUID in the `model` string — optionally `/` to attribute a named task. With a real model name and no gate reference, Verlon auto-creates a passthrough gate on first use. Error responses mirror the format you're calling with: OpenAI-shape routes return OpenAI's error envelope, `/v1/messages` returns Anthropic's, so your SDK's error handling works unchanged. ## Platform Gates, tasks, models, the public registry, usage, traces, and OAuth device flow are documented in the sidebar — generated from the live OpenAPI spec. Dashboard-only and internal routes are not part of the public API surface. ## Guides > **[OpenAI SDK Integration](/integrations/openai-sdk)** > > Setup, streaming, tool calling, migration > **[Anthropic SDK Integration](/integrations/anthropic-sdk)** > > The same two-line swap for Claude-first apps > **[Provider Compatibility](/provider-compatibility/models)** > > Every supported model and route > **[Agent Tracing](/platform/agent-gates)** > > Traces, tasks, and sessions via @verlon-ai/sdk # Delete an owned-model registration Source: https://docs.verlon.ai/api-reference/owned-models/delete-an-owned-model-registration /api-reference/openapi.json delete /v1/owned-models/:slug **Use when:** caller removes an owned model they no longer serve Deletes the registration and its credential. Refused while any of the caller's gates still references `custom:` — repoint or delete those gates first, so routing never dangles. # List the caller's owned-model registrations Source: https://docs.verlon.ai/api-reference/owned-models/list-the-callers-owned-model-registrations /api-reference/openapi.json get /v1/owned-models **Use when:** caller wants the models they registered as their own (fine-tunes, custom deployments) — the "Your own models" section One row per registration: slug (route as `custom:`), host kind, upstream model id, declared pricing, probe status, and whether a credential is stored. Credentials are never returned. # Register a model the caller owns Source: https://docs.verlon.ai/api-reference/owned-models/register-a-model-the-caller-owns /api-reference/openapi.json post /v1/owned-models **Use when:** caller registers a fine-tune or custom deployment as an owned model, with its own credential and declared pricing Creates an owned-model registration. Admits host kinds `fireworks` (hosted fine-tunes; credential required) and `custom-url` (any OpenAI-compatible server you operate; base URL required, credential optional, private/loopback/localhost URLs are refused outright). Other kinds are rejected until their adapters land, never accepted-and-mis-billed. The credential is stored encrypted, scoped to this registration (separate from any BYOK channel key). A reachability probe runs on save but never blocks: unreachable registrations save with an amber status (self-hosted boxes are allowed to be asleep). Pricing is owner-declared; absent rates record cost as $0 with "owned" semantics. # Update an owned-model registration Source: https://docs.verlon.ai/api-reference/owned-models/update-an-owned-model-registration /api-reference/openapi.json patch /v1/owned-models/:slug **Use when:** caller rotates the credential, corrects the upstream model id, or adjusts declared pricing on an owned model Partial update. A new credential replaces the stored one and triggers a fresh reachability probe (warn-and-save). The slug itself is immutable — it is the routing ref; register a new model to rename. # Get the public model + provider registry Source: https://docs.verlon.ai/api-reference/registry/get-the-public-model-+-provider-registry /api-reference/openapi.json get /v1/registry **Use when:** caller wants Verlon's canonical list of supported models, providers, capabilities, pricing, and deprecation status (no auth required; cacheable) Public endpoint (no auth required). Returns Verlon's canonical list of supported models, providers, capabilities, pricing, and deprecation status — the same registry the SDK validates `model` values against and the dashboard renders in its model picker. Cached for 1 hour with an ETag tied to the registry version; agents and clients should respect the Cache-Control headers. # Responses API Source: https://docs.verlon.ai/api-reference/responses OpenAI-compatible Responses endpoint (client.responses.create) Drop-in for `POST https://api.openai.com/v1/responses`. Point your OpenAI SDK's `baseURL` at `https://api.verlon.ai/v1` and pass your Verlon gate id as `model`. The request and response bodies match OpenAI's Responses API verbatim — the Agents SDK and multi-turn tool loops with encrypted reasoning replay work unmodified. ## Request `POST /v1/responses` | Field | Type | Notes | | - | - | - | | `model` | string | Your Verlon **gate id** (routes to the gate's configured OpenAI model). | | `input` | string \| item\[] | A prompt string, or an array of Responses items: `message`, `function_call`, `function_call_output`, `reasoning`. | | `instructions` | string | System prompt. | | `tools` | object\[] | Flattened function tools (`{ type: 'function', name, parameters }`). | | `tool_choice` | string \| object | `auto` / `required` / `none` / `{ type: 'function', name }`. | | `max_output_tokens` | number | Output token cap. | | `reasoning` | object | `{ effort: 'low' \| 'medium' \| 'high' }` for reasoning models. | | `stream` | boolean | Named-event SSE when `true`. | > [!WARNING] > > `store: true` and `previous_response_id` are **not supported** (Verlon is > stateless — zero data retention by default). Echo prior `reasoning` and > `function_call` items back in `input` each turn instead. Both return `400`. ## Reasoning replay For reasoning models, Verlon automatically requests `include: ['reasoning.encrypted_content']` and passes the encrypted reasoning through untouched. To continue a tool loop, send the prior turn's `reasoning` and `function_call` output items back at the front of `input`, followed by your `function_call_output`: ```json theme={null} { "model": "", "input": [ { "role": "user", "content": "What's the weather in Tokyo?" }, { "type": "reasoning", "id": "rs_…", "encrypted_content": "…" }, { "type": "function_call", "call_id": "call_…", "name": "get_weather", "arguments": "{\"city\":\"Tokyo\"}" }, { "type": "function_call_output", "call_id": "call_…", "output": "{\"temp_c\":22}" } ], "tools": [ /* … */ ] } ``` ## Restrictions * **OpenAI-backed gates only.** A gate resolving to Anthropic, Google, or Mistral returns `400` — use [`/v1/chat/completions`](/api-reference/chat-completions) for cross-provider routing. * Errors use OpenAI's nested shape (`{ error: { message, type, code, param? } }`). See [OpenAI compatibility → Responses API](/provider-compatibility/openai#responses-api) for SDK examples. # Get a coding session as a turn timeline (beta) Source: https://docs.verlon.ai/api-reference/traces/get-a-coding-session-as-a-turn-timeline-beta /api-reference/openapi.json get /v3/traces/coding-sessions/turns **Use when:** the dashboard session detail renders one gateway session as turns (idle-bounded traces) with their calls Paginated turns for one session, newest first, each with its generation spans (name, task, model, duration, status). # Get coding-gate overview stats (beta) Source: https://docs.verlon.ai/api-reference/traces/get-coding-gate-overview-stats-beta /api-reference/openapi.json get /v3/traces/coding-sessions/stats **Use when:** the dashboard coding-agents Overview renders spend, session counts, and model/task mix for one connector gate over a chosen window Windowed aggregates for the coding-agents Overview: spend by day, session count, median session cost, cache detail, and per-model / per-task mix. The window is either the last N days (default 7) or the last N sessions — pass windowUnit and windowSize. # Get one trace with its spans and derived agent view Source: https://docs.verlon.ai/api-reference/traces/get-one-trace-with-its-spans-and-derived-agent-view /api-reference/openapi.json get /v3/traces/:traceId **Use when:** caller wants the full record of one agent run: every span plus the derived agent-view projection (ordering, nesting depth, evidenced tool executions, coverage accounting) Returns the trace row, its raw spans (decrypted payloads subject to storage policy), and the derived view: events ordered for display (spans, traffic-evidenced tool executions, labeled unobserved gaps) and a coverage summary (observed wall-clock %, verified span loss, sequence holes). Evidenced entries carry no fabricated measurements. # List agent traces Source: https://docs.verlon.ai/api-reference/traces/list-agent-traces /api-reference/openapi.json get /v3/traces **Use when:** caller wants to browse recorded agent runs (traces), optionally filtered to one agent gate, a status, or a session label Paginated list of traces — one row per bounded agent run. A trace groups every span of the run (LLM generations, tool executions, nested sub-agents). Filter by `gateId`, `status` (`active`, `completed`, `incomplete`, `error`), or `sessionLabel` (the caller-supplied conversation id). # List coding sessions for a connector gate (beta) Source: https://docs.verlon.ai/api-reference/traces/list-coding-sessions-for-a-connector-gate-beta /api-reference/openapi.json get /v3/traces/coding-sessions **Use when:** the dashboard coding-agents lens lists gateway sessions for one connector gate, with display names, turn/call counts, and cost Session rollup enriched for the coding-agents lens: traces grouped by session label with turn and call counts, cost, activity timestamps, and lazy metadata (user rename, mirrored client name, detected client). # List sessions (conversation-label rollups) Source: https://docs.verlon.ai/api-reference/traces/list-sessions-conversation-label-rollups /api-reference/openapi.json get /v3/traces/sessions **Use when:** caller wants the multi-turn view: traces grouped by their session label (conversation id), with per-session trace counts and cost Groups traces by session label and returns one row per label with trace count, total cost, first-seen and last-active timestamps. Sessions are labels, not lifecycle objects — a row with a NULL label aggregates all unlabeled traces. # Rename a coding session (beta) Source: https://docs.verlon.ai/api-reference/traces/rename-a-coding-session-beta /api-reference/openapi.json post /v3/traces/coding-sessions/name **Use when:** the user renames a session in the coding-agents lens; the name pins over any client-derived title Upserts the session metadata display name. A user-set name is pinned — mirrored client titles never overwrite it. # Current unit usage against the plan ceiling Source: https://docs.verlon.ai/api-reference/usage/current-unit-usage-against-the-plan-ceiling /api-reference/openapi.json get /v3/usage/units **Use when:** caller wants the account-level unit meter — units consumed this billing period and over the rolling 30 days, against the plan allowance Returns the tier, its monthly unit allowance (null for Enterprise), units consumed in the current billing period and over the rolling 30 days, and whether free-tier tracking is currently locked (over-allowance traffic keeps routing and recording but is hidden until upgrade or period reset). Units are tokens processed through the gateway; provider cache reads, failed requests, and Verlon-internal verification traffic never count. Usage never blocks requests and never bills overages — it only informs tier fit. # List the caller's model working set Source: https://docs.verlon.ai/api-reference/user-models/list-the-callers-model-working-set /api-reference/openapi.json get /v1/user-models **Use when:** caller wants the models the user actually works with — models any of their gates route to (in use) plus models they pinned from the catalog (added) Returns one row per working-set model with membership flags: `inUse` (referenced by at least one of the caller's gates — primary or fallback — computed live, never stored), `added` (manually pinned from the catalog), `gateCount`, and `addedAt`. Enrich against `/v1/registry` for pricing/display data. # Per-model usage stats from the caller's own traffic Source: https://docs.verlon.ai/api-reference/user-models/per-model-usage-stats-from-the-callers-own-traffic /api-reference/openapi.json get /v1/user-models/usage **Use when:** caller wants observed per-model evidence — 7-day token volume, median latency, effective throughput — computed from their own request history (e.g. the model catalog usage columns) Aggregates the caller's successful requests over the last 7 days into one row per model: `tokens7d` (total tokens), `medianLatencyMs`, `tokensPerSec` (completion tokens over wall-clock latency — effective, not provider-quoted), and `requestCount`. `totals` carries the caller's whole 7-day prompt/completion token mix so clients can reprice it against any model's rates. Models with no traffic simply have no row. # Pin a catalog model to the working set Source: https://docs.verlon.ai/api-reference/user-models/pin-a-catalog-model-to-the-working-set /api-reference/openapi.json post /v1/user-models/added **Use when:** caller clicked "Add" on a catalog model so it shows in their working set before any gate uses it Pins a model from the catalog into the caller's working set. The model must exist in the live registry; pinning is idempotent. Models a gate already routes to are "in use" automatically and need no pin. # Unpin a model from the working set Source: https://docs.verlon.ai/api-reference/user-models/unpin-a-model-from-the-working-set /api-reference/openapi.json delete /v1/user-models/added **Use when:** caller removes a manually-added model from their working set (models in use by gates are not removable here — change the gates instead) Removes a manual pin. The model id travels as a query parameter because channel-qualified ids contain slashes (`accounts/fireworks/models/...`) and cannot be path segments. Only the "added" pin is removed — if the model is in use by gates it remains in the working set until those gates change. # Identify the authenticated principal Source: https://docs.verlon.ai/api-reference/user/identify-the-authenticated-principal /api-reference/openapi.json get /v1/whoami **Use when:** caller wants a minimal "who am I" snapshot — typically a CLI / agent confirming the API key it loaded resolves to the expected Verlon account and tier Returns the authenticated user’s id, email, tier, and the auth mode that resolved the request (`api_key` or `session`). When the auth mode is `api_key`, also returns `apiKeyId` so the caller can identify which of their keys was used. No PII beyond the email; intentionally a smaller surface than `/v1/user` (dashboard-only). Accepts both API key and session auth. # Generate a video via a Verlon gate Source: https://docs.verlon.ai/api-reference/video/generate-a-video-via-a-verlon-gate /api-reference/openapi.json post /v3/video **Use when:** caller wants to generate a video clip from a prompt or seed image routed through a Verlon gate (Sora, Runway, Veo, etc., resolved by the gate) Routes a video-generation request through the named Verlon gate. The gate decides which underlying model+provider serves the request based on its current configuration. Provider auth, parameter overrides, billing, and observability handled by Verlon. # Architecture Overview Source: https://docs.verlon.ai/contributing/architecture Understanding Verlon AI's architecture This documentation page is under development and will be available soon. In the meantime, if you have questions about this topic: * Check our [GitHub](https://docs.verlon.ai) for updates * Reach out to our team at [support@verlon.ai](mailto:support@verlon.ai) ## What's Available Now While this specific documentation is being written, you can explore: > **[Getting Started](/getting-started/quickstart)** > > Quick start guide to get up and running > **[SDK Reference](/sdk-reference/overview)** > > Complete SDK method documentation > **[OpenAI SDK Integration](/integrations/openai-sdk)** > > Use Verlon with OpenAI SDK > **[Gates & Routing](/platform/gates)** > > How Verlon routes requests # Contributing Guide Source: https://docs.verlon.ai/contributing/getting-started How to contribute to Verlon AI This documentation page is under development and will be available soon. In the meantime, if you have questions about this topic: * Check our [GitHub](https://docs.verlon.ai) for updates * Reach out to our team at [support@verlon.ai](mailto:support@verlon.ai) ## What's Available Now While this specific documentation is being written, you can explore: > **[Getting Started](/getting-started/quickstart)** > > Quick start guide to get up and running > **[SDK Reference](/sdk-reference/overview)** > > Complete SDK method documentation > **[OpenAI SDK Integration](/integrations/openai-sdk)** > > Use Verlon with OpenAI SDK > **[Gates & Routing](/platform/gates)** > > How Verlon routes requests # Admin Package Source: https://docs.verlon.ai/contributing/packages/admin Contributing to the Verlon AI admin SDK This documentation page is under development and will be available soon. In the meantime, if you have questions about this topic: * Check our [GitHub](https://docs.verlon.ai) for updates * Reach out to our team at [support@verlon.ai](mailto:support@verlon.ai) ## What's Available Now While this specific documentation is being written, you can explore: > **[Getting Started](/getting-started/quickstart)** > > Quick start guide to get up and running > **[SDK Reference](/sdk-reference/overview)** > > Complete SDK method documentation > **[OpenAI SDK Integration](/integrations/openai-sdk)** > > Use Verlon with OpenAI SDK > **[Gates & Routing](/platform/gates)** > > How Verlon routes requests # CLI Package Source: https://docs.verlon.ai/contributing/packages/cli Contributing to the Verlon AI CLI This documentation page is under development and will be available soon. In the meantime, if you have questions about this topic: * Check our [GitHub](https://docs.verlon.ai) for updates * Reach out to our team at [support@verlon.ai](mailto:support@verlon.ai) ## What's Available Now While this specific documentation is being written, you can explore: > **[Getting Started](/getting-started/quickstart)** > > Quick start guide to get up and running > **[SDK Reference](/sdk-reference/overview)** > > Complete SDK method documentation > **[OpenAI SDK Integration](/integrations/openai-sdk)** > > Use Verlon with OpenAI SDK > **[Gates & Routing](/platform/gates)** > > How Verlon routes requests # SDK Package Source: https://docs.verlon.ai/contributing/packages/sdk Contributing to the Verlon AI SDK package (@verlon-ai/sdk) This documentation page is under development and will be available soon. In the meantime, if you have questions about this topic: * Check our [GitHub](https://docs.verlon.ai) for updates * Reach out to our team at [support@verlon.ai](mailto:support@verlon.ai) ## What's Available Now While this specific documentation is being written, you can explore: > **[Getting Started](/getting-started/quickstart)** > > Quick start guide to get up and running > **[SDK Reference](/sdk-reference/overview)** > > Complete SDK method documentation > **[OpenAI SDK Integration](/integrations/openai-sdk)** > > Use Verlon with OpenAI SDK > **[Gates & Routing](/platform/gates)** > > How Verlon routes requests # Authentication Source: https://docs.verlon.ai/getting-started/authentication Learn how to authenticate with the Verlon AI API Verlon AI uses API keys to authenticate requests. Your API keys grant access to your gates, usage data, and configuration. ## Getting Your API Key 1. Sign in to your [Dashboard](https://verlon.ai/dashboard) 2. Navigate to **API Keys** in the sidebar 3. Click **Create New Key** 4. Give your key a descriptive name (e.g., "Production", "Development") 5. Copy the key immediately - you won't be able to see it again > [!WARNING] > > **Keep your API keys secure!** Never commit them to version control, share them publicly, or expose them in client-side code. ## Using Your API Key ### With the SDK Pass your API key when initializing the Verlon client: ```typescript theme={null} import { Verlon } from '@verlon-ai/sdk'; const verlon = new Verlon({ apiKey: 'your-api-key-here' }); ``` ### With Environment Variables Store your API key in environment variables for security: ```bash .env theme={null} VERLON_API_KEY=your_api_key_here ``` Then reference it in your code: ```typescript theme={null} import { Verlon } from '@verlon-ai/sdk'; const verlon = new Verlon({ apiKey: process.env.VERLON_API_KEY }); ``` ### With Direct API Calls Include your API key in the `Authorization` header: ```bash theme={null} curl https://api.verlon.ai/v1/chat/completions \ -H "Authorization: Bearer your-api-key-here" \ -H "Content-Type: application/json" \ -d '{ "gate_id": "your-gate-id", "messages": [ {"role": "user", "content": "Hello!"} ] }' ``` ## Key Management Best Practices > **Rotate keys regularly** > > Create new keys and delete old ones periodically to maintain security. Verlon AI allows multiple active keys per account. > **Use different keys for different environments** > > Create separate keys for development, staging, and production environments. This makes it easier to rotate keys and track usage. > **Delete compromised keys immediately** > > If you suspect a key has been exposed, delete it from your Dashboard immediately and create a new one. > **Monitor key usage** > > Check your Dashboard regularly to review which keys are being used and their associated costs. ## Key Permissions All Verlon AI API keys have full access to your account: * Make inference requests through your gates * Read and modify gates and configuration * Manage API keys * Access usage logs and analytics > [!NOTE] > > Since all keys have full permissions, treat them with the same level of security you would treat a password. ## Troubleshooting ### Invalid API Key Error If you receive an authentication error: 1. Verify your API key is correctly copied (no extra spaces) 2. Check that the key hasn't been deleted in your Dashboard 3. Ensure you're using the correct format: `Bearer your-api-key` ### Rate Limits API keys are subject to rate limits that exist as abuse prevention — they scale with your plan tier but are not a pricing axis (see [Plans](/platform/plans) for what tiers actually change). If you're hitting rate limits: * Implement exponential backoff in your code * Contact support if your legitimate traffic needs more headroom ## Next Steps > **[Create Your First Gate](/dashboard/creating-gates)** > > Set up a gate to start making requests > **[Make Your First Request](/getting-started/quickstart)** > > Follow the quickstart guide to make your first API call # Migrate to Verlon Source: https://docs.verlon.ai/getting-started/migrate Drop-in migration from LiteLLM, OpenRouter, or a hand-rolled LLM gateway — usually one config change on your side. Verlon is designed to sit exactly where LiteLLM, OpenRouter, or your own LLM proxy sits today. The migration is a **config change**, not a rewrite — pointing your existing SDK client at Verlon's base URL and swapping the API key. ## From LiteLLM If you're already using the OpenAI SDK to talk to a LiteLLM proxy, you can drop that proxy and point straight at Verlon. The SDK stays exactly the same. ```typescript Before (LiteLLM proxy) theme={null} import OpenAI from 'openai'; const openai = new OpenAI({ baseURL: 'https://your-litellm-proxy.example.com', apiKey: process.env.LITELLM_KEY, }); ``` ```typescript After (Verlon) theme={null} import OpenAI from 'openai'; const openai = new OpenAI({ baseURL: 'https://api.verlon.ai/v1', apiKey: process.env.VERLON_API_KEY, }); ``` Every LiteLLM-style feature you were using — routing, fallbacks, retries, cost tracking, budgets — is either configured through Verlon's dashboard or served by default. Your call sites don't change. See the full [OpenAI guide](/provider-compatibility/openai) for the exact base URL and header conventions. ## From OpenRouter OpenRouter is drop-in for the OpenAI SDK too, so the swap is nearly identical: ```typescript Before (OpenRouter) theme={null} import OpenAI from 'openai'; const openai = new OpenAI({ baseURL: 'https://openrouter.ai/api/v1', apiKey: process.env.OPENROUTER_KEY, }); ``` ```typescript After (Verlon) theme={null} import OpenAI from 'openai'; const openai = new OpenAI({ baseURL: 'https://api.verlon.ai/v1', apiKey: process.env.VERLON_API_KEY, }); ``` Verlon's model IDs map to the same public model names OpenRouter surfaces (`gpt-4o`, `claude-sonnet-4-5`, `gemini-2.0-flash`, etc.) — see the [supported LLMs list](/provider-compatibility/models) for the full inventory. Routing rules, spend caps, and quality monitoring live in the Verlon dashboard. ## From a hand-rolled gateway If you built your own proxy that forwards to provider APIs, the win is bigger than a config swap — you're deleting code, not just re-pointing it. The pattern: 1. **Keep your existing SDK client code exactly as-is.** No call-site changes. 2. **Point it at `https://api.verlon.ai/v1`** and use a Verlon API key. 3. **Delete your routing / retry / fallback / cost-tracking code.** Verlon handles them. 4. **Configure any custom routing rules** in the dashboard once, not per-service. The [platform docs](/platform/gates) cover the model picker, routing rules, budgets, and experiments — every knob you rolled by hand. ## Anthropic SDK users Same story on the other side. If your app talks to Anthropic-hosted Claude via the Anthropic SDK, point it at Verlon and get routing, failover, and quality monitoring — with access to every other model in the [registry](/provider-compatibility/models), not just Claude. Details in the [Anthropic guide](/provider-compatibility/anthropic). ## Migrating off the Verlon SDK's inference methods > [!WARNING] > > `@verlon-ai/sdk` has deprecated all of its inference methods — `complete()`, `chat()`, `chatStream()`, `image()`, `embeddings()`, `tts()` (2.5), `video()`, `ocr()` (2.6), and the tracing-surface `task.chat()` / `task.chatStream()` (2.6.1). They keep working through every 2.x release and are **removed in 3.0.0, scheduled December 1, 2026** — after which the SDK performs no inference at all. Inference moved to the format you already know: call Verlon's drop-in routes with the official provider SDKs. The Verlon SDK remains for [agent tracing and the model registry](/sdk-reference/overview), and plugs those same official SDKs into tracing via `clientOptions()`. | Deprecated method | Replacement | Route | | - | - | - | | `chat()` / `chatStream()` / `complete()` | `openai` package, `chat.completions.create()` | `/v1/chat/completions` | | `image()` | `openai` package, `images.generate()` | `/v1/images/generations` | | `embeddings()` | `openai` package, `embeddings.create()` | `/v1/embeddings` | | `tts()` | `openai` package, `audio.speech.create()` | `/v1/audio/speech` | | `video()` | `openai` package, `videos.create()` / `retrieve()` / `downloadContent()` | `/v1/videos` (async job) | | `ocr()` | `@mistralai/mistralai`, `ocr.process()` | `/mistral/v1/ocr` | | `task.chat()` / `task.chatStream()` | official SDK via `task.clientOptions('openai' \| 'anthropic')` — same attribution, correlation, and dashboard model pins | any inference route | The gateway also serves `/v1/responses` (OpenAI Responses API) and `/v1/messages` (`@anthropic-ai/sdk`). Every drop-in route authenticates with your Verlon API key as the Bearer token. Address your gate any of three ways: a `gateId` body field, an `X-Verlon-Gate-Id` header, or the gate UUID in the `model` string (optionally `/` to also name a task). Inside a trace scope, `task.clientOptions('openai')` handles all of it for you — see the [SDK overview](/sdk-reference/overview). Video is an async surface on the gateway, exactly like OpenAI's: `videos.create()` returns a job, poll `videos.retrieve()` until `completed`, then `videos.downloadContent()` streams the bytes. OCR keeps Mistral's own shape — a `document` in, markdown pages out. Before-and-after for the common case: ```typescript Before (Verlon SDK 2.x) theme={null} import { Verlon } from '@verlon-ai/sdk'; const verlon = new Verlon({ apiKey: process.env.VERLON_API_KEY }); const response = await verlon.chat({ gateId: process.env.GATE_ID, data: { messages: [{ role: 'user', content: 'Hello!' }] }, }); ``` ```typescript After (openai package) theme={null} import OpenAI from 'openai'; const openai = new OpenAI({ apiKey: process.env.VERLON_API_KEY, baseURL: 'https://api.verlon.ai/v1', }); const response = await openai.chat.completions.create({ model: process.env.GATE_ID, // gate UUID in the model slot messages: [{ role: 'user', content: 'Hello!' }], }); ``` ## Need a hand? Migration walkthroughs are on the roadmap. In the meantime, email [micah@verlon.ai](mailto:micah@verlon.ai) — most migrations we've seen take under an hour on the customer side. # Welcome to Verlon AI Source: https://docs.verlon.ai/getting-started/overview The AI Gateway for production applications - route, fallback, and track costs across multiple AI providers ## What is Verlon AI? Verlon AI is an AI gateway that sits between your application and AI providers (OpenAI, Anthropic, Google, etc.). It provides intelligent routing, automatic fallbacks, cost tracking, and a unified API across all providers. **Key benefits:** * **Smart Routing** - Automatically route requests to the best model for each task * **Automatic Fallbacks** - Never fail when a model is down * **Cost Tracking** - Real-time cost analytics per request, user, and endpoint * **Unified API** - One interface for GPT, Claude, Gemini, Mistral, and more * **Drop-in Replacement** - Works with OpenAI SDK and Anthropic SDK *** ## Quick Start Get up and running in under 5 minutes. > **[Quickstart Guide](/getting-started/quickstart)** > > Create your first gate and make your first request in 5 minutes. *** ## Platform Manage your AI infrastructure from the Verlon AI dashboard. > **[Gates](/platform/gates)** > > Route AI requests through configurable gates with intelligent model selection. > **[Agent Gates](/platform/agent-gates)** > > Trace every run of your agent and control each call site's model remotely. > **[BYOK](/platform/byok)** > > Bring your own keys from OpenAI, Anthropic, Google, and more. > **[API Keys](/platform/api-keys)** > > Generate and manage Verlon API keys for your applications. *** ## Integrate with Verlon > **[Verlon SDK](/sdk-reference/overview)** > > Full-featured SDK with chat, images, audio, and video support. > **[MCP Server](/integrations/mcp)** > > Query your gates, logs, and recommendations from Claude and other agents. *** ## Provider Compatibility Already using OpenAI or Anthropic SDKs? Verlon works as a drop-in replacement. > **[OpenAI SDK](/provider-compatibility/openai)** > > Drop-in replacement for existing OpenAI code - change 2 lines. > **[Anthropic SDK](/provider-compatibility/anthropic)** > > Use Anthropic SDK with Verlon for routing and fallbacks. *** ## Core Concepts Understand the building blocks of Verlon AI. > **[Gates & Routing](/platform/gates)** > > Learn how Verlon intelligently routes requests across models. > **[Fallback Strategies](/concepts/fallback-strategies)** > > Automatic failover when a model or provider goes down. > **[Cost Tracking](/concepts/cost-tracking)** > > Monitor and optimize your AI spending in real-time. *** ## API Reference Complete API documentation for all Verlon AI endpoints. > **[Chat Completions](/api-reference/chat-completions)** > > OpenAI-compatible chat completions endpoint. > **[Anthropic Messages](/api-reference/anthropic-messages)** > > Anthropic-compatible messages endpoint. *** ## Need Help? > **[Join our community](https://discord.gg/layerai)** > > Get help, share feedback, and connect with other Verlon AI users. # Quickstart Source: https://docs.verlon.ai/getting-started/quickstart Get started with Verlon AI in under 5 minutes ## Before you begin 1. Sign up for a free account at [verlon.ai/signup](https://verlon.ai/signup) 2. Get your API key from the [Dashboard](https://verlon.ai/dashboard) 3. Create your first gate with your desired model configuration ## Install an SDK Verlon is a drop-in gateway: you call it with the official OpenAI SDK (or [Anthropic SDK](/integrations/anthropic-sdk)) in any language — no Verlon-specific client required. ```bash npm theme={null} npm install openai ``` ```bash pnpm theme={null} pnpm add openai ``` ```bash pip theme={null} pip install openai ``` ## Make your first request Point the client at Verlon and pass your gate's ID as the model — Verlon routes the request to whatever model the gate is configured for, with fallbacks, spending caps, and logging applied on the way through. ```typescript TypeScript theme={null} import OpenAI from 'openai'; const client = new OpenAI({ apiKey: process.env.VERLON_API_KEY, baseURL: 'https://api.verlon.ai/v1', }); const response = await client.chat.completions.create({ model: 'your-gate-id', // your gate — Verlon routes it messages: [ { role: 'user', content: 'Explain quantum computing in simple terms' } ], }); console.log(response.choices[0].message.content); ``` ```javascript JavaScript theme={null} const OpenAI = require('openai'); const client = new OpenAI({ apiKey: process.env.VERLON_API_KEY, baseURL: 'https://api.verlon.ai/v1', }); const response = await client.chat.completions.create({ model: 'your-gate-id', // your gate — Verlon routes it messages: [ { role: 'user', content: 'Explain quantum computing in simple terms' } ], }); console.log(response.choices[0].message.content); ``` ```python Python theme={null} import os from openai import OpenAI client = OpenAI( api_key=os.environ["VERLON_API_KEY"], base_url="https://api.verlon.ai/v1", ) response = client.chat.completions.create( model="your-gate-id", # your gate — Verlon routes it messages=[ {"role": "user", "content": "Explain quantum computing in simple terms"} ], ) print(response.choices[0].message.content) ``` ```bash cURL theme={null} curl https://api.verlon.ai/v1/chat/completions \ -H "Content-Type: application/json" \ -H "Authorization: Bearer $VERLON_API_KEY" \ -d '{ "model": "your-gate-id", "messages": [ {"role": "user", "content": "Explain quantum computing in simple terms"} ] }' ``` > [!TIP] > > Already calling OpenAI or Anthropic directly? You don't need the gate ID at > all to start — swap `baseURL` and `apiKey`, keep your model names, and Verlon > picks up the traffic. See [Migrate an existing app](/getting-started/migrate). ## What's Next? > **[OpenAI SDK Integration](/integrations/openai-sdk)** > > Streaming, tool calling, and the full migration guide > **[Gates & Routing](/platform/gates)** > > Understand how gates provide smart routing and fallbacks > **[Agent Tracing](/platform/agent-gates)** > > Trace agent runs with the Verlon SDK — tasks, tools, and sessions > **[Dashboard Guide](/dashboard/creating-gates)** > > Deep dive into creating and configuring gates # Anthropic SDK Integration Source: https://docs.verlon.ai/integrations/anthropic-sdk How to use Verlon AI with the Anthropic SDK Verlon AI provides an Anthropic-compatible API endpoint. You can use the official Anthropic SDK by pointing it to Verlon's base URL instead of Anthropic's. ## Quick Start ### Prerequisites * Verlon AI account ([sign up](https://verlon.ai/signup)) * Verlon API key from your [Dashboard](https://verlon.ai/dashboard) * A configured gate (gate UUID) ### Installation ```bash npm theme={null} npm install @anthropic-ai/sdk ``` ```bash pnpm theme={null} pnpm add @anthropic-ai/sdk ``` ```bash yarn theme={null} yarn add @anthropic-ai/sdk ``` ### Configuration Update your Anthropic client initialization - just 2 lines: ```typescript Before theme={null} import Anthropic from '@anthropic-ai/sdk'; const anthropic = new Anthropic({ apiKey: process.env.ANTHROPIC_API_KEY, }); ``` ```typescript After theme={null} import Anthropic from '@anthropic-ai/sdk'; const anthropic = new Anthropic({ baseURL: 'https://api.verlon.ai', // ✓ Add this apiKey: process.env.VERLON_API_KEY, // ✓ Change this }); ``` ### Making Requests Verlon supports three ways to specify your gate: **Using gateId (Recommended)** ```typescript theme={null} // @ts-ignore - gateId is a Verlon AI extension const response = await anthropic.messages.create({ gateId: process.env.GATE_ID, max_tokens: 1024, messages: [ { role: 'user', content: 'Hello!' } ], }); ``` > [!NOTE] > > Requires `@ts-ignore` comment for TypeScript, but provides the clearest intent. **Using model field** ```typescript theme={null} const response = await anthropic.messages.create({ model: process.env.GATE_ID, // Gate UUID as model max_tokens: 1024, messages: [ { role: 'user', content: 'Hello!' } ], }); ``` > [!TIP] > > No TypeScript errors, works seamlessly with existing code. **Using header** ```typescript theme={null} const anthropic = new Anthropic({ baseURL: 'https://api.verlon.ai', apiKey: process.env.VERLON_API_KEY, defaultHeaders: { 'X-Verlon-Gate-Id': process.env.GATE_ID, }, }); // Then make requests without specifying gateId const response = await anthropic.messages.create({ max_tokens: 1024, messages: [ { role: 'user', content: 'Hello!' } ], }); ``` > [!TIP] > > Cleanest for TypeScript - set once, use everywhere. ## Streaming Example Streaming works exactly like the standard Anthropic SDK: ```typescript theme={null} const stream = await anthropic.messages.create({ gateId: process.env.GATE_ID, max_tokens: 1024, messages: [ { role: 'user', content: 'Tell me a joke.' } ], stream: true, }); for await (const event of stream) { if (event.type === 'content_block_delta' && event.delta.type === 'text_delta') { process.stdout.write(event.delta.text); } } ``` ## What's Supported > **Streaming** > > Full support for `stream: true` with identical behavior to Anthropic > **Tool Use** > > Works across all providers - Verlon handles format conversion > **Vision** > > Image inputs in messages fully supported > **All Message Types** > > System prompts, user, and assistant messages all supported > **Standard Parameters** > > `temperature`, `max_tokens`, `top_p`, and other Anthropic parameters > **Usage & Cost Tracking** > > Standard `response.usage` with token counts, plus Verlon's `cost` field ## Language Examples ```typescript TypeScript theme={null} import Anthropic from '@anthropic-ai/sdk'; const anthropic = new Anthropic({ baseURL: 'https://api.verlon.ai', apiKey: process.env.VERLON_API_KEY, }); const response = await anthropic.messages.create({ gateId: process.env.GATE_ID, max_tokens: 1024, messages: [ { role: 'user', content: 'Hello!' } ], }); console.log(response.content[0].text); ``` ```python Python theme={null} from anthropic import Anthropic client = Anthropic( base_url="https://api.verlon.ai", api_key="sk-vrln-your_api_key", ) response = client.messages.create( model="your-gate-uuid", max_tokens=1024, messages=[ {"role": "user", "content": "Hello!"} ] ) print(response.content[0].text) ``` ```javascript JavaScript theme={null} const Anthropic = require('@anthropic-ai/sdk'); const anthropic = new Anthropic({ baseURL: 'https://api.verlon.ai', apiKey: process.env.VERLON_API_KEY, }); async function chat() { const response = await anthropic.messages.create({ gateId: process.env.GATE_ID, max_tokens: 1024, messages: [ { role: 'user', content: 'Hello!' } ], }); console.log(response.content[0].text); } chat(); ``` ```bash cURL theme={null} curl https://api.verlon.ai/v1/messages \ -H "Content-Type: application/json" \ -H "Authorization: Bearer sk-vrln-your_api_key" \ -H "anthropic-version: 2023-06-01" \ -d '{ "gateId": "your-gate-uuid", "max_tokens": 1024, "messages": [ {"role": "user", "content": "Hello!"} ] }' ``` ## Migration Guide Follow these steps to migrate an existing Anthropic application: ### 1. Update Client Initialization Find where you initialize the Anthropic client and update it: ```typescript theme={null} // Before const anthropic = new Anthropic({ apiKey: process.env.ANTHROPIC_API_KEY }); // After const anthropic = new Anthropic({ baseURL: 'https://api.verlon.ai', apiKey: process.env.VERLON_API_KEY, }); ``` ### 2. Add Gate ID Add `gateId` to your message calls: ```typescript theme={null} // Before const response = await anthropic.messages.create({ model: 'claude-3-5-sonnet-20241022', max_tokens: 1024, messages: [...], }); // After const response = await anthropic.messages.create({ gateId: process.env.GATE_ID, // ✓ Add this max_tokens: 1024, messages: [...], // Everything else stays the same }); ``` ### 3. Update Environment Variables ```bash .env theme={null} VERLON_API_KEY=sk-vrln-your_key GATE_ID=your-gate-uuid ``` ### 4. Test & Deploy 1. Test in development 2. Verify requests appear in Verlon dashboard 3. Check cost tracking is working 4. Deploy to production > [!NOTE] > > **Easy Rollback:** If anything breaks, revert the 2 changes (`baseURL` + `apiKey`) - takes 30 seconds. ## Advanced Usage ### Multiple Gates Use different gates based on the task: ```typescript theme={null} const COMPLEX_TASK_GATE = 'uuid-for-claude-opus'; const SIMPLE_TASK_GATE = 'uuid-for-claude-haiku'; async function complexTask(prompt: string) { return await anthropic.messages.create({ gateId: COMPLEX_TASK_GATE, max_tokens: 4096, messages: [{ role: 'user', content: prompt }], }); } async function simpleTask(prompt: string) { return await anthropic.messages.create({ gateId: SIMPLE_TASK_GATE, max_tokens: 1024, messages: [{ role: 'user', content: prompt }], }); } ``` ### Environment-Specific Configuration ```typescript theme={null} const baseURL = process.env.NODE_ENV === 'production' ? 'https://api.verlon.ai' : 'http://localhost:3001'; // Local Verlon API for dev const anthropic = new Anthropic({ baseURL, apiKey: process.env.VERLON_API_KEY }); ``` ### Error Handling ```typescript theme={null} try { const response = await anthropic.messages.create({ gateId: process.env.GATE_ID, max_tokens: 1024, messages: [{ role: 'user', content: 'Hello!' }], }); console.log(response.content[0].text); } catch (error) { if (error.status === 404) { console.error('Gate not found - check your GATE_ID'); } else if (error.status === 429) { console.error('Rate limit exceeded'); } else { console.error('Request failed:', error.message); } } ``` ## How It Works 1. **You send** → Anthropic SDK request to `https://api.verlon.ai/v1/messages` 2. **Verlon receives** → Validates gate, applies routing rules 3. **Verlon routes** → Sends to the model configured in your gate (Claude, GPT, Gemini, etc.) 4. **Provider responds** → Returns response in provider's format 5. **Verlon normalizes** → Converts back to Anthropic format 6. **You receive** → Standard Anthropic response with added cost/metadata Your code doesn't know the difference. ## FAQ > **Do I need to change my code?** > > No. Just change `baseURL` and `apiKey` in the Anthropic client initialization. > **Can I still use Anthropic directly?** > > Yes. Just revert `baseURL` and `apiKey` to Anthropic values. > **Does streaming work?** > > Yes. `stream: true` works exactly like Anthropic. > **Can I use GPT or Gemini?** > > Yes. Configure your gate to use any model — your code stays the same. > **What about tool use?** > > Fully supported. Verlon handles the conversion across all providers. > **Do I need the Verlon SDK?** > > No. This approach uses only the Anthropic SDK. ## Comparison: Anthropic SDK vs Verlon SDK | Feature | Anthropic SDK + Verlon | Verlon SDK | | - | - | - | | **Migration Effort** | 2 lines of code | Full refactor | | **API Format** | Anthropic format | Verlon native format | | **Use Case** | Drop-in replacement | New projects | | **Multi-modal** | Chat only | Chat, image, video, audio | | **Streaming** | ✓ | ✓ | | **Tool Use** | ✓ | ✓ | | **Cost Tracking** | ✓ | ✓ | | **Admin Operations** | ✗ | ✓ (via Admin SDK) | **Recommendation:** * **Existing Anthropic apps** → Use Anthropic SDK + Verlon (this guide) * **New projects** → Use [Verlon SDK](/sdk-reference/overview) for full feature set ## Next Steps > **[Create a Gate](/dashboard/creating-gates)** > > Set up routing rules and configure models > **[View Request Logs](/dashboard/monitoring)** > > Monitor your API usage and costs > **[Configure Fallbacks](/concepts/fallback-strategies)** > > Learn about fallback strategies > **[Verlon SDK](/sdk-reference/overview)** > > Explore the full-featured Verlon SDK # Claude Code Integration Source: https://docs.verlon.ai/integrations/claude-code Run Claude Code on any model through a Verlon coding gate > [!NOTE] > > The Coding Agents experience is in **beta**. Claude Code speaks the Anthropic Messages API, and Verlon provides an Anthropic-compatible endpoint — so Claude Code can run through a Verlon coding gate, backed by any model in the registry (GPT, Gemini, Mistral, or Claude). You get dashboard-controlled model switching, per-session observability with turn-by-turn timelines, and cost tracking for every coding session, with no changes to Claude Code itself. ## Step zero — two commands, once per machine Everything on this page needs Node.js and the Verlon CLI. That's the whole dependency story: **the CLI is standalone and does everything; the plugin (later on this page) is an optional wrapper so you can stay inside Claude Code, and it bundles the Verlon MCP server. There is no install order — pick one path and follow it.** **1. Make sure you have Node** — check with `node --version`. If it's missing: **macOS / Linux** ```bash theme={null} brew install node ``` (or your distro's package manager) **Windows** ```powershell theme={null} winget install OpenJS.NodeJS.LTS ``` Then open a new terminal and confirm with `node --version`. **2. Install the Verlon CLI** (`-g` recommended — it gives you the `verlon` command used throughout these docs): ```bash theme={null} npm install -g @verlon-ai/cli ``` (One-off alternative without installing: prefix any command in these docs with `npx -y @verlon-ai/cli@latest` instead of `verlon`.) **New to Verlon?** Create an account at [verlon.ai/signup](https://verlon.ai/signup) and verify your email (verification activates your trial credit). You do **not** need an Anthropic API key — in routed mode your traffic runs on the gate's model through Verlon. Claude Code itself installs separately ([Anthropic's setup guide](https://code.claude.com/docs/en/setup)) — and if you only use the VS Code extension, that's fine: the path below covers you. ## Check your setup ```bash theme={null} verlon preflight ``` This checks everything — Node, Claude Code (terminal or VS Code extension), your working directory, your account — and tells you exactly what's missing and how to fix it. `connect` runs the same checks automatically and stops with a remedy instead of failing halfway. If you're helping someone set up: have them run this and read you the output. Then pick your mode: * **Routed (default)** — Claude Code's traffic flows through Verlon to whatever model the gate selects. Model switching, payload observability, cost tracking. * **Observability-only** — Claude Code keeps talking to Anthropic directly; only usage telemetry (never your code) streams to your dashboard. The minimal-trust way to try Verlon. See [Observability-only mode](#observability-only-mode). ## Setup ### Path 1 — terminal (recommended) In a **terminal** (not inside Claude Code), from the project directory you want to connect: ```bash theme={null} verlon connect claude-code ``` It checks your setup, prints the account it is acting as, authenticates (browser device flow, or reuses `VERLON_API_KEY`), resolves your coding gate, mints a dedicated API key **bound to that gate**, writes the settings into `.claude/settings.local.json` (git-ignored, so the key stays out of your repo), and enables usage telemetry. **Which gate?** Nothing is ever picked silently. Your first connect creates a gate. After that, the terminal shows a menu of your claude-code gates — the project's current gate (on a re-connect) or your default gate pre-highlighted, "create a new gate" always an option. Keeping what you have is one Enter; switching is two arrow presses. Then **restart Claude Code**. Settings are read at process start — starting a new chat is *not* a restart. In a terminal that means exit and re-run `claude`; in VS Code it means reloading the window (`Developer: Reload Window`). Your next session appears on the dashboard within seconds. Already have a gate you want to use? Pass it explicitly: ```bash theme={null} verlon connect claude-code --gate ``` Connect the same gate everywhere without picking each time? Mark it as your **default** — the menu pre-highlights it, scripts can rely on it, and the Claude Code plugin resolves to it without asking: ```bash theme={null} verlon gate set-default verlon connect claude-code --default # non-interactive: uses the default, fails clearly if none is set ``` (Also settable from the dashboard: the gate's Settings → "Default for new connections". `verlon gate unset-default ` clears it.) Verify any time with `verlon doctor --gate ` (traffic arriving, sessions grouping, costs accruing — plus which account you're acting as and which account this project's key belongs to). Switch models mid-session with `verlon switch ` (takes effect next turn, no restart — `verlon models` lists the ids). Undo everything with `verlon disconnect`. ### Path 2 — inside Claude Code (plugin; works for VS Code panel users) Prefer never leaving Claude Code — or you only use the VS Code side panel? Open Claude Code **in the project you want to connect** and install the [Verlon plugin](https://github.com/verlon-ai/claude-plugin), either way: **Slash commands** Run these one at a time — the first registers Verlon's plugin source on your machine (nothing is published anywhere), the second installs from it: ``` /plugin marketplace add https://github.com/verlon-ai/claude-plugin ``` ``` /plugin install verlon ``` **VS Code panel (point-and-click)** Type `/plugins` in the prompt box to open the **Manage plugins** interface. In the **Marketplaces** tab, add `https://github.com/verlon-ai/claude-plugin`; then find **verlon** in the plugin list and click **Install**. No terminal needed. Then connect (the settings are written into the **current** project, so make sure you opened the right one): ``` /verlon:connect ``` …and restart Claude Code as above (VS Code: reload the window). You get `/verlon:connect`, `/verlon:switch`, `/verlon:models`, `/verlon:doctor`, and `/verlon:disconnect` — thin wrappers over the same CLI — plus the bundled Verlon MCP server, so "switch my model to gemini-3-pro" works conversationally too. ### Observability-only mode Don't want to route your traffic through anyone — but still want the usage picture? Connect in observability-only mode: ```bash theme={null} verlon connect claude-code --observe-only ``` Claude Code keeps talking to Anthropic **directly** — zero added latency, no routing, nothing between you and the API. Its standard usage telemetry (tokens, cost, models, tool activity, session ids — **never your code or prompts**) streams to your Verlon dashboard, where sessions appear with the same turn-by-turn timelines. Your traffic never touches us, and the telemetry's own cost numbers are authoritative since nothing is being rerouted. What you give up versus routed mode: model switching (traffic goes straight to Anthropic, so the gate's model doesn't apply) and per-call payload observability. Switching a project between modes is just re-running `connect` with or without `--observe-only` — same gate, same dashboard. ### Disconnecting From a terminal in the project (or `/verlon:disconnect` inside Claude Code): ```bash theme={null} verlon disconnect ``` It removes exactly what `connect` wrote from `.claude/settings.local.json` and touches nothing else. Then **restart Claude Code** — this is the step people miss: sessions already running keep routing through Verlon (and keep sending telemetry) until their *process* restarts, and a new chat is not a restart; in VS Code, reload the window. Your gate and its history stay on the dashboard, and the API key stays valid until you revoke it under **Settings → API Keys** — disconnecting never silently revokes anything. ### Manual setup The dashboard shows a ready-to-copy settings block under **Coding agents → your gate → Setup**. If you'd rather wire it yourself: create a coding gate (Dashboard → Coding agents → New Claude Code gate), pick a coding model and a background model that is **cheap and large-context** (Claude Code's background summaries routinely exceed small context windows), grab an API key, and set the environment: ```bash theme={null} export ANTHROPIC_BASE_URL=https://api.verlon.ai export ANTHROPIC_AUTH_TOKEN= export ANTHROPIC_MODEL= export ANTHROPIC_DEFAULT_SONNET_MODEL= export ANTHROPIC_DEFAULT_OPUS_MODEL= export ANTHROPIC_DEFAULT_FABLE_MODEL= export ANTHROPIC_DEFAULT_HAIKU_MODEL=/background export ANTHROPIC_SMALL_FAST_MODEL=/background export CLAUDE_CODE_SUBAGENT_MODEL=/subagents export ENABLE_TOOL_SEARCH=true export CLAUDE_CODE_ENABLE_FINE_GRAINED_TOOL_STREAMING=1 export CLAUDE_CODE_ATTRIBUTION_HEADER=0 claude ``` The variables live only in that shell — other terminals keep talking to Anthropic directly. For the VS Code extension (which doesn't inherit shell exports), put the same values in the project's `.claude/settings.local.json` under `"env"` — this is exactly what `connect` writes. ## How it works Claude Code reads environment variables that redirect its API traffic. You point its base URL at Verlon, authenticate with a Verlon API key, and put a **gate id** where Claude Code expects a model name. One gate covers the whole session: * The gate's **coding model** answers your sessions (Claude Code's main-model traffic). * Claude Code's automatic housekeeping calls (session titles, summaries — its "haiku tier") carry a `/background` suffix on the gate id and run on the gate's **background model**. * Subagent traffic carries a `/subagents` suffix so it's attributed separately; it runs on the coding model unless you route it elsewhere in the dashboard. Claude Code decides which role each call belongs to on its own — you only decide what each role means, and you can change that from the dashboard at any time, mid-session. **Direct model requests.** If Claude Code ever sends a literal model id instead of the gate id (a tier your env mapping missed, a subagent pinning a full model id), the request still resolves through your gate: keys minted by `connect` are bound to their gate, and the gate's **Direct model requests** policy decides what happens — **Remap** (default: route to the gate's model for that role, so the dashboard stays authoritative), **Honor** (serve the literal id, attributed through the gate), or **Block**. The requested id is always recorded, so the timeline shows "asked for X, served Y". | Variable | Purpose | | - | - | | `ANTHROPIC_BASE_URL` | Sends Claude Code's requests to Verlon instead of Anthropic | | `ANTHROPIC_AUTH_TOKEN` | Your Verlon API key (sent as a bearer token) | | `ANTHROPIC_MODEL` | The gate id — main-model traffic runs on the gate's coding model | | `ANTHROPIC_DEFAULT_SONNET_MODEL` / `ANTHROPIC_DEFAULT_OPUS_MODEL` / `ANTHROPIC_DEFAULT_FABLE_MODEL` | Keep `/model` tier switches resolving to the gate (the Fable var requires a recent Claude Code; older clients are caught by the server-side remap) | | `ANTHROPIC_DEFAULT_HAIKU_MODEL` / `ANTHROPIC_SMALL_FAST_MODEL` | Background/housekeeping traffic → the gate's background model (two spellings of one knob across Claude Code versions — set both) | | `CLAUDE_CODE_SUBAGENT_MODEL` | Attributes subagent traffic separately; runs on the coding model until you route it otherwise | ## Switching accounts The CLI's login is machine-global (like `gh`): the most recent `verlon login` wins in every terminal, and inside a connected session the project's key (injected via settings) outranks the login entirely. To see who you're acting as: `verlon whoami`. To keep multiple accounts: `verlon login --profile ` and `verlon auth switch `. To reconnect a project under a different account from inside a connected session: `verlon connect claude-code --fresh` — it ignores the environment key and acts as your login. `verlon doctor` reports all of it: your acting identity, whether an environment key is shadowing your login, and which account the project's key belongs to. ## What you get * **Sessions, not just logs** — every Claude Code conversation appears under the gate's Sessions tab, grouped into turns you can open: each model call with its role, model, duration, and cost. * **Tool visibility** — with telemetry on (the connect default), the tools Claude Code runs locally (Bash, file edits) appear in the turn timeline between the model calls, with timing and status. * **Live model switching** — `verlon switch `, `/verlon:switch`, or the gate's Settings page; the running session switches on its next request, no restart. * **Cost clarity** — spend per session and per turn, model mix, cache savings, and daily totals on the gate's Overview. * **Failover** — optionally add a fallback model; sessions ride out a provider outage automatically. Note that failover sends session content to the fallback provider — it's opt-in for that reason. ## Privacy & security * **Where your code goes:** routed sessions transit Verlon to reach the model provider. Verlon does not store prompt or response content by default; customers opt in to encrypted payload storage from the dashboard — and note the opt-in covers failed requests too. See your dashboard's privacy settings for exactly what is stored and for how long. * **Where your key lives:** `connect` writes it to `.claude/settings.local.json`, which Claude Code keeps out of git. The key is minted for this machine and bound to this gate — revoke it any time under **Settings → API Keys** without touching your other keys. * **What telemetry sends:** metadata only — token counts, cost, model, tool names and timings, session ids. Your prompts, code, and tool outputs are **not** included (Claude Code redacts them by default; we don't ask you to change that). Remove the `OTEL_*` lines from the settings file — or run `verlon connect claude-code --no-telemetry` — to turn it off; the connector works either way. * **Leaving:** `verlon disconnect` (or `/verlon:disconnect`) removes exactly what connect wrote and leaves your other Claude Code settings untouched — then **restart Claude Code**: sessions already running keep routing (and keep sending telemetry) until their process restarts. Your gate history stays on the dashboard; the key stays valid until you revoke it. ## Gotchas * **Restarts are per-process, not per-chat.** Connect and disconnect take effect when the Claude Code *process* restarts. A new conversation in the same window is not a restart; in VS Code, reload the window. * **Pick a large-context background model.** "Cheap" isn't enough — Claude Code's background summarization calls routinely exceed small context windows (an 8K-context model fails them outright). Any modern mini-class model works. * **Lots of MCP tools? Handled.** Sessions with many MCP servers can exceed OpenAI's legacy 128-tool limit — Verlon routes OpenAI-backed gates through the newer Responses API automatically, so large tool sets just work (500+ tools verified). * **Subagent definitions can pin literal model ids.** A subagent whose frontmatter says `model: haiku` resolves through the env mapping; a literal id like `model: claude-haiku-4-5-20251001` reaches the server directly and follows your gate's Direct-model-requests policy (remapped to the background role by default). * **Model self-reports are unreliable.** Don't ask the assistant which model it is — the gate's request log records what was asked for and what actually served every call. * **Experimental client features.** If you see 400s naming unfamiliar request fields, set `CLAUDE_CODE_DISABLE_EXPERIMENTAL_BETAS=1` to stop Claude Code sending pre-release capabilities. # MCP (Model Context Protocol) Source: https://docs.verlon.ai/integrations/mcp Inspect and manage your Verlon AI account from any MCP-compatible client — Claude Desktop, Claude Code, Cursor, Cline, Windsurf. ## Overview [MCP (Model Context Protocol)](https://modelcontextprotocol.io) is the open standard for how coding agents and LLM clients connect to external tools and data. The [`@verlon-ai/mcp`](https://www.npmjs.com/package/@verlon-ai/mcp) server exposes your Verlon gates, logs, recommendations, and experiments as MCP tools so any compatible host (Claude Desktop, Claude Code, Cursor, Cline, Windsurf, etc.) can inspect and reason about your AI infrastructure natively — without you copy-pasting curl commands into a chat. The server is open source (MIT) at [github.com/verlon-ai/mcp](https://github.com/verlon-ai/mcp), and every npm release carries a [provenance attestation](https://www.npmjs.com/package/@verlon-ai/mcp#provenance) tying the published package to that repo. The server is **read-only by default**. The current 0.3.x releases ship five read tools — `list_gates`, `get_gate`, `list_logs`, `get_recommendations`, `list_experiments`. Write tools (`create_gate`, `update_gate`, etc., behind an explicit `--enable-writes` opt-in) follow in a future release. ### Use Cases * **Agent introspection** — Ask Claude "what gates do I have?" and have it call `list_gates` rather than you opening the dashboard * **Gate auditing during refactors** — Use Claude Code to inventory your AI infrastructure while editing the surrounding application code * **Log inspection** — Tail recent requests inline with `list_logs` while debugging an integration ### Why the Verlon MCP server? * **Read-only by default** — A misaligned agent host can only inspect, never destroy. Write tools require the explicit `--enable-writes` flag * **Structured JSON tool outputs** — Every tool returns parseable JSON, not free-form text — so the calling model can reason about the result without re-prompting * **One install line** — `npx -y @verlon-ai/mcp` works in any MCP client; no compile step, no Docker * **Targeted error recovery** — Auth and API errors surface as `isError: true` content with explicit recovery hints, so the model can self-correct mid-conversation ## Quick Start ### Step 1: Get a Verlon API key Generate one at [verlon.ai/dashboard/settings/keys](https://verlon.ai/dashboard/settings/keys). It looks like `sk-vrln-...`. > [!WARNING] > > The API key gives the MCP server full read access to your account. Treat it > like any other secret — don't commit it to git, don't paste it into shared > chats. Per-client config files are the right place. ### Step 2: Wire the server into your MCP client **Claude Code / Claude Desktop** — add to `~/Library/Application Support/Claude/claude_desktop_config.json` (macOS) or the equivalent on your OS: ```json theme={null} { "mcpServers": { "verlon": { "command": "npx", "args": ["-y", "@verlon-ai/mcp"], "env": { "VERLON_API_KEY": "sk-vrln-..." } } } } ``` **Cursor** — add the same snippet under Settings → Features → MCP Servers. **Any other MCP-compatible client** — spawn `npx -y @verlon-ai/mcp` over stdio with `VERLON_API_KEY` in the subprocess environment. ### Step 3: Restart the client and try it Restart Claude Code (or your chosen host). Ask: > List the gates in my Verlon account. The host discovers the `list_gates` tool, invokes it, and shows the gates as a JSON response inline. > [!NOTE] > > The default is **read-only**. If you opt into write tools via > `--enable-writes`, do it deliberately and after you understand which > tools mutate account state. Per-tool destructive hints are surfaced to the > host so it can prompt before invoking risky operations. ## Tools All current tools are **read-only**. Write tools (`create_gate`, `update_gate`, `run_chat`, `start_experiment`) ship behind an explicit `--enable-writes` opt-in in a future release. | Tool | Inputs | What it returns | | - | - | - | | `list_gates` | *none* | Every gate in the account — id, name, description, model, task type, creation date | | `get_gate` | `gateId` (UUID) | Full gate config — model, fallback chain, task type, spending limits | | `list_logs` | `gate?`, `since?` (ISO 8601), `success?`, `limit?` (1–100, default 20) | Recent request logs — timestamp, gate, model, cost, latency, success/failure | | `get_recommendations` | `gateId` (UUID) | Cortex intelligence report — themes, drift detection, optimization recommendations. `{ report: null }` when no Cortex run exists yet | | `list_experiments` | `gateId?`, `status?`, `projectId?` | Experiments (shadow + split) — id, name, status, test type, variants, goal metric, configuration | The npm package always reflects the current toolset; check `verlon-mcp --help` after install. ## Configuration | Env var | Default | Notes | | - | - | - | | `VERLON_API_KEY` | — | **Required.** Your Verlon API key (`sk-vrln-...`) | | `VERLON_BASE_URL` | `https://api.verlon.ai` | Optional override for self-hosted Verlon | ## Dashboard Generate API keys for the MCP server at [verlon.ai/dashboard/settings/keys](https://verlon.ai/dashboard/settings/keys). Manage gates the MCP server will inspect at [verlon.ai/dashboard/gates](https://verlon.ai/dashboard/gates). # OpenAI SDK Integration Source: https://docs.verlon.ai/integrations/openai-sdk How to use Verlon AI with the OpenAI SDK Verlon AI provides an OpenAI-compatible API endpoint. You can use the official OpenAI SDK by pointing it to Verlon's base URL instead of OpenAI's. ## Quick Start ### Prerequisites * Verlon AI account ([sign up](https://verlon.ai/signup)) * Verlon API key from your [Dashboard](https://verlon.ai/dashboard) * A configured gate (gate UUID) ### Installation ```bash npm theme={null} npm install openai ``` ```bash pnpm theme={null} pnpm add openai ``` ```bash yarn theme={null} yarn add openai ``` ### Configuration Update your OpenAI client initialization - just 2 lines: ```typescript Before theme={null} import OpenAI from 'openai'; const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY, }); ``` ```typescript After theme={null} import OpenAI from 'openai'; const openai = new OpenAI({ baseURL: 'https://api.verlon.ai/v1', // Add this apiKey: process.env.VERLON_API_KEY, // Change this }); ``` ### Making Requests Verlon supports three ways to specify your gate: **Using gateId (Recommended)** ```typescript theme={null} // @ts-ignore - gateId is a Verlon AI extension const response = await openai.chat.completions.create({ gateId: process.env.GATE_ID, messages: [ { role: 'user', content: 'Hello!' } ], }); ``` > [!NOTE] > > Requires `@ts-ignore` comment for TypeScript, but provides the clearest intent. **Using model field** ```typescript theme={null} const response = await openai.chat.completions.create({ model: process.env.GATE_ID, // Gate UUID as model messages: [ { role: 'user', content: 'Hello!' } ], }); ``` > [!TIP] > > No TypeScript errors, works seamlessly with existing code. **Using header** ```typescript theme={null} const openai = new OpenAI({ baseURL: 'https://api.verlon.ai/v1', apiKey: process.env.VERLON_API_KEY, defaultHeaders: { 'X-Verlon-Gate-Id': process.env.GATE_ID, }, }); // Then make requests without specifying gateId const response = await openai.chat.completions.create({ messages: [ { role: 'user', content: 'Hello!' } ], }); ``` > [!TIP] > > Cleanest for TypeScript - set once, use everywhere. ## Responses API Verlon also serves OpenAI's newer Responses API at `/v1/responses` — the same two-line client setup, then `openai.responses.create()`: ```typescript theme={null} const response = await openai.responses.create({ model: process.env.GATE_ID, // must be an OpenAI-backed gate input: 'Summarize this support ticket for the on-call engineer.', reasoning: { effort: 'medium' }, }); console.log(response.output_text); ``` **When to prefer it over chat completions:** tool-heavy agent loops (Responses removes chat completions' tool-count ceiling) and reasoning models — the route passes `reasoning` items through verbatim, including `encrypted_content`, so multi-turn tool loops replay correctly. **Two constraints to know:** * **OpenAI-backed gates only.** The Responses dialect is OpenAI-specific; a gate that routes to another provider returns a 400. Chat completions remains the portable default that works with every provider behind any gate. * **Stateless.** `store: true` and `previous_response_id` are rejected — echo prior `reasoning` and `function_call` items back in `input` each turn, exactly as you would against a stateless upstream. ## Streaming Example Streaming works exactly like the standard OpenAI SDK: ```typescript theme={null} const stream = await openai.chat.completions.create({ gateId: process.env.GATE_ID, messages: [ { role: 'system', content: 'You are a helpful assistant.' }, { role: 'user', content: 'Tell me a joke.' } ], stream: true, }); for await (const chunk of stream) { const content = chunk.choices[0]?.delta?.content || ''; process.stdout.write(content); } ``` ## What's Supported > **Streaming** > > Full support for `stream: true` with identical behavior to OpenAI > **Tool/Function Calling** > > Works across all providers - Verlon handles format conversion > **Vision** > > Image inputs in messages fully supported > **All Message Types** > > System, user, assistant, and tool messages all supported > **Standard Parameters** > > `temperature`, `max_tokens`, `top_p`, and other OpenAI parameters > **Usage & Cost Tracking** > > Standard `response.usage` with token counts, plus Verlon's `cost` field ## Language Examples ```typescript TypeScript theme={null} import OpenAI from 'openai'; const openai = new OpenAI({ baseURL: 'https://api.verlon.ai/v1', apiKey: process.env.VERLON_API_KEY, }); const response = await openai.chat.completions.create({ gateId: process.env.GATE_ID, messages: [ { role: 'user', content: 'Hello!' } ], }); console.log(response.choices[0].message.content); ``` ```python Python theme={null} from openai import OpenAI client = OpenAI( base_url="https://api.verlon.ai/v1", api_key="sk-vrln-your_api_key", ) response = client.chat.completions.create( model="your-gate-uuid", messages=[ {"role": "user", "content": "Hello!"} ] ) print(response.choices[0].message.content) ``` ```javascript JavaScript theme={null} const OpenAI = require('openai'); const openai = new OpenAI({ baseURL: 'https://api.verlon.ai/v1', apiKey: process.env.VERLON_API_KEY, }); async function chat() { const response = await openai.chat.completions.create({ gateId: process.env.GATE_ID, messages: [ { role: 'user', content: 'Hello!' } ], }); console.log(response.choices[0].message.content); } chat(); ``` ```bash cURL theme={null} curl https://api.verlon.ai/v1/chat/completions \ -H "Content-Type: application/json" \ -H "Authorization: Bearer sk-vrln-your_api_key" \ -d '{ "gateId": "your-gate-uuid", "messages": [ {"role": "user", "content": "Hello!"} ] }' ``` ## Migration Guide Follow these steps to migrate an existing OpenAI application: ### 1. Update Client Initialization Find where you initialize the OpenAI client and update it: ```typescript theme={null} // Before const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY }); // After const openai = new OpenAI({ baseURL: 'https://api.verlon.ai/v1', apiKey: process.env.VERLON_API_KEY, }); ``` ### 2. Add Gate ID Add `gateId` to your completion calls: ```typescript theme={null} // Before const response = await openai.chat.completions.create({ model: 'gpt-4o', messages: [...], }); // After const response = await openai.chat.completions.create({ gateId: process.env.GATE_ID, // Add this messages: [...], // Everything else stays the same }); ``` ### 3. Update Environment Variables ```bash .env theme={null} VERLON_API_KEY=sk-vrln-your_key GATE_ID=your-gate-uuid ``` ### 4. Test & Deploy 1. Test in development 2. Verify requests appear in Verlon dashboard 3. Check cost tracking is working 4. Deploy to production > [!NOTE] > > **Easy Rollback:** If anything breaks, revert the 2 changes (`baseURL` + `apiKey`) - takes 30 seconds. ## Advanced Usage ### Multiple Gates Use different gates based on the task: ```typescript theme={null} const COMPLEX_TASK_GATE = 'uuid-for-gpt-4o'; const SIMPLE_TASK_GATE = 'uuid-for-gpt-4o-mini'; async function complexTask(prompt: string) { return await openai.chat.completions.create({ gateId: COMPLEX_TASK_GATE, messages: [{ role: 'user', content: prompt }], }); } async function simpleTask(prompt: string) { return await openai.chat.completions.create({ gateId: SIMPLE_TASK_GATE, messages: [{ role: 'user', content: prompt }], }); } ``` ### Environment-Specific Configuration ```typescript theme={null} const baseURL = process.env.NODE_ENV === 'production' ? 'https://api.verlon.ai/v1' : 'http://localhost:3001/v1'; // Local Verlon API for dev const openai = new OpenAI({ baseURL, apiKey: process.env.VERLON_API_KEY }); ``` ### Error Handling ```typescript theme={null} try { const response = await openai.chat.completions.create({ gateId: process.env.GATE_ID, messages: [{ role: 'user', content: 'Hello!' }], }); console.log(response.choices[0].message.content); } catch (error) { if (error.status === 404) { console.error('Gate not found - check your GATE_ID'); } else if (error.status === 429) { console.error('Rate limit exceeded'); } else { console.error('Request failed:', error.message); } } ``` ## How It Works 1. **You send**  OpenAI SDK request to `https://api.verlon.ai/v1/chat/completions` 2. **Verlon receives**  Validates gate, applies routing rules 3. **Verlon routes**  Sends to the model configured in your gate (GPT, Claude, Gemini, etc.) 4. **Provider responds**  Returns response in provider's format 5. **Verlon normalizes**  Converts back to OpenAI format 6. **You receive**  Standard OpenAI response with added cost/metadata Your code doesn't know the difference. ## FAQ > **Do I need to change my code?** > > No. Just change `baseURL` and `apiKey` in the OpenAI client initialization. > **Can I still use OpenAI directly?** > > Yes. Just revert `baseURL` and `apiKey` to OpenAI values. > **Does streaming work?** > > Yes. `stream: true` works exactly like OpenAI. > **Can I use Claude or Gemini?** > > Yes. Configure your gate to use any model  your code stays the same. > **What about function calling?** > > Fully supported. Verlon handles the conversion across all providers. > **Do I need the Verlon SDK?** > > No. This approach uses only the OpenAI SDK. > **Does this work with Vercel AI SDK?** > > Yes. The Vercel AI SDK's OpenAI adapter works with Verlon's OpenAI-compatible endpoint. ## Where the Verlon SDK fits This guide is the recommended integration path for inference — new projects and existing apps alike. The [Verlon SDK](/sdk-reference/overview) is not an alternative inference client; it's the observability SDK, and it composes with this setup rather than replacing it: ```typescript theme={null} import OpenAI from 'openai'; import { Verlon } from '@verlon-ai/sdk'; const verlon = new Verlon({ apiKey: process.env.VERLON_API_KEY }); const agent = verlon.agent(process.env.GATE_ID); const respond = agent.task('respond'); await agent.trace({ conversationId: chatId }, async () => { // clientOptions() supplies baseURL, a trace-aware fetch, and // gate/task identity headers — this call lands on the trace timeline const openai = new OpenAI({ apiKey: process.env.VERLON_API_KEY, ...respond.clientOptions('openai'), }); return openai.chat.completions.create({ messages }); }); ``` Reach for it when you want agent traces, per-task model control, and session grouping in the dashboard — see [Agent Tracing](/platform/agent-gates). The gateway does not need it for routing, logging, or cost tracking; those come with this guide's two-line setup. Video generation is served drop-in too — the async Videos surface (`openai.videos.create()` / `retrieve()` / `downloadContent()`) works against `/v1/videos`, and OCR against [`/mistral/v1/ocr`](/provider-compatibility/mistral). ## Next Steps > **[Create a Gate](/dashboard/creating-gates)** > > Set up routing rules and configure models > **[View Request Logs](/dashboard/monitoring)** > > Monitor your API usage and costs > **[Configure Fallbacks](/platform/gates)** > > Learn about fallback strategies > **[Verlon SDK](/sdk-reference/overview)** > > Add agent tracing and task attribution on top of this setup # Agent Gates Source: https://docs.verlon.ai/platform/agent-gates Trace every run of your agent — every LLM call, tool execution, and sub-agent — and control each call site's model remotely ## What are Agent Gates? An agent gate is the identity of one agent in Verlon, the way a standard gate is the identity of one LLM call. It answers two questions: * **What did my agent do?** Every run of your agent becomes a **trace**: a grouped timeline of its LLM calls, tool executions, and sub-agents, with cost, latency, and coverage accounting. * **What should each call site run on?** Your agent's LLM call sites become **tasks** you can control from the dashboard: pin a model per call site or delegate a call site to an existing gate — without redeploying code. Verlon sits in the request path, so LLM calls are captured automatically. Tools and run boundaries take one line each. ### When to use an agent gate | Scenario | Gate type | | - | - | | Single LLM call (summarization endpoint, one-shot API) | Standard gate | | A chatbot — one call per turn, grouped into conversations | Agent gate | | A loop or workflow that makes several LLM calls per run | Agent gate | | Agents that call tools you want on the timeline | Agent gate | | Different models for different call sites, managed remotely | Agent gate | A chatbot's traces are legitimately thin — one generation per turn — and that's fine: the session grouping (one row per conversation, turns in order) is what the agent gate buys you there. ## Vocabulary Agent gates use standard observability terms with their standard meanings: | Term | Meaning | | - | - | | **Agent gate** | The registered identity of one agent — home of its tasks, traces, and sessions. | | **Task** | A named LLM call site inside the agent (`classify`, `respond`, …). The unit of remote control. | | **Trace** | One bounded run of the agent, end to end — including sub-agents, which appear as nested spans in the *same* trace. W3C trace ids. | | **Span** | One step inside a trace: an `agent` span, a `generation` (LLM call), or a `tool` execution. | | **Session** | A conversation label on traces — *your* id (chat id, ticket number). Sessions never start, end, or expire; they're a grouping key, not a state. | Every span also carries **provenance**: `Observed` (captured at the gateway — ground truth), `Reported` (measured by the SDK), or `OTel` (ingested from your own OpenTelemetry instrumentation). Tool calls that only appear in proxied traffic show as `Evidenced` — facts without fabricated measurements. ## Quickstart **1. Create an agent gate** Dashboard → Agent Gates → **Create Agent Gate**. Only a name is required — no model configuration. Calls run on whatever model your code requests until you pin models per task. **2. Wrap your agent's entry point** One trace per run. Everything inside the scope lands on the same timeline. ```typescript theme={null} import { Verlon } from '@verlon-ai/sdk'; import OpenAI from 'openai'; const verlon = new Verlon({ apiKey: process.env.VERLON_API_KEY }); const agent = verlon.agent(process.env.AGENT_GATE_ID); const respond = agent.task('respond'); // one task per LLM call site export function handleMessage(chatId: string, message: string) { return agent.trace({ conversationId: chatId }, async () => { const openai = new OpenAI({ apiKey: process.env.VERLON_API_KEY, ...respond.clientOptions('openai'), }); const reply = await openai.chat.completions.create({ model: 'gpt-4o', // pass-through — a dashboard pin overrides it messages: [{ role: 'user', content: message }], }); return reply.choices[0].message.content; }); } ``` **3. Open the gate** The gate's **Traces** tab shows every run; click one for the timeline. The **Tasks** tab shows your call sites, ready to control. ## Using native provider SDKs If your agent is written with the OpenAI or Anthropic SDK, keep it. Spread `task.clientOptions()` into the client constructor — it carries the Verlon base URL, a trace-aware `fetch`, and the call site's identity headers: ```typescript theme={null} import Anthropic from '@anthropic-ai/sdk'; import OpenAI from 'openai'; import { Verlon } from '@verlon-ai/sdk'; const verlon = new Verlon({ apiKey: process.env.VERLON_API_KEY }); const agent = verlon.agent(process.env.AGENT_GATE_ID); const classify = agent.task('classify'); const respond = agent.task('respond'); // One client per call site — then your agent code is untouched vanilla SDK usage. // The provider hint sets the right base path for each SDK's URL conventions. const classifier = new Anthropic({ apiKey: process.env.VERLON_API_KEY, ...classify.clientOptions('anthropic'), }); const responder = new OpenAI({ apiKey: process.env.VERLON_API_KEY, ...respond.clientOptions('openai'), }); ``` Inside a trace scope, every call these clients make joins the run automatically — no per-call code. ## Tools on the timeline Two capture modes, and you usually get the first one for free: **Evidenced (zero setup).** When the model requests a tool and your loop feeds the result back, both directions pass through Verlon. The timeline reconstructs the execution — name, arguments, result — marked `Evidenced` because the execution itself wasn't observed (so no made-up durations). **Reported (one line per tool).** Wrap a function and every execution becomes a measured tool span — exact duration, error capture, visibility for tools the model never sees: ```typescript theme={null} const lookupOrder = verlon.tool('lookup_order', async (orderId: string) => { return db.orders.find(orderId); }); ``` Wrapped tools behave identically outside a trace scope (pure pass-through), record errors and rethrow them unchanged, and support a `redact` option for sensitive arguments. ## Sessions: multi-turn conversations `conversationId` is exactly what the name says — the id of the **conversation**, not the message. Each user message becomes one trace; the shared conversation id is what groups those traces into a session: ```typescript theme={null} // chatId is created ONCE per conversation and reused for every turn agent.trace({ conversationId: chatId }, () => handleTurn(message)); ``` > [!WARNING] > > Don't generate a fresh `conversationId` per message (`crypto.randomUUID()` inside > the message handler is the classic mistake). That creates a one-trace session > for every message and grouping never happens. Pass the id your app already > has for the conversation — the chat id, thread id, or ticket number. If your > Sessions tab shows exactly one trace in every session, this is why. A session is only a label — a chat resumed after a month just keeps grouping, with no lifecycle to manage. The gate's **Sessions** tab shows one row per label with trace counts and cumulative cost. Click a session to read the conversation forward, turn by turn. ### In the trace, or beside it? A trace is the work that produced one run's output. When a run triggers side work that doesn't shape its output — generating a chat title, indexing the exchange, firing a notification — give that work its **own trace in the same conversation, linked to the run that caused it**: ```typescript theme={null} let turnParent = ''; await agent.trace({ conversationId: chatId }, async (t) => { turnParent = t.traceparent(); return respondToUser(message); }); // Triggered side work: a sibling trace in the same session, // linked back to the turn that spawned it. await agent.trace( { conversationId: chatId, links: [{ traceparent: turnParent }] }, () => generateTitle(chatId) ); ``` The rule: **if the work contributes to the run's output, it's a span inside the trace; if the run merely triggers it, it's a sibling trace with a link.** Keeping side work out of the run's trace keeps the run's duration and cost honest; the link keeps the causation visible — the run's page shows "spawned →", the side work's page shows "triggered by →", and the session view draws the edge between the siblings. A link is a reference, never parentage: durations, costs, and coverage never aggregate across it (to *join* a trace instead, pass `traceparent`). Links to runs that were never observed are kept and shown as "triggered by an unobserved run" — same honesty rule as coverage. If your own OpenTelemetry instrumentation emits span links, they're ingested as the same edges automatically. ## Tasks: remote control per call site Tasks are how the dashboard controls your agent's models. Three states: 1. **Unconfigured (default).** A task with no config is pass-through: the call runs whatever model your code requested. Control is never required. 2. **Pinned.** Set a model on the task and every call through it runs that model — changed from the dashboard, live, no redeploy. 3. **Delegated.** Bind the task to an existing standard gate and calls inherit that gate's model, parameters, and managed instructions. Tune "how to summarize" once on a `summarize` gate; every agent that delegates to it follows. > [!NOTE] > > A model must resolve from *somewhere* on every call. The chain, in > order: **task pin or delegation → the model your code requested → the > gate's own model**. An agent gate has no model of its own; the `model` your provider SDK > passes is the pass-through value a dashboard pin or delegation > overrides. If > nothing in the chain resolves, the call fails with a clear 400 — > hardcoding a model into the gate to avoid that error defeats the > point of per-task control. You never have to pre-register tasks. Call sites seen in traffic appear on the **Tasks** tab automatically as claimable rows — the loop is *see traffic → name it → control it*. Typo'd names are flagged when they sit next to a similarly-named configured task. > [!NOTE] > > The SDK validates your declared task names against the gate when the first > trace starts and warns on mismatches (set `strictTasks: true` in the client > config to throw instead). A typo'd task name is never an error at request > time — the call passes through and the name shows up as unclaimed traffic. ## Sub-agents An agent calling another agent nests naturally — open the sub-agent's trace scope inside the parent's and it becomes an agent span in the *same* trace: ```typescript theme={null} const researcher = verlon.agent(RESEARCHER_GATE_ID); await orchestrator.trace({ conversationId }, async () => { // ...orchestrator calls... await researcher.trace(async () => { // these calls nest under the researcher's agent span }); }); ``` Recursive agents are safe: each invocation is a new span, so a run always renders as a finite tree. ## Crossing process boundaries Trace context propagates automatically within a process. Across processes or services, pass the standard W3C `traceparent` and resume: ```typescript theme={null} // producer queue.send({ job, traceparent: t.traceparent() }); // consumer — continues the SAME trace agent.trace({ traceparent: job.traceparent }, () => processJob(job)); ``` If your services already propagate `traceparent` (most tracing middleware does), Verlon rides it as-is. ## OpenTelemetry ingestion Already instrumented? Point any OTLP exporter at Verlon and your spans land in the same traces: ```bash theme={null} OTEL_EXPORTER_OTLP_ENDPOINT=https://api.verlon.ai/otlp OTEL_EXPORTER_OTLP_HEADERS="Authorization=Bearer $VERLON_API_KEY" ``` The endpoint accepts OTLP/HTTP (protobuf and JSON) and normalizes the common GenAI dialects — the official OTel `gen_ai` conventions, OpenInference, and OpenLLMetry — so framework instrumentation (Vercel AI SDK telemetry, LangChain instrumentors, the OpenAI Agents SDK via an OTel processor) works without changes. Attributes we don't recognize are preserved verbatim on the span. ## Cost alerts Per-gate thresholds under **Settings → Cost alerts**, both alert-only — a fired alert notifies you and never interrupts a run: * **Per-trace cost** — one run exceeded the threshold (fires once per trace). * **Per-session cumulative cost** — one conversation's traces add up past the threshold (fires at most once per day per session). ## Coverage: what the timeline doesn't know Agent gates never present a partial picture as a complete one. Each trace shows a coverage strip: the share of wall-clock covered by observed work (computed correctly under parallelism), unobserved gaps labeled inline, evidenced-only executions counted, and **verified span loss** — the SDK counts anything it had to drop, and sequence gaps are detected server-side, so "0 lost" is a checked claim. Two boundary rules make the math predictable: * **The trace's wall-clock is the `trace()` scope's lifetime** — scope entry to scope exit, not first span to last span. Work you do inside the scope that isn't instrumented (a database write before the first model call, post-processing after the last) shows up as an unobserved gap. That's deliberate: the gap is the honest record that something happened the timeline can't account for. Wrap it in `verlon.tool()` if you want it measured, or move it outside the scope if it isn't part of the run. * **The scope itself doesn't count as observation.** Coverage measures generations and tools, so a trace whose LLM calls bypassed Verlon reads as 0% covered — not 100% because the run boundary technically spans everything. ## Migrating from v1 agent gates The v1 session system (SDK 1.x) was replaced end to end. If you integrated against it: | v1 | v2 | | - | - | | `verlon.generateSessionId()` + `sessionId` per call | `agent.trace({ conversationId }, fn)` — ambient within the scope | | `x-verlon-session-id` as the run identifier | `traceparent` (automatic via `instrumentFetch` / `clientOptions`) | | `verlon.endSession(id)` | Not needed — a trace ends when its scope exits | | Sub-gates attached to an agent gate | Tasks with a gate binding (dashboard → Tasks → Delegate to a gate) | | Modes (`observability` / `orchestrated`), model pools | Removed — everything observes; per-task control replaces orchestration | `x-verlon-session-id` still works as a legacy alias for `x-verlon-conversation-id` — both carry the session *label* (your conversation id) rather than identifying a run. # API Keys Source: https://docs.verlon.ai/platform/api-keys Generate and manage Verlon API keys to authenticate your requests ## Overview Verlon API keys authenticate your application's requests to the Verlon AI platform. Every request to Verlon requires a valid API key in the `Authorization` header. ## Key Format Verlon API keys follow this format: ``` sk-vrln-9e1417ca5c9b793875f12b6cf3185b9d... ``` All newly minted keys start with the `sk-vrln-` prefix followed by 64 hex characters. Legacy keys minted before this prefix shipped (`verlon_*` and `layer_*`) continue to validate — no rotation required. ## Creating a Key ### From the Dashboard 1. Go to **Dashboard → API Keys** 2. Click **Create New Key** 3. Enter a name for the key (e.g., "Production", "Development", "CI/CD") 4. Click **Create** > [!WARNING] > > The full key is shown **only once** after creation. Copy it immediately and store it securely. You won't be able to see the full key again. After creation, only a masked prefix is displayed in the dashboard (e.g., `sk-vrln-9e14...`). ## Using a Key Include your API key in the `Authorization` header of every request: **Verlon SDK** ```typescript theme={null} import { Verlon } from '@verlon-ai/sdk'; const verlon = new Verlon({ apiKey: process.env.VERLON_API_KEY }); ``` **cURL** ```bash theme={null} curl -X POST https://api.verlon.ai/v3/chat \ -H "Authorization: Bearer sk-vrln-your_key" \ -H "Content-Type: application/json" \ -d '{ ... }' ``` **OpenAI SDK** ```typescript theme={null} import OpenAI from 'openai'; const openai = new OpenAI({ baseURL: 'https://api.verlon.ai/v1', apiKey: process.env.VERLON_API_KEY, }); ``` ## Managing Keys ### Multiple Keys You can create multiple API keys for different environments or services. Each key has independent access to all your gates and resources. ### Revoking a Key To revoke a key, click the **Delete** button next to it in the dashboard. Revoked keys immediately stop working — any application using that key will receive authentication errors. ### Last Used Tracking The dashboard shows when each key was last used, helping you identify unused keys that should be cleaned up. ## Security Best Practices * **Never commit API keys to source control.** Use environment variables or a secrets manager. * **Use separate keys** for development, staging, and production environments. * **Rotate keys periodically.** Create a new key, update your application, then revoke the old one. * **Revoke unused keys.** If a key hasn't been used recently, consider deleting it. ## API Keys vs Provider Keys | | Verlon API Keys | Provider Keys (BYOK) | | - | - | - | | **Purpose** | Authenticate with Verlon AI | Route through your own provider account | | **Format** | `sk-vrln-...` | Provider-specific (e.g., `sk-proj-...`) | | **Count** | Multiple allowed | One per provider | | **Affects billing** | Uses your Verlon plan quota | Charges appear on your provider account | For more on provider keys, see [Bring Your Own Keys](/platform/byok). # Bring Your Own Keys Source: https://docs.verlon.ai/platform/byok Use your own API keys from OpenAI, Anthropic, Google, Mistral, xAI, and Meta with Verlon AI ## Overview By default, requests through Verlon use Verlon's platform keys. With BYOK (Bring Your Own Keys), you can provide your own provider API keys. Requests are then billed directly to your provider account. > [!NOTE] > > BYOK is *your keys, our catalog* — your provider account serving the public > models Verlon already knows about. To register a model of your own that the > catalog has never heard of — a fine-tune, or your own inference server — see > [Bring Your Own Models](/platform/owned-models). **Why use BYOK?** * **Bypass platform rate limits** — Use your own quota * **Direct billing** — Charges appear on your provider account * **Compliance** — Some organizations require direct provider relationships * **Control** — Maintain your own API key lifecycle ## Supported Providers | Provider | Key Format | Validation Method | | - | - | - | | **OpenAI** | `sk-proj-...` | Lists models (free operation) | | **Anthropic** | `sk-ant-...` | Minimal message request | | **Google** | `AIza...` | Minimal content generation | | **Mistral** | API key | Lists models (free operation) | | **xAI** | `xai-...` | Lists models (free operation) | | **Meta** | API key | Lists models (free operation) | ## Adding a Provider Key ### From the Dashboard 1. Go to **Dashboard → Provider Keys** 2. Find the provider you want to configure and click **Add key** 3. Give the key a label (e.g. `default`, `rotation-2026q3`) and paste the credential 4. The key is validated against the provider before it saves — a bad key never lands Your key is encrypted immediately using AES-256-GCM and stored securely. Only a masked prefix is visible in the dashboard (e.g., `sk-proj-...abc123`). ### Key Management * **Multiple keys per provider** — Each key has a unique label; exactly one is the **default** * **Default key** — Serves all your traffic for that provider unless a gate pins a specific key * **Gate pinning** — A gate's Model & routing settings can pin it to a specific key; pinned gates keep their key through default rotations * **Enable/Disable** — Toggle a key without deleting it * **Test** — Re-validate any stored key against the provider at any time * **Per-key usage** — Rolling request/error counts per key, from per-request attribution * **Activity trail** — Every add, rename, default change, toggle, and delete is recorded and shown on the page * **Delete** — Soft-deleted (recoverable for 90 days, then permanently removed). The default key can't be deleted while sibling keys exist — hand the default over first ### Zero-Downtime Rotation To rotate a credential without dropping a request: 1. **Add** the new key under a fresh label 2. **Make it the default** — unpinned traffic switches on the next request 3. **Watch the old key's usage drain** to zero in the per-key usage column 4. **Delete** the old key here and **revoke** it in your provider's console Gates pinned to the old key keep working until you unpin them; a pin whose key is deleted or disabled falls back to the default automatically, so rotation can never break a gate. ## How Keys Are Used When you make a request through Verlon: 1. If the gate is **pinned** to one of your keys (and that key is active and matches the serving provider) → the pinned key is used 2. Otherwise, if you have an active **default** BYOK key for the target provider → it is used 3. Otherwise → Verlon's platform key is used as a fallback This happens transparently. Your application code doesn't change regardless of whether BYOK is configured. ## Security ### Encryption All provider keys are encrypted using **AES-256-GCM** — the same standard used by AWS KMS, Google Cloud KMS, and 1Password. * **Fresh IV** generated for every encryption operation * **Auth tag** detects any tampering with stored data * **Master key** stored separately from the database as an environment variable ### What Verlon Stores | Data | Storage | | - | - | | Full API key | Encrypted (AES-256-GCM) | | Key prefix | Plaintext (for display, e.g., `sk-proj-...abc`) | | Provider name | Plaintext | | Active status | Plaintext | ### Trust Model Verlon's BYOK uses platform-controlled encryption, which is the industry standard for API proxies. Your keys are encrypted at rest but decrypted server-side when making provider API calls on your behalf — the same model used by Stripe, Vercel, and similar platforms. ## Cost Tracking When BYOK is active, the actual provider charges appear on your provider account. Verlon still tracks estimated costs in the dashboard for analytics and spending limit enforcement, but these are calculated from token counts and published pricing — your actual provider bill may vary slightly. ## Fallback Behavior If your BYOK key becomes invalid (expired, revoked, rate-limited): * Verlon detects the authentication error * The key is marked as invalid in the dashboard * Requests fall back to Verlon's platform key (if available) * You're notified to update your key # Gates Source: https://docs.verlon.ai/platform/gates Route AI requests through configurable gates with intelligent model selection, fallbacks, and cost controls ## What is a Gate? A gate is the core building block of Verlon AI. It's a configuration that sits between your application and AI providers, controlling how requests are routed, which models are used, and how failures are handled. When you make a request through Verlon, you reference a gate by its ID (a UUID). The gate determines: * Which model handles the request * What happens if that model fails * What parameters are applied (temperature, max tokens, etc.) * Whether spending limits are enforced * What system prompt is prepended ## Gate Types Verlon supports two gate types: | Type | Use Case | | - | - | | **Standard** | Discrete, stateless LLM calls — chat, image generation, embeddings, etc. | | **Agent** | The identity of one agent: run traces (LLM calls, tools, sub-agents on one timeline) and per-call-site model control via tasks. See [Agent Gates](/platform/agent-gates). | ## Task Types Every gate is configured for a specific task type, which determines which models are available: | Task Type | Description | Example Models | | - | - | - | | `chat` | Text generation, conversation | GPT-4o, Claude Sonnet, Gemini | | `image` | Image generation | DALL-E, Stable Diffusion | | `video` | Video generation | Runway, Pika | | `tts` | Text-to-speech | OpenAI TTS, ElevenLabs | | `embeddings` | Text embeddings | text-embedding-3, Ada | | `ocr` | Document processing | OpenAI Vision, Claude Vision | Chat gates also support optional subtypes for specialized models: * **Reasoning** — o3, o4-mini, Gemini 2.5 Pro * **Code** — Codestral, Devstral * **Realtime** — gpt-4o-realtime ## Routing Strategies Gates support three strategies for handling requests: ### Single (Default) Use only the primary model. No fallbacks. Simplest and most predictable. ### Fallback Try the primary model first. If it fails (provider outage, rate limit, etc.), try each fallback model in order until one succeeds. ``` Primary: claude-sonnet-4 → fails Fallback 1: gpt-4o → fails Fallback 2: gemini-2.0-flash → succeeds ✓ ``` ### Round-Robin Randomly distribute requests across the primary model and all fallback models. Useful for load balancing or informal cost distribution across providers. ## Creating a Gate ### From the Dashboard 1. Go to **Dashboard → Gates → Create New Gate** 2. Fill in the required fields: * **Name** — A human-readable label for the gate (e.g., `customer-support`). The gate ID (UUID) is what you use in API calls. * **Task Type** — What kind of requests this gate handles * **Model** — The primary model to use 3. Optionally configure: * **Fallback models** and routing strategy * **System prompt** applied to all requests * **Temperature**, **max tokens**, and **top P** defaults * **Spending limits** with alert or block enforcement * **Structured output** (JSON schema) for consistent response formats ### Via the API ```bash theme={null} curl -X POST https://api.verlon.ai/v1/gates \ -H "Authorization: Bearer sk-vrln-your_key" \ -H "Content-Type: application/json" \ -d '{ "name": "customer-support", "taskType": "chat", "model": "claude-sonnet-4-20250514", "description": "Handles customer support queries", "fallbackModels": ["gpt-4o"], "routingStrategy": "fallback", "temperature": 0.7, "systemPrompt": "You are a helpful customer support agent." }' ``` ## Using a Gate Once created, reference the gate in your requests: **OpenAI SDK** ```typescript theme={null} import OpenAI from 'openai'; const openai = new OpenAI({ baseURL: 'https://api.verlon.ai/v1', apiKey: process.env.VERLON_API_KEY }); const response = await openai.chat.completions.create({ model: 'your-gate-uuid', // gate UUID in the model slot messages: [ { role: 'user', content: 'How do I reset my password?' } ] }); ``` **OpenAI SDK (default header)** ```typescript theme={null} import OpenAI from 'openai'; const openai = new OpenAI({ baseURL: 'https://api.verlon.ai/v1', apiKey: process.env.VERLON_API_KEY, defaultHeaders: { 'X-Verlon-Gate-Id': 'your-gate-uuid' } }); const response = await openai.chat.completions.create({ model: 'claude-sonnet-4-20250514', messages: [ { role: 'user', content: 'How do I reset my password?' } ] }); ``` **cURL** ```bash theme={null} curl -X POST https://api.verlon.ai/v3/chat \ -H "Authorization: Bearer sk-vrln-your_key" \ -H "Content-Type: application/json" \ -d '{ "gateId": "your-gate-uuid", "data": { "messages": [ { "role": "user", "content": "How do I reset my password?" } ] } }' ``` ## Gate Configuration Reference The full gate configuration interface. Fields marked **required** must be provided when creating a gate. All other fields are optional. ### Core Fields | Field | Type | Required | Description | | - | - | - | - | | `name` | `string` | Yes | Human-readable label for the gate (e.g., `customer-support`). Must be unique per user. Used for display in the dashboard and logs. | | `model` | `SupportedModel` | Yes | Primary model to use for requests (e.g., `claude-sonnet-4-20250514`, `gpt-4o`). | | `taskType` | `ModelType` | Yes | Determines which models are available. One of: `chat`, `image`, `video`, `tts`, `embeddings`, `ocr`, `audio`, `stt`, `moderation`. | | `taskSubtype` | `ModelSubtype` | No | Specialization for chat models. One of: `reasoning`, `code`, `realtime`. | | `description` | `string` | No | Describes what this gate is for. Used by the Architect for smart routing recommendations. The dashboard offers an **Auto-enhance description** option — if you accept the Architect's suggestion, the enhanced version replaces this field. | | `tags` | `string[]` | No | Organizational labels for filtering and grouping gates. | ### Model Routing | Field | Type | Default | Description | | - | - | - | - | | `routingStrategy` | `'single' \| 'fallback' \| 'round-robin'` | `'single'` | How requests are distributed across models. | | `fallbackModels` | `SupportedModel[]` | `[]` | Ordered list of fallback models. Used by `fallback` and `round-robin` strategies. | ### Request Parameters Defaults applied to all requests through the gate. Can be overridden per-request if `allowOverrides` is configured. | Field | Type | Default | Description | | - | - | - | - | | `systemPrompt` | `string` | — | System prompt prepended to all requests. | | `temperature` | `number` | — | Controls randomness (0 = deterministic, 2 = most random). | | `maxTokens` | `number` | — | Maximum response length in tokens. | | `topP` | `number` | — | Nucleus sampling threshold (0 - 1). Lower = more focused. | | `allowOverrides` | `boolean \| OverrideConfig` | `false` | Whether clients can override gate defaults per-request. Can be `true` (all overrides) or an object specifying which fields: `{ model?: boolean, temperature?: boolean, maxTokens?: boolean, topP?: boolean }`. | ### Smart Routing (Architect) Verlon's AI agent (the Architect) analyzes your gate's description and usage patterns to recommend optimal models. | Field | Type | Default | Description | | - | - | - | - | | `costWeight` | `number` | — | Weight for cost optimization (0 - 1). | | `latencyWeight` | `number` | — | Weight for latency optimization (0 - 1). | | `qualityWeight` | `number` | — | Weight for quality optimization (0 - 1). | | `analysisMethod` | `'cost' \| 'balanced' \| 'performance' \| 'custom'` | — | Preset optimization profile. `custom` uses the individual weights above. | | `maxCostPer1kTokens` | `number` | — | Maximum acceptable cost per 1,000 tokens (USD). | | `maxLatencyMs` | `number` | — | Maximum acceptable latency in milliseconds. | | `reanalysisPeriod` | `'daily' \| 'weekly' \| 'monthly' \| 'never'` | `'never'` | How often the Architect re-evaluates model recommendations. | | `autoApplyRecommendations` | `boolean` | `false` | Automatically apply Architect recommendations without manual review. | | `taskAnalysis` | `TaskAnalysis` | — | Read-only. The Architect's current recommendation including primary model, alternatives, and reasoning. | ### Structured Output Force responses into a consistent format. Native JSON schema support for OpenAI models; prompt-injected for other providers. | Field | Type | Default | Description | | - | - | - | - | | `responseFormatEnabled` | `boolean` | `false` | Enable structured output. | | `responseFormatType` | `'text' \| 'json_object' \| 'json_schema'` | `'text'` | Output format. `json_object` guarantees valid JSON. `json_schema` validates against a strict schema. | | `responseFormatSchema` | `object` | — | JSON schema for `json_schema` mode. Defines the exact structure of responses. | ### Spending Limits Control costs at the gate level. See [Spending](/platform/spending) for account-level controls. | Field | Type | Default | Description | | - | - | - | - | | `spendingLimit` | `number \| null` | `null` | Dollar cap per period. `null` = no limit. | | `spendingLimitPeriod` | `'monthly' \| 'daily'` | `'monthly'` | How often the spending counter resets. | | `spendingEnforcement` | `'alert_only' \| 'block'` | `'alert_only'` | `alert_only` warns but allows requests. `block` rejects requests when the limit is hit. | > [!NOTE] > > Agent gates are created with identity only (name + description) — their control surface is per-task model config. See [Agent Gates](/platform/agent-gates). ### Read-Only Fields These fields are returned by the API but cannot be set directly. | Field | Type | Description | | - | - | - | | `id` | `string` | Gate UUID. This is what you use in API calls. | | `userId` | `string` | Owner's user ID. | | `createdAt` | `Date` | When the gate was created. | | `updatedAt` | `Date` | When the gate was last modified. | | `spendingCurrent` | `number` | Current period spending (USD). | | `spendingPeriodStart` | `string` | Start of the current spending period. | | `spendingStatus` | `'active' \| 'suspended'` | Whether the gate is active or suspended due to spending limit. | ## Dashboard Tabs When editing a gate in the dashboard, configuration is organized into tabs: ### Basic Info Name, description, task type, subtypes, and tags. Includes an **Auto-enhance description** toggle — when enabled, the Architect generates an improved version of your description optimized for smart routing analysis. If you accept the suggestion, it replaces your gate's `description` field with the enhanced version. ### Models & Routing Primary model, fallback models, routing strategy, optimization weights, and smart routing (Architect) configuration. ### Request Config System prompt, temperature, max tokens, top P, and override permissions. ### Spending Limits Gate-level spending cap, period, and enforcement type. ### Connections Shows which agent gates delegate a task to this gate. When an agent gate's task is bound to this gate, calls through that task inherit this gate's model, parameters, and instructions. This is read-only context — delegation is managed from the agent gate's Tasks tab. See [Agent Gates](/platform/agent-gates). ### Danger Zone Delete the gate. This action is irreversible. Agent-gate tasks that delegated to this gate fall back to pass-through. ## Gate Limits The number of gates you can create depends on your plan tier. Check your current usage at **Dashboard → Settings**. # Logs Source: https://docs.verlon.ai/platform/logs View detailed request logs with metrics, costs, and full request/response data ## Overview Every request through Verlon is logged with comprehensive metadata. Logs let you inspect individual requests, debug issues, and understand how your gates are performing. ## Viewing Logs Go to **Dashboard → Logs** to see a table of all requests with: | Column | Description | | - | - | | **Status** | Success or error | | **Gate** | Which gate handled the request | | **Model** | The model that was used (may differ from requested model if fallback triggered) | | **Tokens** | Input + output token count | | **Cost** | Estimated cost in USD | | **Latency** | Response time in milliseconds | | **Time** | When the request was made | ### Filtering Filter logs by: * **Gate** — See requests for a specific gate * **Status** — Show only successful or failed requests ### Pagination Logs are paginated with 50 entries per page by default. ## Request Detail Click any log entry to see full details: ### Overview * Status, timestamp, gate name, model used * Token breakdown (input/output), cost, latency ### Error Details If the request failed, the error message and details are shown. ### Request Payload The full JSON request sent to the provider, including: * Messages array * System prompt * Tool definitions * Model parameters (temperature, max\_tokens, etc.) > [!NOTE] > > Prompt and response content is only present in your logs if you have > opted in to payload storage in **Settings → Security & Privacy**. By > default, only metadata is captured (see "Storage and retention" below). ### Response Payload The full JSON response from the provider, including: * Generated content * Usage statistics * For image generation: inline preview with copy/download options * For tool calls: extracted tool use blocks ## What's Tracked Every request log captures: | Field | Description | | - | - | | `model_requested` | Model specified in the request | | `model_used` | Model that actually handled the request (may differ with fallback routing) | | `prompt_tokens` | Input token count | | `completion_tokens` | Output token count | | `cost_usd` | Estimated cost | | `latency_ms` | Response time | | `success` | Whether the request completed successfully | | `error_message` | Error details (if failed) | | `tool_calls` | Extracted tool use blocks from the response | | `tool_results` | Extracted tool results from the request | ## Logs API Query logs programmatically: ```bash theme={null} # List recent logs curl "https://api.verlon.ai/v1/logs?limit=20" \ -H "Authorization: Bearer sk-vrln-your_key" # Filter by gate curl "https://api.verlon.ai/v1/logs?gateId=your-gate-id&limit=20" \ -H "Authorization: Bearer sk-vrln-your_key" # Get overview analytics curl "https://api.verlon.ai/v1/logs/overview" \ -H "Authorization: Bearer sk-vrln-your_key" ``` The overview endpoint returns aggregated stats: * Total requests, total cost, average latency * Active gate count * Recent request summaries ## Storage and retention Verlon's storage model is privacy-first by default. ### What is always stored For every request, Verlon records **metadata only**: * Gate id and gate configuration at request time * Model used, token counts, cost, latency, success / error status * Timestamps This metadata powers your dashboard charts and stays available for as long as your account is active. ### What is optional The **content** of the request and response — the messages array, system prompt, tool definitions, tool calls, tool results — is *not* stored by default. To capture it for your dashboard logs, flip the **Store request and response content** toggle in **Settings → Security & Privacy**. When enabled: * New requests are logged with their full prompt and response bodies. * Content is wrapped in an AES-256-GCM envelope before being written to the database; the master key is held separately in the application environment. * A nightly retention worker nulls payload columns once they age past 30 days for free accounts. Metadata on those rows survives the sweep. * Verlon does not scrub, modify, or filter the content you send. What you sent is what we store. You are responsible for deciding whether a given prompt is safe to send through us. ### Turning it off Switching the toggle back to off stops *new* requests from capturing content. Rows that were stored before the change continue to exist in encrypted form until the nightly retention worker nulls them within 30 days. ### Internal access Verlon employees cannot decrypt your stored payloads in bulk. The admin listing tool returns metadata only; revealing a single row requires the employee to pick a reason category and type a specific detail. Every reveal writes an audit row before the decrypted content is returned. Email [micah@verlon.ai](mailto:micah@verlon.ai) if you want a report of every internal view of your account's data. # Bring Your Own Models Source: https://docs.verlon.ai/platform/owned-models Register fine-tunes and self-hosted endpoints as first-class models you can route to through Verlon AI ## Overview An **owned model** is a model *you* serve: a fine-tune you deployed on Fireworks, or your own OpenAI-compatible inference server. You register it once, and it becomes routable anywhere Verlon takes a model — gates, fallback chains, the model picker — with the same observability, cost tracking, and hot-swapping as a catalog model. > [!NOTE] > > **Owned models are not [BYOK](/platform/byok).** BYOK means *your keys, our > catalog* — your provider account serving the public models Verlon already > knows about. Owned models mean *your model*: something the catalog has never > heard of, registered with its own credential. Rotating a BYOK channel key > never affects an owned model, and deleting an owned model never touches your > catalog access. ## Where a model can live | Host | What it's for | Credential | | - | - | - | | **Fireworks** | A fine-tune or custom deployment on your Fireworks account | Required — a Fireworks API key | | **Self-hosted URL** | Any OpenAI-compatible server you operate — vLLM, SGLang, LM Studio, llama.cpp, a dedicated cloud deployment | Optional — many self-hosted servers have no auth | Together AI, Bedrock Custom Model Import, and SageMaker are planned; registering them is refused today rather than accepted and mis-billed. Owned models are **chat models** for now. Embeddings arrive alongside embeddings support for the Fireworks channel; image and audio wait for their own pricing units. ## Registering a model From the dashboard, go to **Models → Your Models → Your own models → Register a model**. The form asks four questions. **1. Name** A human name. Verlon derives the routing reference from it — "Support summarizer tune" becomes `custom:support-summarizer-tune`. The reference is permanent; renaming means registering again. **2. Where it lives** Fireworks, or a self-hosted URL. For Fireworks, the **model id** is the fully-qualified id from your account, like `accounts/your-org/models/your-tune-v1`. For a self-hosted endpoint, it's whatever model name your server exposes. **3. Credential** Stored encrypted and scoped to this registration only — never shared with your channel keys, and never returned by the API. Self-hosted endpoints without auth can leave it empty. **4. Cost per 1M tokens** What this model costs *you*. See [Declaring cost](#declaring-cost) below. Verlon probes the endpoint when you save, but **saves either way** — a self-hosted box that's asleep is registered and marked amber until a probe succeeds, rather than blocking you. ## Routing to an owned model Reference it as `custom:` anywhere a model id is expected: ```json theme={null} { "name": "support-summarizer", "taskType": "chat", "model": "custom:support-summarizer-tune" } ``` It appears in the model picker under **Your own models** — listed first, since a model you went to the trouble of deploying is usually the one you're looking for. It works in fallback chains alongside catalog models, and its traffic shows up in logs and the catalog's usage columns like anything else. References resolve **per user**. `custom:support-tune` means your registration and nobody else's. ## Declaring cost Verlon records a cost for every request — it feeds your logs, analytics, spending limits, and the model comparison view. The catalog knows what public models cost; it cannot know what *your* deployment costs you. So you declare it. * **Input** and **output** rates, in USD per 1M tokens. * **Cached input** is optional. Left empty, cached tokens bill at the full input rate — which can only *overstate* your real bill, never hide spend. * **Reasoning tokens need no field.** OpenAI-compatible servers report them inside the completion count, so your output rate already covers them. **Leave every rate empty and the model records `$0`,** shown as `owned · $0`. For a server you already pay for by the hour, zero per-token cost is the honest answer — not a pricing error. If you'd rather see your infrastructure amortized per token, declare a rate that reflects it. > [!WARNING] > > **Pricing changes apply to future requests only.** Every request's cost is > computed and stored when it runs, so editing rates never rewrites history. If > a pricing mistake has already distorted your recorded spend, contact support > and we'll correct the affected data. ## Self-hosted endpoints must be reachable from the internet Verlon calls your endpoint from our servers. That has consequences worth knowing before you register one: * **`localhost` will not work.** Not `http://localhost:11434/v1`, not a private LAN address, not a VPN-only host — our servers cannot see them. Expose the server through a tunnel (Cloudflare Tunnel, ngrok, Tailscale Funnel) or put it behind a public hostname. * **Private and reserved addresses are refused** at registration, along with cloud metadata endpoints and URLs with credentials embedded in them. * **Redirects are not followed**, and hostnames are re-resolved and checked at connection time — so give us the final URL. * **Use HTTPS and an auth token** if the endpoint is public. Anything reachable by us is reachable by others. If your endpoint is up but a probe fails, the registration still saves and shows **Unreachable**. Routing to it will fail until the box answers — the badge is telling you the truth, not blocking you. ## Managing a registration From **Manage** on any owned model you can: * **Edit pricing** — forward-looking, per the warning above. * **Replace the credential** — rotating a key triggers a fresh reachability probe. * **Delete the registration** — refused while any of your gates still routes to it. Repoint those gates first, so routing never dangles. ## What Verlon does and doesn't see Your inference runs on your account or your hardware, on your credential. Verlon routes the request, records what it cost and how it performed, and never becomes an intermediary for your inference spend. Owned models are never added to the shared catalog — they're yours, visible only to you. # Plans Source: https://docs.verlon.ai/platform/plans What each plan includes, how unit allowances work, and what happens when you grow past one ## The model Everything that moves a request is free on every plan — routing across every provider, live model swapping, failover chains, spend caps, and bringing your own keys at zero markup. The subscription pays for the intelligence layer: the machinery that judges, compares, and proves your AI works on your real traffic. We never mark up tokens, never charge per seat, and never bill overages. ## Plans at a glance | | Free | Pro \$49/mo | Team \$149/mo | Enterprise | | - | - | - | - | - | | [Units](/platform/units) per month | 10M | 250M | 1B | Custom | | Log history | 7 days | 30 days | 90 days | Custom | | Every model, live swap, failover, spend caps | ✓ | ✓ | ✓ | ✓ | | Bring your own keys — zero markup | ✓ | ✓ | ✓ | ✓ | | Your own models as routing targets | ✓ | ✓ | ✓ | ✓ | | Claude Code, Cursor, Codex, MCP connectors | ✓ | ✓ | ✓ | ✓ | | Observability — logs, analytics, tracing | ✓ | ✓ | ✓ | ✓ | | Quality scoring on real traffic | — | ✓ | ✓ | ✓ | | Experiments and shadow comparisons | — | ✓ | ✓ | ✓ | | Independence Reports | — | ✓ | ✓ | ✓ | | Shared gates, RBAC, team budgets | — | — | ✓ | ✓ | | SSO/SAML, SLA, self-host, DPA | — | — | — | ✓ | | Seats | 1 | Unlimited | Unlimited | Unlimited | Rate limits exist on every plan as abuse prevention — they scale with your tier and are not something we sell. [Units](/platform/units) never gate routing: on Free they meter tracked traffic, on paid plans they size intelligence coverage. ## Why seats are unlimited Units already capture how much your team uses the platform. Charging per seat on top of that would double-count the same growth, so we don't. Invite everyone. ## What happens if you go over Your traffic is never blocked, on any plan: 1. We notify you at **80%** and again at **100%** of your plan's monthly units. These are informational — nothing bills extra. 2. On paid plans, requests keep serving **and recording at full quality through the end of the period**, no matter how far over you go. 3. There's a grace band of roughly **20%** before a plan change is even considered, and only **two consecutive periods** over moves you up — one spiky month never bounces you. 4. Plan moves take effect at the **next** billing period, with advance notice — and only upward. We never downgrade you automatically; dropping your plan is one click in Settings, anytime. On the Free plan, routing never stops. Past the 10M-unit allowance, your requests keep flowing and stay in your history — they're just locked from view until you upgrade (upgrading unlocks them retroactively) or your period resets and new traffic records again. ## Add-on: managed dedicated deployment Open-weight models in your own cloud account. We write the deployment, run it with you, monitor it, upgrade models, and keep parity proven — compute bills to your cloud directly, never through us. From $1,000 setup, then $499/month per environment. Honestly: this only makes financial sense above roughly \$2,000/month of model spend, or when data residency outweighs cost. Below that, serverless open-weights setup (Together, Fireworks, and similar) is included free on every plan. ## Cancelling Anytime, from Settings → Billing → Manage subscription. You keep your plan until the end of the period you paid for, then land back on Free. Your gates, keys, and analytics are untouched — and leaving Verlon entirely is a base-URL change, documented at [verlon.ai/leave](https://www.verlon.ai/leave). # Spending Source: https://docs.verlon.ai/platform/spending Monitor and control your AI spending with real-time cost tracking and configurable limits ## Overview Verlon tracks the cost of every request in real-time, giving you visibility into how much you're spending across all your gates. You can set spending limits at both the account level and per-gate level. > [!NOTE] > > Spend caps and plan [units](/platform/units) are different levers: > spending limits are **your** cost controls in dollars, enforced how > you choose. Units measure gateway work and only inform which > [plan](/platform/plans) fits — they never block requests and never > bill overages. ## Account-Level Spending ### Viewing Your Spending Go to **Dashboard → Spending** to see: * **Current spending** — Total cost in the current billing period * **Spending limit** — Your configured cap (if set) * **Percentage used** — How close you are to your limit * **Period start** — When the current 30-day rolling period began ### Setting a Spending Limit 1. Go to **Dashboard → Spending** 2. Set your **monthly spending limit** (in USD) 3. Choose an **enforcement type**: * **Alert only** — You're warned when you exceed the limit, but requests continue * **Block** — Requests are rejected once the limit is hit ### Alert Threshold By default, Verlon alerts you when spending reaches 80% of your limit. This gives you time to react before hitting the cap. ## Gate-Level Spending Each gate can have its own independent spending limit, separate from your account limit. ### Configuration When creating or editing a gate: 1. Go to the **Spending Limits** tab 2. Set a **limit amount** (USD) 3. Choose a **period**: `monthly` or `daily` 4. Choose **enforcement**: `alert_only` or `block` ### How It Works * Each request through a gate increments that gate's spending counter * When the counter exceeds the limit: * **Alert only**: Request proceeds, but the gate's status shows a warning * **Block**: Request is rejected. The gate's status changes to `suspended` until the period resets. * Counters reset automatically at the start of each period ## Cost Calculation Verlon calculates costs based on: * **Token count** (input + output tokens) * **Published model pricing** from each provider * **Request type** (chat, image generation, etc.) Cost is tracked per request and aggregated into gate-level and account-level totals. > [!NOTE] > > If you're using BYOK (Bring Your Own Keys), Verlon still tracks estimated costs for analytics and limit enforcement. Your actual provider bill may vary slightly from Verlon's estimates. ## Cost Alerts (Agent Gates) Agent gates support alert-only cost thresholds — per-trace and per-session-cumulative — that notify you without ever interrupting a run. See [Agent Gates](/platform/agent-gates#cost-alerts) for details. ## Spending API You can programmatically check and manage spending via the API: ```bash theme={null} # Get current spending curl https://api.verlon.ai/v1/spending \ -H "Authorization: Bearer sk-vrln-your_key" # Response { "currentSpending": 12.45, "limit": 50.00, "periodStart": "2026-02-08T00:00:00Z", "status": "active", "limitEnforcementType": "alert_only", "percentUsed": 24.9 } ``` # Units Source: https://docs.verlon.ai/platform/units What a unit is, what counts against your plan, and how non-token work converts ## What is a unit A **unit is a token processed through the gateway** — input and output, on any model, including requests routed to your own endpoints. Units measure how much work Verlon does for you; they decide which [plan](/platform/plans) fits your usage and nothing else. There is no per-unit price and no overage billing. Units never gate routing. On the Free plan they meter how much of your traffic we **track** (logs, traces, analytics); on paid plans they size how much traffic the **intelligence layer** covers — quality scoring, shadow comparisons, and Independence Reports. For a sense of scale: **250M units a month is roughly what a Sonnet-based product with a few thousand monthly users burns.** ## What counts | Traffic | Counts? | Why | | - | - | - | | Requests that reach a model (input + output tokens) | **Yes** | The work you asked for | | Requests routed to your own endpoints | **Yes** | We still did the routing, tracing, and judging work | | Refusals and content-filtered responses | **Yes** | The model ran; tokens were consumed | | Provider-side cache reads | No | Cheap replayed context — excluding it rewards behaviour that saves you money | | Client errors that never reached a model | No | Nothing was served | | Failed requests where nothing succeeded | No | We never bill for an outage | | Retries and fallbacks | Counts **once** | A request that fails upstream and succeeds on a fallback is one request | | Verlon's own verification traffic (quality judging, shadow comparisons) | No | Our cost, absorbed into the subscription | | Dashboard actions, key management, MCP configuration, health checks | No | Not model traffic | The principle: **we count work you asked for and received.** Reliability retries, proof machinery, and provider caching are ours to absorb. ## Non-token work Some providers report no tokens at all — image generation, text-to-speech, video. Those convert to units at cost parity: \*\*$1 of provider cost ≈ 200,000 units** (about $5 per million units, a blended mid-market token price). Models that report real token usage for media (like `gpt-image-1`) meter on their actual tokens instead. | Work | Approximate units | | - | - | | One 1024×1024 image at \$0.04 | \~8,000 | | One second of generated video at \~\$0.35 | \~70,000 | | One minute of standard TTS audio | \~3,000–6,000 | These equivalents follow the single published anchor above; if the anchor ever changes, this table changes with it. ## Watching your usage The dashboard sidebar shows your units this period against your plan's allowance, and Settings → Billing carries the same meter. The API surface is `GET /v3/usage/units`: ```json theme={null} { "tier": "pro", "ceilingUnits": 250000000, "period": { "start": "2026-07-19", "end": "2026-08-19", "units": 1055682, "pctUsed": 0.4 }, "rolling30d": { "units": 2238601 }, "platformTrafficPaused": false } ``` Going over your allowance never blocks anything — see [what happens if you go over](/platform/plans#what-happens-if-you-go-over). # Anthropic Source: https://docs.verlon.ai/provider-compatibility/anthropic Every Claude model Verlon supports — plus quickstart and SDK compatibility (use the Anthropic SDK unchanged against Verlon). Every Claude model — Opus, Sonnet, and Haiku across the 3.5, 4, 4.5, 4.6, 4.7, and 4.8 generations — is routable through Verlon with your existing Anthropic SDK unchanged. Just point `baseURL` at Verlon and swap the API key. ## Chat models | Model ID | Type | Context | | - | - | - | | `claude-fable-5` | chat | 1000K | | `claude-fable-5-1` | chat | 1000K | | `claude-haiku-4-5-20251001` | chat | 200K | | `claude-haiku-5-5` | chat | 200K | | `claude-opus-4-5` | chat | 200K | | `claude-opus-4-5-20251101` | chat | 200K | | `claude-opus-4-6` | chat | 1000K | | `claude-opus-4-7` | chat | 1000K | | `claude-opus-4-8` | chat | 1000K | | `claude-opus-5` | chat | 1000K | | `claude-opus-5-5` | chat | 1000K | | `claude-sonnet-4-5-20250929` | chat | 200K | | `claude-sonnet-4-6` | chat | 1000K | | `claude-sonnet-5` | chat | 1000K | | `claude-sonnet-5-5` | chat | 1000K | > [!NOTE] > > This table is auto-generated from the Verlon model registry on every sync — > see the [live registry](/provider-compatibility/models#live-registry) for > pricing, benchmarks, and deprecation dates. ## Other modalities *No non-chat models.* ## Quickstart > [!NOTE] > > First time? [Create a gate](/platform/gates) and grab an [API > key](/getting-started/authentication) — the examples below need a `GATE_ID` > and `VERLON_API_KEY`. The fastest path — the official Anthropic SDK, pointed at Verlon: ```typescript theme={null} import Anthropic from '@anthropic-ai/sdk'; const anthropic = new Anthropic({ baseURL: 'https://api.verlon.ai', apiKey: process.env.VERLON_API_KEY, }); // @ts-ignore - gateId is a Verlon AI extension const response = await anthropic.messages.create({ gateId: process.env.GATE_ID, model: 'claude-sonnet-4-6', // optional — overrides the gate's configured model max_tokens: 1024, messages: [{ role: 'user', content: 'Hello!' }], }); console.log(response.content[0].text); ``` Any chat model from the table above works — swap `claude-sonnet-4-6` for `claude-opus-4-8`, `claude-haiku-4-5-20251001`, or any other ID. ## SDK compatibility Verlon exposes an Anthropic-compatible endpoint. The official Anthropic SDK works unchanged — swap two lines and every model in the table above becomes callable through the SDK you already know. ### Configuration — just two lines ```typescript Before theme={null} import Anthropic from '@anthropic-ai/sdk'; const anthropic = new Anthropic({ apiKey: process.env.ANTHROPIC_API_KEY, }); ``` ```typescript After (Verlon) theme={null} import Anthropic from '@anthropic-ai/sdk'; const anthropic = new Anthropic({ baseURL: 'https://api.verlon.ai', // ← add this apiKey: process.env.VERLON_API_KEY, // ← change this }); ``` ### Specifying your gate Verlon supports three ways to point a request at a gate: **Using gateId (Recommended)** ```typescript theme={null} // @ts-ignore - gateId is a Verlon AI extension const response = await anthropic.messages.create({ gateId: process.env.GATE_ID, max_tokens: 1024, messages: [{ role: 'user', content: 'Hello!' }], }); ``` > [!NOTE] > > Requires `@ts-ignore` in TypeScript, but the clearest intent. **Using model field** ```typescript theme={null} const response = await anthropic.messages.create({ model: process.env.GATE_ID, // gate UUID in the model slot max_tokens: 1024, messages: [{ role: 'user', content: 'Hello!' }], }); ``` > [!TIP] > > No TypeScript errors, drop-in with existing code. **Using a default header** ```typescript theme={null} const anthropic = new Anthropic({ baseURL: 'https://api.verlon.ai', apiKey: process.env.VERLON_API_KEY, defaultHeaders: { 'X-Verlon-Gate-Id': process.env.GATE_ID }, }); const response = await anthropic.messages.create({ max_tokens: 1024, messages: [{ role: 'user', content: 'Hello!' }], }); ``` > [!TIP] > > Cleanest for TypeScript — set once, use everywhere. ### Streaming Identical behavior to the Anthropic SDK — `stream: true` returns an async iterable of `MessageStreamEvent` objects. ```typescript theme={null} const stream = await anthropic.messages.create({ gateId: process.env.GATE_ID, max_tokens: 1024, messages: [{ role: 'user', content: 'Tell me a joke.' }], stream: true, }); for await (const event of stream) { if ( event.type === 'content_block_delta' && event.delta.type === 'text_delta' ) { process.stdout.write(event.delta.text); } } ``` ### Language coverage ```typescript TypeScript theme={null} import Anthropic from '@anthropic-ai/sdk'; const anthropic = new Anthropic({ baseURL: 'https://api.verlon.ai', apiKey: process.env.VERLON_API_KEY, }); const response = await anthropic.messages.create({ gateId: process.env.GATE_ID, max_tokens: 1024, messages: [{ role: 'user', content: 'Hello!' }], }); console.log(response.content[0].text); ``` ```python Python theme={null} from anthropic import Anthropic client = Anthropic( base_url="https://api.verlon.ai", api_key=os.environ["VERLON_API_KEY"], ) response = client.messages.create( model=os.environ["GATE_ID"], max_tokens=1024, messages=[{"role": "user", "content": "Hello!"}], ) print(response.content[0].text) ``` ```bash cURL theme={null} curl https://api.verlon.ai/v1/messages \ -H "Content-Type: application/json" \ -H "Authorization: Bearer $VERLON_API_KEY" \ -H "anthropic-version: 2023-06-01" \ -d '{ "gateId": "'"$GATE_ID"'", "max_tokens": 1024, "messages": [{"role": "user", "content": "Hello!"}] }' ``` ### What's supported > **Streaming** > > Full support for `stream: true`, identical semantics to Anthropic's SDK. > **Tool use** > > Fully supported. Verlon translates across providers if your gate routes to a > non-Claude model. > **Vision** > > Image inputs in messages work with any vision-capable model. > **System messages + all message types** > > System prompts, user, assistant, and tool\_result blocks all pass through. > **Standard parameters** > > `temperature`, `max_tokens`, `top_p`, `top_k`, `stop_sequences`, and the > rest — all passed through. > **Usage + cost tracking** > > Standard `response.usage` with token counts, plus Verlon's `cost` field with > per-call dollar amount. ## Migrating an existing Anthropic app **1. Update client initialization** Swap `baseURL` + `apiKey`. Two lines. **2. Add a gate reference** Pass `gateId` (or set the default header). `max_tokens`, `messages`, and everything else stay the same. **3. Test in development** Verify requests appear in the Verlon dashboard and cost tracking works. **4. Deploy** Ship. If anything breaks, revert the two-line diff — takes 30 seconds. ## How it works 1. **You send** an Anthropic-format request to `https://api.verlon.ai/v1/messages` 2. **Verlon receives** — validates the gate, applies routing rules 3. **Verlon routes** to whichever model your gate points at (Claude, GPT, Gemini, Mistral, or anything else in the registry) 4. **The provider responds** in its native format 5. **Verlon normalizes** back to Anthropic format 6. **You receive** a standard Anthropic response with an added `cost` field Your code doesn't know the difference. ## Errors Errors come back Anthropic-shaped — `{ "type": "error", "error": { "type", "message" } }` — so the Anthropic SDK's error classes work unchanged. | Status | Meaning | What to check | | - | - | - | | `400` | Invalid request | Malformed body or missing `gateId` | | `401` | Invalid API key | Your `VERLON_API_KEY` value and the `Authorization: Bearer` header | | `404` | Gate not found | Your `GATE_ID` — the error message names the offending ID | | `429` | Rate limit exceeded | Back off and retry; the body includes `reset_at`, and every response carries `X-RateLimit-*` headers | ```typescript theme={null} try { const response = await anthropic.messages.create({ gateId: process.env.GATE_ID, max_tokens: 1024, messages: [{ role: 'user', content: 'Hello!' }], }); } catch (error) { if (error.status === 404) { console.error('Gate not found — check GATE_ID'); } else if (error.status === 429) { console.error('Rate limited — retry after the reset'); } else { throw error; } } ``` ## FAQ > **Do I need to change my request code?** > > No. Change `baseURL` + `apiKey` in the client. The `messages.create` calls > stay identical. > **Can I still call Anthropic directly?** > > Yes. Revert `baseURL` + `apiKey` to Anthropic values and you're back on > their API. > **Can I route a GPT or Gemini model through the Anthropic SDK?** > > Yes — that's the point. Configure your gate to route to any model in the > [registry](/provider-compatibility/models) and your Anthropic SDK code stays > the same. > **Do I need the Verlon SDK?** > > No. This whole guide uses only the Anthropic SDK. ## Full non-chat model list *This provider is chat-only — see the table above.* ## Related * [OpenAI](/provider-compatibility/openai) · [Google](/provider-compatibility/google) · [Mistral](/provider-compatibility/mistral) * [Verlon SDK reference](/sdk-reference/overview) — agent tracing, observability, and the model registry * [Gates](/platform/gates) — set up routing rules and model configuration * [Bring your own key](/platform/byok) — use your own Anthropic account for provider costs * [Anthropic's official docs ↗](https://docs.anthropic.com/en/docs/build-with-claude/overview) # Google Source: https://docs.verlon.ai/provider-compatibility/google Every Gemini and Gemma model Verlon supports — plus quickstart and how to reach them through the SDK you already use. Gemini 1.5 through 3.5 (chat and reasoning), plus Gemma open-weights and the full multimodal lineup — image, video, TTS, audio, embeddings — all routable through Verlon in whichever SDK you already use. No Google-specific SDK required. ## Chat models | Model ID | Type | Context | | - | - | - | | `gemini-2.5-flash` | chat | 1000K | | `gemini-2.5-flash-lite` | chat | 1000K | | `gemini-2.5-pro` | chat | 1000K | | `gemini-3-flash-preview` | chat | 1000K | | `gemini-3.1-flash-lite` | chat | 1000K | | `gemini-3.1-flash-lite-preview` | chat | 1000K | | `gemini-3.1-pro-preview` | chat | 2000K | | `gemini-3.1-pro-preview-customtools` | chat | 2000K | | `gemini-3.5-flash` | chat | 1049K | | `gemini-3.5-flash-lite` | chat | 1000K | | `gemini-3.6-flash` | chat | 1000K | | `gemini-3.7-flash` | chat | 1049K | | `gemini-flash-latest` | chat | 1000K | | `gemini-flash-lite-latest` | chat | 1000K | | `gemini-pro-latest` | chat | 2000K | | `gemini-robotics-er-2-preview` | chat | 32K | | `gemma-4-26b-a4b-it` | chat | 262K | | `gemma-4-31b-it` | chat | 262K | > [!NOTE] > > This table is auto-generated from the Verlon model registry on every sync — > see the [live registry](/provider-compatibility/models#live-registry) for > pricing, benchmarks, and deprecation dates. ## Other modalities * **audio** — 5 models * **embeddings** — 3 models * **image generation** — 7 models * **text-to-speech** — 3 models * **video generation** — 3 models Non-chat Gemini models are called through the OpenAI SDK using the same model IDs — Verlon mirrors `/v1/embeddings`, `/v1/images/generations`, and `/v1/audio/speech` as drop-in routes. Video generation is drop-in too: the `openai` package's async Videos surface (`videos.create()` / `retrieve()` / `downloadContent()`) works against `/v1/videos`. ## Quickstart > [!NOTE] > > First time? [Create a gate](/platform/gates) and grab an [API > key](/getting-started/authentication) — the examples below need a `GATE_ID` > and `VERLON_API_KEY`. The fastest path — the official OpenAI SDK, pointed at Verlon: ```typescript theme={null} import OpenAI from 'openai'; const openai = new OpenAI({ baseURL: 'https://api.verlon.ai/v1', apiKey: process.env.VERLON_API_KEY, }); // @ts-ignore - gateId is a Verlon AI extension const response = await openai.chat.completions.create({ gateId: process.env.GATE_ID, model: 'gemini-2.5-pro', // optional — overrides the gate's configured model messages: [{ role: 'user', content: 'Hello!' }], }); console.log(response.choices[0].message.content); ``` Any chat model from the table above works — swap `gemini-2.5-pro` for `gemini-3.5-flash`, `gemini-flash-latest`, or any other ID. ## SDK compatibility Any SDK with a Verlon drop-in endpoint can reach every model on this page — today that's the [OpenAI SDK](/provider-compatibility/openai) and the [Anthropic SDK](/provider-compatibility/anthropic): point it at Verlon, reference a gate that routes here, and your code stays unchanged. The [quickstart above](#quickstart) is all it takes. Verlon also mirrors Gemini's native API at `/v1beta/models/...`, so tooling that speaks the Gemini REST shape works unchanged too. The [Verlon SDK](/sdk-reference/overview) isn't needed for inference — its home is agent tracing, observability, and the model registry. ### Streaming ```typescript theme={null} const stream = await openai.chat.completions.create({ gateId: process.env.GATE_ID, messages: [{ role: 'user', content: 'Tell me a joke.' }], stream: true, }); for await (const chunk of stream) { process.stdout.write(chunk.choices[0]?.delta?.content ?? ''); } ``` ## What's supported > **Streaming** > > Full support for `stream: true` in every supported SDK — identical semantics > to the source SDK. > **Tool / function calling** > > Fully supported. Verlon translates the tool-call shape between OpenAI / > Anthropic / Gemini native formats. > **Vision** > > Image inputs work with any vision-capable Gemini model — including the > multimodal Gemini 2.5 Flash and Pro lines. > **Standard parameters** > > `temperature`, `max_tokens`, `top_p`, `stop_sequences`, and the rest are > passed through and mapped to Gemini's native fields. > **Usage + cost tracking** > > Every response includes token usage plus Verlon's `cost` field with per-call > dollar amount. ## How it works 1. **You send** a request in your chosen SDK's format 2. **Verlon receives** — validates the gate, applies routing rules 3. **Verlon translates** to Gemini's native format and dispatches to Google's API 4. **Google responds** with the raw Gemini response 5. **Verlon normalizes** back to your SDK's expected shape (OpenAI, Anthropic, or Gemini-native) 6. **You receive** a response in the format your code already handles, with an added `cost` field Your existing code doesn't know it's talking to Gemini. ## Errors | Status | Meaning | What to check | | - | - | - | | `400` | Invalid request | Malformed body or missing `gateId` | | `401` | Invalid API key | Your `VERLON_API_KEY` value and the `Authorization: Bearer` header | | `404` | Gate not found | Your `GATE_ID` — the error message names the offending ID | | `429` | Rate limit exceeded | Back off and retry; the body includes `reset_at`, and every response carries `X-RateLimit-*` headers | The error shape always matches the SDK you're calling with. Through the OpenAI SDK it surfaces as a thrown `APIError`: ```typescript theme={null} try { await openai.chat.completions.create({ gateId: process.env.GATE_ID, messages: [{ role: 'user', content: 'Hello!' }], }); } catch (error) { // error.status and error.message carry the API's answer, // e.g. 404 'Gate with ID "…" not found' console.error(error.status, error.message); } ``` ## Full non-chat model list Every non-chat Google model Verlon routes to, grouped by modality. Chat models are in the table near the top of this page. ### Audio | Model ID | | - | | `gemini-2.5-flash-native-audio-latest` | | `gemini-2.5-flash-native-audio-preview-12-2025` | | `gemini-3.5-live-translate-preview` | | `lyria-3-clip-preview` | | `lyria-3-pro-preview` | ### Embeddings | Model ID | | - | | `gemini-embedding-001` | | `gemini-embedding-2` | | `gemini-embedding-2-preview` | ### Image generation | Model ID | | - | | `gemini-2.5-flash-image` | | `gemini-3-pro-image` | | `gemini-3-pro-image-preview` | | `gemini-3.1-flash-image` | | `gemini-3.1-flash-image-preview` | | `gemini-3.1-flash-lite-image` | | `nano-banana-pro-preview` | ### Text-to-speech | Model ID | | - | | `gemini-2.5-flash-preview-tts` | | `gemini-2.5-pro-preview-tts` | | `gemini-3.1-flash-tts-preview` | ### Video generation | Model ID | | - | | `veo-3.1-fast-generate-preview` | | `veo-3.1-generate-preview` | | `veo-3.1-lite-generate-preview` | ## Related * [OpenAI](/provider-compatibility/openai) · [Anthropic](/provider-compatibility/anthropic) · [Mistral](/provider-compatibility/mistral) * [Verlon SDK reference](/sdk-reference/overview) — agent tracing, observability, and the model registry * [Gates](/platform/gates) — set up routing rules and model configuration * [Bring your own key](/platform/byok) — use your own Google account for provider costs * [Google's Gemini API docs ↗](https://ai.google.dev/gemini-api/docs) # Meta Source: https://docs.verlon.ai/provider-compatibility/meta Every Meta Model API model Verlon supports — the Muse Spark reasoning family — with quickstart, reasoning behaviour, and SDK compatibility. Meta's Model API — the Muse Spark family, built for long, tool-heavy tasks that plan and orchestrate across services — is routable through Verlon in whichever SDK you already use. No Meta-specific SDK required. ## Chat models | Model ID | Type | Context | | - | - | - | | `muse-spark-1.1` | chat | 1049K | | `muse-spark-1.2` | chat | 1049K | | `muse-spark-1.2-contributor` | chat | 1049K | | `muse-spark-1.3` | chat | 1049K | | `muse-spark-1.3-contributor` | chat | 1049K | > [!NOTE] > > This table is auto-generated from the Verlon model registry on every sync — > see the [live registry](/provider-compatibility/models#live-registry) for > pricing, benchmarks, and deprecation dates. ## Quickstart The fastest path — the official OpenAI SDK, pointed at Verlon: ```typescript theme={null} import OpenAI from 'openai'; const openai = new OpenAI({ baseURL: 'https://api.verlon.ai/v1', apiKey: process.env.VERLON_API_KEY, }); // @ts-ignore - gateId is a Verlon AI extension const response = await openai.chat.completions.create({ gateId: process.env.GATE_ID, model: 'muse-spark-1.2', // optional — overrides the gate's configured model messages: [{ role: 'user', content: 'Hello!' }], }); console.log(response.choices[0].message.content); ``` Any chat model from the table above works — swap `muse-spark-1.2` for any other ID. ## Reasoning Muse Spark reasons on every request and **reasoning cannot be turned off**. `reasoning_effort` controls how much of it happens, but even the lowest setting still thinks: ```typescript theme={null} const response = await openai.chat.completions.create({ gateId: process.env.GATE_ID, messages: [ { role: 'user', content: 'Prove there are infinitely many primes.' }, ], reasoning_effort: 'low', // 'low' | 'medium' | 'high' — Meta defaults to 'high' }); ``` This matters more here than on most providers. A trivial prompt like "say hi" spends hundreds of reasoning tokens before it produces a single visible word, and those tokens are billed as output and count against your output cap. Verlon automatically reserves extra headroom on top of your `max_tokens` for reasoning models, so a small `max_tokens` won't be consumed entirely by thinking and leave you with an empty response. `reasoning_effort` is portable: Verlon passes the same value straight through to OpenAI and Meta, maps it to a thinking-token budget on Anthropic and Google, and ignores it for Mistral. Switching models doesn't require changing your code. ## SDK compatibility The official OpenAI SDK against Verlon's drop-in endpoint is the native path to every model on this page — the [quickstart above](#quickstart) is all it takes. The [Verlon SDK](/sdk-reference/overview) isn't needed for inference; its home is agent tracing, observability, and the model registry. Already using Meta's own API? Meta serves OpenAI Chat Completions, OpenAI Responses, and Anthropic Messages, and Verlon mirrors all three — so the swap is a base URL and nothing else: ```typescript theme={null} import OpenAI from 'openai'; const client = new OpenAI({ apiKey: process.env.VERLON_API_KEY, baseURL: 'https://api.verlon.ai/v1', // was https://api.meta.ai/v1 }); const response = await client.chat.completions.create({ model: 'muse-spark-1.2', messages: [{ role: 'user', content: 'Hello!' }], }); ``` The same applies to the [Anthropic SDK](/provider-compatibility/anthropic) against `/v1/messages`. Any SDK with a Verlon drop-in endpoint can reach these models: point it at Verlon, reference a gate that routes here, and your code stays unchanged. ### Streaming ```typescript theme={null} const stream = await openai.chat.completions.create({ gateId: process.env.GATE_ID, messages: [{ role: 'user', content: 'Tell me a joke.' }], stream: true, }); for await (const chunk of stream) { process.stdout.write(chunk.choices[0]?.delta?.content ?? ''); } ``` ## What's supported > **Streaming** > > Full support for `stream: true` in every supported SDK — identical semantics > to the source SDK. > **Tool / function calling** > > Fully supported, including parallel tool calls. Verlon translates the > tool-call shape between OpenAI / Anthropic / Meta native formats. > **Structured output** > > `response_format` with `json_object` or a `json_schema` works the same as on > OpenAI. > **Reasoning effort** > > `reasoning_effort` passes straight through to Meta. See > [Reasoning](#reasoning) above. > **Standard parameters** > > `temperature`, `max_tokens`, and `top_p` are passed through. > **Prompt caching** > > Cached input tokens are billed at a large discount and reported in every > response's usage. No configuration needed. > **Usage + cost tracking** > > Every response includes token usage — visible output and reasoning tokens > separately — plus Verlon's `cost` field with the per-call dollar amount. ### Not available * **Image, video, audio, embeddings, OCR.** The Model API is text-only; Muse Spark is a text and reasoning family. Route those modalities to OpenAI, Google, Anthropic, or Mistral and keep your chat traffic on Meta — that mix is exactly what gates are for. ## How it works 1. **You send** a request in your chosen SDK's format 2. **Verlon receives** — validates the gate, applies routing rules 3. **Verlon translates** to Meta's format and dispatches to the Model API 4. **Meta responds** with its raw response 5. **Verlon normalizes** back to your SDK's expected shape 6. **You receive** a response in the format your code already handles, with an added `cost` field Your existing code doesn't know it's talking to Meta. ## Errors | Status | Meaning | What to check | | - | - | - | | `400` | Invalid request | Malformed body or missing `gateId` | | `401` | Invalid API key | Your `VERLON_API_KEY` value and the `Authorization: Bearer` header | | `404` | Gate not found | Your `GATE_ID` — the error message names the offending ID | | `429` | Rate limit exceeded | Back off and retry; the body includes `reset_at`, and every response carries `X-RateLimit-*` headers | The error shape always matches the SDK you're calling with. Through the OpenAI SDK it surfaces as a thrown `APIError`: ```typescript theme={null} try { await openai.chat.completions.create({ gateId: process.env.GATE_ID, messages: [{ role: 'user', content: 'Hello!' }], }); } catch (error) { // error.status and error.message carry the API's answer, // e.g. 404 'Gate with ID "…" not found' console.error(error.status, error.message); } ``` ## Related * [OpenAI](/provider-compatibility/openai) · [Anthropic](/provider-compatibility/anthropic) · [Google](/provider-compatibility/google) · [Mistral](/provider-compatibility/mistral) · [xAI](/provider-compatibility/xai) * [Verlon SDK reference](/sdk-reference/overview) — agent tracing, observability, and the model registry * [Gates](/platform/gates) — set up routing rules and model configuration # Mistral Source: https://docs.verlon.ai/provider-compatibility/mistral Every Mistral model Verlon supports — Large, Medium, Small, Ministral, Codestral, Devstral, Magistral, Pixtral — with quickstart and SDK compatibility. Mistral's full lineup — Large, Medium, Small, Ministral (edge sizes), Codestral (code), Devstral (dev tools), Magistral (reasoning), Pixtral (multimodal) — is routable through Verlon in whichever SDK you already use. No Mistral-specific SDK required. ## Chat models | Model ID | Type | Context | | - | - | - | | `codestral-2508` | chat | 256K | | `codestral-latest` | chat | 256K | | `devstral-2512` | chat | 262K | | `devstral-latest` | chat | 262K | | `devstral-medium-latest` | chat | 131K | | `devstral-small-latest` | chat | — | | `glm-5-2` | chat | 1049K | | `magistral-medium-latest` | chat | 40K | | `magistral-small-latest` | chat | 131K | | `ministral-14b-2512` | chat | 262K | | `ministral-14b-latest` | chat | 262K | | `ministral-3b-2512` | chat | 131K | | `ministral-3b-latest` | chat | 131K | | `ministral-8b-2512` | chat | 262K | | `ministral-8b-latest` | chat | 262K | | `mistral-code-agent-latest` | chat | 262K | | `mistral-code-fim-latest` | chat | 256K | | `mistral-code-latest` | chat | 256K | | `mistral-large-2512` | chat | 256K | | `mistral-large-4` | chat | 1049K | | `mistral-large-4-0` | chat | 1049K | | `mistral-large-latest` | chat | 256K | | `mistral-medium` | chat | 262K | | `mistral-medium-2505` | chat | 131K | | `mistral-medium-2508` | chat | 131K | | `mistral-medium-2604` | chat | 262K | | `mistral-medium-3` | chat | 32K | | `mistral-medium-3-5` | chat | 256K | | `mistral-medium-3.5` | chat | 256K | | `mistral-medium-latest` | chat | 131K | | `mistral-small-2506` | chat | 128K | | `mistral-small-2603` | chat | 262K | | `mistral-small-latest` | chat | 262K | | `mistral-tiny-2407` | chat | 131K | | `mistral-tiny-latest` | chat | 131K | | `mistral-vibe-cli-fast` | chat | 262K | | `mistral-vibe-cli-latest` | chat | 262K | | `mistral-vibe-cli-with-tools` | chat | 262K | | `open-mistral-nemo` | chat | 128K | | `open-mistral-nemo-2407` | chat | 131K | | `zai-glm-5` | chat | 1049K | | `zai-glm-5-2` | chat | 1049K | | `zai-glm-5-3` | chat | 1000K | | `zai-glm-latest` | chat | 1000K | > [!NOTE] > > This table is auto-generated from the Verlon model registry on every sync — > see the [live registry](/provider-compatibility/models#live-registry) for > pricing, benchmarks, and deprecation dates. ## Other modalities * **audio** — 4 models * **embeddings** — 4 models * **moderation** — 1 models * **OCR / document processing** — 6 models * **speech-to-text** — 3 models * **text-to-speech** — 2 models Non-chat Mistral models are called through the OpenAI SDK using the same model IDs — Verlon mirrors `/v1/embeddings`, `/v1/images/generations`, and `/v1/audio/speech` as drop-in routes. Mistral's OCR line is particularly strong for document processing; `mistral.ocr.process()` works drop-in against `/mistral/v1/ocr` — the same serverURL swap as chat and FIM. ## Quickstart > [!NOTE] > > First time? [Create a gate](/platform/gates) and grab an [API > key](/getting-started/authentication) — the examples below need a `GATE_ID` > and `VERLON_API_KEY`. The fastest path — the official OpenAI SDK, pointed at Verlon: ```typescript theme={null} import OpenAI from 'openai'; const openai = new OpenAI({ baseURL: 'https://api.verlon.ai/v1', apiKey: process.env.VERLON_API_KEY, }); // @ts-ignore - gateId is a Verlon AI extension const response = await openai.chat.completions.create({ gateId: process.env.GATE_ID, model: 'mistral-large-latest', // optional — overrides the gate's configured model messages: [{ role: 'user', content: 'Hello!' }], }); console.log(response.choices[0].message.content); ``` Any chat model from the table above works — swap `mistral-large-latest` for `codestral-latest`, `magistral-medium-latest`, `pixtral-large-latest`, or any other ID. ## SDK compatibility Any SDK with a Verlon drop-in endpoint can reach every model on this page — today that's the [OpenAI SDK](/provider-compatibility/openai) and the [Anthropic SDK](/provider-compatibility/anthropic): point it at Verlon, reference a gate that routes here, and your code stays unchanged. The [quickstart above](#quickstart) is all it takes. Verlon also mirrors Mistral's own API shape at `/mistral/v1/chat/completions` and `/mistral/v1/fim/completions` for clients that already speak it. The [Verlon SDK](/sdk-reference/overview) isn't needed for inference — its home is agent tracing, observability, and the model registry. ### Streaming ```typescript theme={null} const stream = await openai.chat.completions.create({ gateId: process.env.GATE_ID, messages: [{ role: 'user', content: 'Tell me a joke.' }], stream: true, }); for await (const chunk of stream) { process.stdout.write(chunk.choices[0]?.delta?.content ?? ''); } ``` ## What's supported > **Streaming** > > Full support for `stream: true` in every supported SDK — identical semantics > to the source SDK. > **Tool / function calling** > > Fully supported. Verlon translates the tool-call shape between OpenAI / > Anthropic / Mistral native formats. > **Vision** > > Image inputs work with Pixtral models — Mistral's vision-capable line. > **Code + reasoning specializations** > > Codestral, Devstral, and Magistral models are available with the same call > shape — swap the model ID. > **Standard parameters** > > `temperature`, `max_tokens`, `top_p`, `stop`, and the rest are passed > through. > **Usage + cost tracking** > > Every response includes token usage plus Verlon's `cost` field with per-call > dollar amount. ## How it works 1. **You send** a request in your chosen SDK's format 2. **Verlon receives** — validates the gate, applies routing rules 3. **Verlon translates** to Mistral's native format and dispatches to Mistral's API 4. **Mistral responds** with its raw response 5. **Verlon normalizes** back to your SDK's expected shape 6. **You receive** a response in the format your code already handles, with an added `cost` field Your existing code doesn't know it's talking to Mistral. ## Errors | Status | Meaning | What to check | | - | - | - | | `400` | Invalid request | Malformed body or missing `gateId` | | `401` | Invalid API key | Your `VERLON_API_KEY` value and the `Authorization: Bearer` header | | `404` | Gate not found | Your `GATE_ID` — the error message names the offending ID | | `429` | Rate limit exceeded | Back off and retry; the body includes `reset_at`, and every response carries `X-RateLimit-*` headers | The error shape always matches the SDK you're calling with. Through the OpenAI SDK it surfaces as a thrown `APIError`: ```typescript theme={null} try { await openai.chat.completions.create({ gateId: process.env.GATE_ID, messages: [{ role: 'user', content: 'Hello!' }], }); } catch (error) { // error.status and error.message carry the API's answer, // e.g. 404 'Gate with ID "…" not found' console.error(error.status, error.message); } ``` ## Full non-chat model list Every non-chat Mistral model Verlon routes to, grouped by modality. Chat models are in the table near the top of this page. ### Audio | Model ID | | - | | `voxtral-mini-2602` | | `voxtral-mini-realtime-2602` | | `voxtral-mini-realtime-latest` | | `voxtral-mini-transcribe-realtime-2602` | ### Embeddings | Model ID | | - | | `codestral-embed` | | `codestral-embed-2505` | | `mistral-embed` | | `mistral-embed-2312` | ### Moderation | Model ID | | - | | `mistral-moderation-2603` | ### OCR / document processing | Model ID | | - | | `mistral-ocr-2512` | | `mistral-ocr-3` | | `mistral-ocr-3-0` | | `mistral-ocr-4` | | `mistral-ocr-4-1` | | `mistral-ocr-latest` | ### Speech-to-text | Model ID | | - | | `voxtral-mini-latest` | | `voxtral-small-2507` | | `voxtral-small-latest` | ### Text-to-speech | Model ID | | - | | `voxtral-mini-tts-2603` | | `voxtral-mini-tts-latest` | ## Related * [OpenAI](/provider-compatibility/openai) · [Anthropic](/provider-compatibility/anthropic) · [Google](/provider-compatibility/google) * [Verlon SDK reference](/sdk-reference/overview) — agent tracing, observability, and the model registry * [Gates](/platform/gates) — set up routing rules and model configuration * [Bring your own key](/platform/byok) — use your own Mistral account for provider costs * [Mistral's official docs ↗](https://docs.mistral.ai/) # Supported LLMs Source: https://docs.verlon.ai/provider-compatibility/models The models Verlon AI routes across today — grouped by provider, with links to per-provider references. Verlon AI ships adapters for four providers and gives you access to ** 250 ** language models through a single API. Pick a provider below for its full model list, quickstart, and SDK compatibility notes — or hit the live registry endpoint for machine-readable data (pricing, capabilities, deprecation status). > **[OpenAI](/provider-compatibility/openai)** > > Frontier chat and reasoning across the GPT-4o, GPT-5, and o-series lineups — > plus image, audio, TTS, STT, moderation, and embeddings. > **[Anthropic](/provider-compatibility/anthropic)** > > The full Claude family — Opus, Sonnet, and Haiku across the 3.5, 4, 4.5, > 4.6, 4.7, and 4.8 generations. > **[Google](/provider-compatibility/google)** > > Gemini 1.5 through 3.5 (chat and reasoning), plus Gemma open-weights and the > full multimodal lineup — image, video, TTS, audio, embeddings. > **[Mistral](/provider-compatibility/mistral)** > > Large, Medium, Small, Ministral, Codestral, Devstral, Magistral, and Pixtral > — plus embeddings, moderation, OCR, STT, and TTS. > **[xAI](/provider-compatibility/xai)** > > The Grok line — 4.5, 4.3, the 4.20 family, and Grok Build for code — plus > Grok Imagine for image and video. > **[Meta](/provider-compatibility/meta)** > > The Muse Spark family on Meta's Model API — reasoning models built for long, > tool-heavy tasks. ## Live registry For the machine-readable list with per-model pricing, capabilities, context windows, and deprecation status: ``` GET https://api.verlon.ai/v1/registry ``` Public endpoint, no authentication required. Cached for one hour with an ETag; safe to poll from clients and agents. Every Verlon SDK validates `model` values against this registry, and the dashboard's model picker renders directly from it. > [!NOTE] > > Missing a model or provider? Ping us at > [micah@verlon.ai](mailto:micah@verlon.ai) — most additions ship inside a week. # OpenAI Source: https://docs.verlon.ai/provider-compatibility/openai Every OpenAI model Verlon supports — plus quickstart and SDK compatibility (use the OpenAI SDK unchanged against Verlon). Every OpenAI model — GPT-3.5, GPT-4, GPT-4o, GPT-5, and the o-series reasoning models — is routable through Verlon with your existing OpenAI SDK unchanged. Just point `baseURL` at Verlon and swap the API key. ## Chat models | Model ID | Type | Context | | - | - | - | | `chat-latest` | chat | 1000K | | `gpt-3.5-turbo` | chat | 16K | | `gpt-3.5-turbo-0125` | chat | 16K | | `gpt-3.5-turbo-16k` | chat | 16K | | `gpt-4` | chat | 8K | | `gpt-4-0613` | chat | 8K | | `gpt-4-turbo` | chat | 128K | | `gpt-4-turbo-2024-04-09` | chat | 128K | | `gpt-4.1` | chat | 1048K | | `gpt-4.1-2025-04-14` | chat | 1048K | | `gpt-4.1-mini` | chat | 1048K | | `gpt-4.1-mini-2025-04-14` | chat | 1048K | | `gpt-4.1-nano` | chat | 1048K | | `gpt-4.1-nano-2025-04-14` | chat | 1048K | | `gpt-4o` | chat | 128K | | `gpt-4o-2024-05-13` | chat | 128K | | `gpt-4o-2024-08-06` | chat | 128K | | `gpt-4o-2024-11-20` | chat | 128K | | `gpt-4o-mini` | chat | 128K | | `gpt-4o-mini-2024-07-18` | chat | 128K | | `gpt-5-nano` | chat | 128K | | `gpt-5-nano-2025-08-07` | chat | 128K | | `gpt-5-search-api` | chat | 128K | | `gpt-5-search-api-2025-10-14` | chat | 128K | | `gpt-5.1` | chat | 128K | | `gpt-5.1-2025-11-13` | chat | 128K | | `gpt-5.2` | chat | 128K | | `gpt-5.2-2025-12-11` | chat | 128K | | `gpt-5.4` | chat | 128K | | `gpt-5.4-2026-03-05` | chat | 128K | | `gpt-5.4-mini` | chat | 128K | | `gpt-5.4-mini-2026-03-17` | chat | 128K | | `gpt-5.4-nano` | chat | 128K | | `gpt-5.4-nano-2026-03-17` | chat | 128K | | `gpt-5.5` | chat | 1000K | | `gpt-5.5-2026-04-23` | chat | 1000K | | `gpt-5.6-luna` | chat | 1049K | | `gpt-5.6-sol` | chat | 1049K | | `gpt-5.6-terra` | chat | 1049K | | `gpt-6-astra` | chat | 1050K | | `gpt-6-luna` | chat | 1050K | | `gpt-6-sol` | chat | 1050K | | `gpt-6.1-sol` | chat | 1050K | | `o1` | chat | 200K | | `o1-2024-12-17` | chat | 200K | | `o3` | chat | 200K | | `o3-2025-04-16` | chat | 200K | | `o3-mini` | chat | 200K | | `o3-mini-2025-01-31` | chat | 200K | | `o4-mini` | chat | 200K | | `o4-mini-2025-04-16` | chat | 200K | > [!NOTE] > > This table is auto-generated from the Verlon model registry on every sync — > see the [live registry](/provider-compatibility/models#live-registry) for > pricing, benchmarks, and deprecation dates. Any model listed here is available > today. ## Other modalities * **audio** — 9 models * **embeddings** — 3 models * **image generation** — 9 models * **moderation** — 2 models * **speech-to-text** — 7 models * **text-to-speech** — 7 models * **video generation** — 2 models Non-chat OpenAI models are called through the same OpenAI SDK using the same model IDs — Verlon mirrors `/v1/embeddings`, `/v1/images/generations`, and `/v1/audio/speech` as drop-in routes. Video generation is drop-in too: the `openai` package's async Videos surface (`videos.create()` / `retrieve()` / `downloadContent()`) works against `/v1/videos`. ## Quickstart > [!NOTE] > > First time? [Create a gate](/platform/gates) and grab an [API > key](/getting-started/authentication) — the examples below need a `GATE_ID` > and `VERLON_API_KEY`. The fastest path — the official OpenAI SDK, pointed at Verlon: ```typescript theme={null} import OpenAI from 'openai'; const openai = new OpenAI({ baseURL: 'https://api.verlon.ai/v1', apiKey: process.env.VERLON_API_KEY, }); // @ts-ignore - gateId is a Verlon AI extension const response = await openai.chat.completions.create({ gateId: process.env.GATE_ID, model: 'gpt-4o', // optional — overrides the gate's configured model messages: [{ role: 'user', content: 'Hello!' }], }); console.log(response.choices[0].message.content); ``` Any chat model from the table above works — swap `gpt-4o` for `gpt-5.4`, `o4-mini`, `o1-preview`, or any other ID. ## SDK compatibility Verlon exposes an OpenAI-compatible endpoint. The official OpenAI SDK works unchanged — you swap two lines and every model in the table above becomes callable via the SDK you already know. ### Configuration — just two lines ```typescript Before theme={null} import OpenAI from 'openai'; const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY, }); ``` ```typescript After (Verlon) theme={null} import OpenAI from 'openai'; const openai = new OpenAI({ baseURL: 'https://api.verlon.ai/v1', // ← add this apiKey: process.env.VERLON_API_KEY, // ← change this }); ``` ### Specifying your gate Verlon supports three ways to point a request at a gate: **Using gateId (Recommended)** ```typescript theme={null} // @ts-ignore - gateId is a Verlon AI extension const response = await openai.chat.completions.create({ gateId: process.env.GATE_ID, messages: [{ role: 'user', content: 'Hello!' }], }); ``` > [!NOTE] > > Requires `@ts-ignore` in TypeScript, but the clearest intent. **Using model field** ```typescript theme={null} const response = await openai.chat.completions.create({ model: process.env.GATE_ID, // gate UUID in the model slot messages: [{ role: 'user', content: 'Hello!' }], }); ``` > [!TIP] > > No TypeScript errors, drop-in with existing code. **Using a default header** ```typescript theme={null} const openai = new OpenAI({ baseURL: 'https://api.verlon.ai/v1', apiKey: process.env.VERLON_API_KEY, defaultHeaders: { 'X-Verlon-Gate-Id': process.env.GATE_ID }, }); // Every subsequent call inherits the gate const response = await openai.chat.completions.create({ messages: [{ role: 'user', content: 'Hello!' }], }); ``` > [!TIP] > > Cleanest for TypeScript — set once, use everywhere. ### Streaming Identical behavior to the OpenAI SDK — `stream: true` returns an async iterable. ```typescript theme={null} const stream = await openai.chat.completions.create({ gateId: process.env.GATE_ID, messages: [{ role: 'user', content: 'Tell me a joke.' }], stream: true, }); for await (const chunk of stream) { process.stdout.write(chunk.choices[0]?.delta?.content ?? ''); } ``` ### Language coverage ```typescript TypeScript theme={null} import OpenAI from 'openai'; const openai = new OpenAI({ baseURL: 'https://api.verlon.ai/v1', apiKey: process.env.VERLON_API_KEY, }); const response = await openai.chat.completions.create({ gateId: process.env.GATE_ID, messages: [{ role: 'user', content: 'Hello!' }], }); console.log(response.choices[0].message.content); ``` ```python Python theme={null} from openai import OpenAI client = OpenAI( base_url="https://api.verlon.ai/v1", api_key=os.environ["VERLON_API_KEY"], ) response = client.chat.completions.create( model=os.environ["GATE_ID"], messages=[{"role": "user", "content": "Hello!"}], ) print(response.choices[0].message.content) ``` ```bash cURL theme={null} curl https://api.verlon.ai/v1/chat/completions \ -H "Content-Type: application/json" \ -H "Authorization: Bearer $VERLON_API_KEY" \ -d '{ "gateId": "'"$GATE_ID"'", "messages": [{"role": "user", "content": "Hello!"}] }' ``` ### What's supported > **Streaming** > > Full support for `stream: true`, identical semantics to OpenAI. > **Tool / function calling** > > Fully supported. Verlon translates across providers if your gate routes to > Claude, Gemini, etc. > **Vision** > > Image inputs (`type: 'image_url'`) work with any vision-capable model. > **Standard parameters** > > `temperature`, `max_tokens`, `top_p`, `frequency_penalty`, > `presence_penalty`, `logprobs`, and the rest — all passed through. > **Usage + cost tracking** > > Standard `response.usage` with token counts, plus Verlon's `cost` field with > per-call dollar amount. ## Responses API Verlon is also a drop-in for OpenAI's **Responses API** (`client.responses.create`) — the surface the OpenAI Agents SDK and multi-turn tool loops with encrypted reasoning replay are built on. Point `baseURL` at Verlon and pass your gate id as `model`, exactly like chat completions. ```python theme={null} from openai import OpenAI client = OpenAI(base_url="https://api.verlon.ai/v1", api_key="sk-vrln-...") resp = client.responses.create( model="", input=[{"role": "user", "content": "What's the weather in Tokyo?"}], tools=[weather_tool], ) ``` > [!NOTE] > > **Multi-turn tool loops with reasoning replay work.** On each turn, echo the > prior `reasoning` and `function_call` items back in `input` — Verlon requests > `include: ['reasoning.encrypted_content']` for reasoning models and passes the > encrypted reasoning through untouched, so the model keeps its chain of thought > across tool calls. > **Stateless only (v1)** > > `store: true` and `previous_response_id` are **not** supported — Verlon > holds no server-side conversation state (matching our zero-data-retention > default). Send the full `input` each turn, echoing prior `reasoning` / > `function_call` items back. Requests using `store: true` or > `previous_response_id` return `400`. > **OpenAI-backed gates only** > > The Responses API is OpenAI-specific. A gate that resolves to Anthropic, > Google, or Mistral returns `400` — use `/v1/chat/completions` for those, > which normalizes tool calls across providers. > **Streaming** > > `stream: true` returns the named-event SSE stream (`event: > response.created`, `response.output_text.delta`, `response.completed`, …) > verbatim. ## Migrating an existing OpenAI app **1. Update client initialization** Swap `baseURL` + `apiKey`. Two lines. **2. Add a gate reference** Pass `gateId` (or set the default header). The `model` field, `messages`, and everything else stay the same. **3. Test in development** Verify requests appear in the Verlon dashboard and cost tracking works. **4. Deploy** Ship. If anything breaks, revert the two-line diff — takes 30 seconds. ## How it works 1. **You send** an OpenAI-format request to `https://api.verlon.ai/v1/chat/completions` 2. **Verlon receives** — validates the gate, applies routing rules 3. **Verlon routes** to whichever model your gate points at (GPT, Claude, Gemini, Mistral, or anything else in the registry) 4. **The provider responds** in its native format 5. **Verlon normalizes** back to OpenAI format 6. **You receive** a standard OpenAI response with an added `cost` field Your code doesn't know the difference. ## Errors Errors come back OpenAI-shaped — `{ "error": { "message", "type", "code", "param" } }` — so the OpenAI SDK's error classes work unchanged. | Status | Meaning | What to check | | - | - | - | | `400` | Invalid request | Malformed body or missing `gateId` | | `401` | Invalid API key | Your `VERLON_API_KEY` value and the `Authorization: Bearer` header | | `404` | Gate not found | Your `GATE_ID` — the error message names the offending ID | | `429` | Rate limit exceeded | Back off and retry; the body includes `reset_at`, and every response carries `X-RateLimit-*` headers | ```typescript theme={null} try { const response = await openai.chat.completions.create({ gateId: process.env.GATE_ID, messages: [{ role: 'user', content: 'Hello!' }], }); } catch (error) { if (error.status === 404) { console.error('Gate not found — check GATE_ID'); } else if (error.status === 429) { console.error('Rate limited — retry after the reset'); } else { throw error; } } ``` ## FAQ > **Do I need to change my request code?** > > No. Change `baseURL` + `apiKey` in the client. The completion calls stay > identical. > **Can I still call OpenAI directly?** > > Yes. Revert `baseURL` + `apiKey` to OpenAI values and you're back on their > API. > **Can I route a Claude or Gemini model through the OpenAI SDK?** > > Yes — that's the point. Configure your gate to route to any model in the > [registry](/provider-compatibility/models) and your OpenAI SDK code stays > the same. > **Does the Vercel AI SDK work?** > > Yes. Its OpenAI adapter works against Verlon's endpoint unchanged. > **Do I need the Verlon SDK?** > > No. This whole guide uses only the OpenAI SDK. ## Full non-chat model list Every non-chat OpenAI model Verlon routes to, grouped by modality. Chat models are in the table near the top of this page. ### Audio | Model ID | | - | | `gpt-4o-mini-transcribe` | | `gpt-4o-mini-transcribe-2025-03-20` | | `gpt-audio` | | `gpt-audio-1.5` | | `gpt-audio-2025-08-28` | | `gpt-audio-mini` | | `gpt-audio-mini-2025-10-06` | | `gpt-audio-mini-2025-12-15` | | `gpt-realtime-translate` | ### Embeddings | Model ID | | - | | `text-embedding-3-large` | | `text-embedding-3-small` | | `text-embedding-ada-002` | ### Image generation | Model ID | | - | | `chatgpt-image-latest` | | `gpt-image-1-mini` | | `gpt-image-1.5` | | `gpt-image-2` | | `gpt-image-2-2026-04-21` | | `gpt-image-2.5-flare` | | `gpt-image-2.5-flare-2026-09-08` | | `gpt-image-2.5-sunburst` | | `gpt-image-2.5-sunburst-2026-09-08` | ### Moderation | Model ID | | - | | `omni-moderation-2024-09-26` | | `omni-moderation-latest` | ### Speech-to-text | Model ID | | - | | `gpt-4o-mini-transcribe-2025-12-15` | | `gpt-4o-transcribe` | | `gpt-4o-transcribe-diarize` | | `gpt-live-transcribe` | | `gpt-realtime-whisper` | | `gpt-transcribe` | | `whisper-1` | ### Text-to-speech | Model ID | | - | | `gpt-4o-mini-tts` | | `gpt-4o-mini-tts-2025-03-20` | | `gpt-4o-mini-tts-2025-12-15` | | `tts-1` | | `tts-1-1106` | | `tts-1-hd` | | `tts-1-hd-1106` | ### Video generation | Model ID | | - | | `sora-2` | | `sora-2-pro` | ## Related * [Anthropic](/provider-compatibility/anthropic) · [Google](/provider-compatibility/google) · [Mistral](/provider-compatibility/mistral) * [Verlon SDK reference](/sdk-reference/overview) — agent tracing, observability, and the model registry * [Gates](/platform/gates) — set up routing rules and model configuration * [Bring your own key](/platform/byok) — use your own OpenAI account for provider costs * [OpenAI's official docs ↗](https://platform.openai.com/docs) # xAI Source: https://docs.verlon.ai/provider-compatibility/xai Every xAI model Verlon supports — Grok 4.5, Grok 4.3, the Grok 4.20 line, Grok Build, and Grok Imagine — with quickstart, long-context pricing, and SDK compatibility. xAI's Grok lineup — Grok 4.5 (flagship reasoning), Grok 4.3 and the Grok 4.20 line (1M context), Grok Build (code, also known as `grok-code-fast-1`), and Grok Imagine (image and video) — is routable through Verlon in whichever SDK you already use. No xAI-specific SDK required. ## Chat models | Model ID | Type | Context | | - | - | - | | `grok-4.20-0309-non-reasoning` | chat | 1000K | | `grok-4.20-0309-reasoning` | chat | 1000K | | `grok-4.20-multi-agent-0309` | chat | 1000K | | `grok-4.3` | chat | 1000K | | `grok-4.5` | chat | 500K | | `grok-4.6` | chat | 500K | | `grok-4.7` | chat | 500K | | `grok-build-0.1` | chat | 256K | > [!NOTE] > > This table is auto-generated from the Verlon model registry on every sync — > see the [live registry](/provider-compatibility/models#live-registry) for > pricing, benchmarks, and deprecation dates. xAI also publishes aliases that track the newest release of a line — `grok-4.5-latest`, `grok-latest`, `grok-code-fast-1`. Those resolve on xAI's side, so you can use them here too; pin a dated ID when you need version stability. ## Long-context pricing Grok is the first provider on Verlon that **re-prices an entire request** once its prompt gets large. Every current Grok text model doubles its rates — input, cached input, and output alike — when the prompt reaches **200,000 tokens**: | Model | Standard (\< 200K prompt) | Long context (≥ 200K prompt) | | - | - | - | | `grok-4.5` | $2.00 in · $0.30 cached · \$6.00 out | $4.00 in · $0.60 cached · \$12.00 out | | `grok-4.3`, `grok-4.20-*` | $1.25 in · $0.20 cached · \$2.50 out | $2.50 in · $0.40 cached · \$5.00 out | | `grok-build-0.1` | $1.00 in · $0.20 cached · \$2.00 out | $2.00 in · $0.40 cached · \$4.00 out | Three things worth knowing: * **The trigger is prompt size only.** A long *response* never moves you into the higher tier. * **It applies to the whole request**, not just the tokens past the threshold. A 201K-token prompt bills every one of those tokens at the doubled rate. * **Verlon bills this correctly for you.** The `cost` field on every response already reflects whichever tier applied, so you don't need to compute it yourself. If you're near the boundary and cost-sensitive, trimming a prompt from 205K to 195K tokens roughly halves the bill for that call. ## Quickstart > [!NOTE] > > First time? [Create a gate](/platform/gates) and grab an [API > key](/getting-started/authentication) — the examples below need a `GATE_ID` > and `VERLON_API_KEY`. The fastest path — the official OpenAI SDK, pointed at Verlon: ```typescript theme={null} import OpenAI from 'openai'; const openai = new OpenAI({ baseURL: 'https://api.verlon.ai/v1', apiKey: process.env.VERLON_API_KEY, }); // @ts-ignore - gateId is a Verlon AI extension const response = await openai.chat.completions.create({ gateId: process.env.GATE_ID, model: 'grok-4.5', // optional — overrides the gate's configured model messages: [{ role: 'user', content: 'Hello!' }], }); console.log(response.choices[0].message.content); ``` Any chat model from the table above works — swap `grok-4.5` for `grok-4.3`, `grok-build-0.1`, or any other ID. ## Reasoning Grok reasons by default and **reasoning cannot be turned off**. You can control how much of it happens with `reasoning_effort`: ```typescript theme={null} const response = await openai.chat.completions.create({ gateId: process.env.GATE_ID, messages: [ { role: 'user', content: 'Prove there are infinitely many primes.' }, ], reasoning_effort: 'high', // 'low' | 'medium' | 'high' — xAI defaults to 'high' }); ``` Because reasoning tokens are billed as output and count against your output cap, Verlon automatically reserves extra headroom on top of your `max_tokens` for reasoning models — so a small `max_tokens` won't get consumed entirely by thinking, leaving no room for the answer. `reasoning_effort` is portable: Verlon passes the same value straight through to OpenAI, maps it to a thinking-token budget on Anthropic and Google, and ignores it for Mistral. Switching models doesn't require changing your code. ## SDK compatibility Any SDK with a Verlon drop-in endpoint can reach every model on this page — today that's the [OpenAI SDK](/provider-compatibility/openai) and the [Anthropic SDK](/provider-compatibility/anthropic): point it at Verlon, reference a gate that routes here, and your code stays unchanged. The [quickstart above](#quickstart) is all it takes. The [Verlon SDK](/sdk-reference/overview) isn't needed for inference — its home is agent tracing, observability, and the model registry. ### Streaming ```typescript theme={null} const stream = await openai.chat.completions.create({ gateId: process.env.GATE_ID, messages: [{ role: 'user', content: 'Tell me a joke.' }], stream: true, }); for await (const chunk of stream) { process.stdout.write(chunk.choices[0]?.delta?.content ?? ''); } ``` ## What's supported > **Streaming** > > Full support for `stream: true` in every supported SDK — identical semantics > to the source SDK. > **Tool / function calling** > > Fully supported. Verlon translates the tool-call shape between OpenAI / > Anthropic / xAI native formats. > **Vision** > > Every Grok chat model accepts image input alongside text. Max 20MiB per > image, JPEG or PNG, no limit on image count. > **Structured output** > > `response_format` with `json_object` or a `json_schema` works the same as on > OpenAI. > **Reasoning effort** > > `reasoning_effort` maps to xAI's `reasoning.effort`. See > [Reasoning](#reasoning) above. > **Standard parameters** > > `temperature`, `max_tokens`, `top_p`, and `stop` are passed through. Note > that `frequency_penalty` and `presence_penalty` are rejected outright by > Grok's reasoning models — Verlon strips them for you rather than letting the > request fail. > **Image + video generation** > > Grok Imagine image models are available through the drop-in > `/v1/images/generations` route. Video generation is served through the > drop-in `/v1/videos` async surface (`openai.videos.create()` → poll → > download). See [Other modalities](#other-modalities). > **Usage + cost tracking** > > Every response includes token usage plus Verlon's `cost` field with per-call > dollar amount, already reflecting the correct [pricing > tier](#long-context-pricing). ### Not available * **Embeddings.** xAI publishes no embeddings model. Use OpenAI, Google, or Mistral embeddings and route your chat traffic to Grok — that mix is exactly what gates are for. * **Text-to-speech.** xAI's voice models are realtime/websocket-only and aren't reachable through the standard TTS surface. * **OCR.** No xAI equivalent; Mistral's OCR line is the recommended path. ## Other modalities * **image generation** — 3 models * **video generation** — 3 models ```typescript theme={null} const image = await openai.images.generate({ gateId: process.env.GATE_ID, model: 'grok-imagine-image', prompt: 'A single red maple leaf on white', n: 1, }); ``` Image generation is billed per image ($0.02 for `grok-imagine-image`, $0.05 for `grok-imagine-image-quality`); video is billed per second of output. Video generation runs through the drop-in `/v1/videos` async surface — create a job with `openai.videos.create()`, poll it, download the result. ## How it works 1. **You send** a request in your chosen SDK's format 2. **Verlon receives** — validates the gate, applies routing rules 3. **Verlon translates** to xAI's format and dispatches to xAI's API 4. **xAI responds** with its raw response 5. **Verlon normalizes** back to your SDK's expected shape 6. **You receive** a response in the format your code already handles, with an added `cost` field Your existing code doesn't know it's talking to xAI. ## Errors | Status | Meaning | What to check | | - | - | - | | `400` | Invalid request | Malformed body or missing `gateId` | | `401` | Invalid API key | Your `VERLON_API_KEY` value and the `Authorization: Bearer` header | | `404` | Gate not found | Your `GATE_ID` — the error message names the offending ID | | `429` | Rate limit exceeded | Back off and retry; the body includes `reset_at`, and every response carries `X-RateLimit-*` headers | The error shape always matches the SDK you're calling with. Through the OpenAI SDK it surfaces as a thrown `APIError`: ```typescript theme={null} try { await openai.chat.completions.create({ gateId: process.env.GATE_ID, messages: [{ role: 'user', content: 'Hello!' }], }); } catch (error) { // error.status and error.message carry the API's answer, // e.g. 404 'Gate with ID "…" not found' console.error(error.status, error.message); } ``` ## Full non-chat model list Every non-chat xAI model Verlon routes to, grouped by modality. Chat models are in the table near the top of this page. ### Image generation | Model ID | | - | | `grok-imagine-image` | | `grok-imagine-image-2.0` | | `grok-imagine-image-quality` | ### Video generation | Model ID | | - | | `grok-imagine-video` | | `grok-imagine-video-1.5` | | `grok-imagine-video-1.5-lite` | ## Related * [OpenAI](/provider-compatibility/openai) · [Anthropic](/provider-compatibility/anthropic) · [Google](/provider-compatibility/google) · [Mistral](/provider-compatibility/mistral) * [Verlon SDK reference](/sdk-reference/overview) — agent tracing, observability, and the model registry * [Gates](/platform/gates) — set up routing rules and model configuration * [Bring your own key](/platform/byok) — use your own xAI account for provider costs * [xAI's official docs ↗](https://docs.x.ai/) # Overview Source: https://docs.verlon.ai/sdk-reference/overview The Verlon SDK for TypeScript/JavaScript: agent tracing, task attribution, and the live model registry The Verlon SDK is the observability and control-plane companion to the Verlon gateway. It gives your agents traces, tasks, and tool spans on the dashboard timeline, bridges the official provider SDKs into that tracing, and exposes the live model registry. Inference itself no longer goes through this SDK. Chat, images, embeddings, and speech run through the official provider SDKs (`openai`, `@anthropic-ai/sdk`) pointed at Verlon's drop-in gateway routes — see the [OpenAI SDK integration guide](/integrations/openai-sdk) and the [Anthropic SDK integration guide](/integrations/anthropic-sdk). The Verlon SDK's job is to make those calls observable and attributable, not to make them. ## Installation ```bash npm theme={null} npm install @verlon-ai/sdk ``` ```bash pnpm theme={null} pnpm add @verlon-ai/sdk ``` ```bash yarn theme={null} yarn add @verlon-ai/sdk ``` ## Get Your API Key 1. Sign up at [verlon.ai/signup](https://verlon.ai/signup) 2. Get your API key from the [Dashboard](https://verlon.ai/dashboard) 3. Store it in an environment variable ```bash .env theme={null} VERLON_API_KEY=your-api-key-here ``` > [!WARNING] > > Never commit API keys to version control. Use environment variables. ## Quickstart: Trace an Agent Create a client, get a handle for your agent gate, declare the tasks it performs, and open a trace scope. Any LLM call made inside the scope with `task.clientOptions()` carries gate and task attribution and lands on the trace timeline automatically. ```typescript theme={null} import { Verlon } from '@verlon-ai/sdk'; import OpenAI from 'openai'; const verlon = new Verlon({ apiKey: process.env.VERLON_API_KEY }); const support = verlon.agent(process.env.SUPPORT_GATE_ID); const classify = support.task('classify'); await support.trace({ conversationId: chatId }, async () => { const openai = new OpenAI({ apiKey: process.env.VERLON_API_KEY, ...classify.clientOptions('openai') }); const completion = await openai.chat.completions.create({ model: 'gpt-4o', messages: [{ role: 'user', content: ticketText }] }); console.log(completion.choices[0].message.content); }); ``` `clientOptions('openai' | 'anthropic')` returns `{ baseURL, fetch, defaultHeaders }` to spread into the official SDK's constructor: * `baseURL` routes the call through the Verlon gateway (the provider hint sets the right base path for each SDK). * `defaultHeaders` carries the gate and task identity. * `fetch` is an instrumented wrapper that injects the active trace context per call, so a client constructed once never pins a stale trace. Use `agent.clientOptions(...)` instead of a task's when a call belongs to the agent but no specific task. Everything the trace records — spans, timing, cost, model choice — appears on the gate's timeline in the dashboard. See [Agent Gates](/platform/agent-gates) for the platform side. ### Other HTTP clients For an HTTP client that accepts a custom `fetch`, `verlon.instrumentFetch()` returns the same instrumented wrapper on its own. For clients that don't, `ambientHeaders()` (exported from the package root) returns the active trace context as plain headers — call it per request, inside the trace scope: ```typescript theme={null} import { ambientHeaders } from '@verlon-ai/sdk'; const response = await got.post(url, { headers: { ...ambientHeaders(), 'x-verlon-gate-id': gateId } }); ``` ## Tool Spans Wrap a function with `verlon.tool()` and each execution becomes a tool span on the active trace, with arguments, timing, and errors recorded. Outside a trace scope the wrapper is a pure pass-through; errors are rethrown unchanged. ```typescript theme={null} const searchDocs = verlon.tool('search_docs', async (query: string) => { return index.search(query); }); await support.trace({ conversationId: chatId }, async () => { const results = await searchDocs('refund policy'); // shows up as a tool span on the trace timeline }); ``` ## Model Registry `verlon.models()` fetches the platform's current model registry — models, capabilities, and pricing. It's unauthenticated (the registry is public), so it's safe to call before a key is configured. ```typescript theme={null} const { models, version } = await verlon.models(); const grok = models['grok-4.5']; console.log(grok.pricing?.input); ``` Prefer this over the exported `MODEL_REGISTRY` constant when the answer matters at runtime: the constant is a snapshot frozen at publish time (it exists to generate the `ModelId` union at compile time), while `models()` asks the platform. ## Configuration ```typescript theme={null} import { Verlon } from '@verlon-ai/sdk'; const verlon = new Verlon({ apiKey: process.env.VERLON_API_KEY, // Required baseUrl: 'https://custom.api' // Optional, defaults to https://api.verlon.ai }); ``` ## TypeScript Support The SDK is fully typed. The tracing surface (`AgentHandle`, `TaskHandle`, trace options), the registry snapshot, and the `ModelId` union are all exported from the package root: ```typescript theme={null} import { Verlon, type ModelRegistrySnapshot } from '@verlon-ai/sdk'; const verlon = new Verlon({ apiKey: process.env.VERLON_API_KEY! }); const snapshot: ModelRegistrySnapshot = await verlon.models(); ``` ## Deprecated: Inference Methods > [!WARNING] > > All of the SDK's inference methods — `complete()`, `chat()`, `chatStream()`, `image()`, `embeddings()`, `tts()`, `video()`, and `ocr()` — are deprecated and **will be removed in 3.0.0, scheduled December 1, 2026**. Inference belongs to the official provider SDKs against Verlon's drop-in routes; see the [migration guide](/getting-started/migrate#migrating-off-the-verlon-sdks-inference-methods). Setup and migration guides: > **[OpenAI SDK Integration](/integrations/openai-sdk)** > > Point the openai package at Verlon > **[Anthropic SDK Integration](/integrations/anthropic-sdk)** > > Point the Anthropic SDK at Verlon > **[OpenAI Compatibility](/provider-compatibility/openai)** > > Supported routes and parameters > **[Anthropic Compatibility](/provider-compatibility/anthropic)** > > Supported routes and parameters > [!NOTE] > > Video and OCR have drop-in mirrors too: the `openai` package's async Videos surface (`videos.create` / `retrieve` / `downloadContent`) against `/v1/videos`, and `mistral.ocr.process()` against `/mistral/v1/ocr`. The [migration guide](/getting-started/migrate#migrating-off-the-verlon-sdks-inference-methods) has the full method-by-method table. ## Next Steps > **[Agent Gates](/platform/agent-gates)** > > Traces, tasks, and sessions on the platform > **[Creating Gates](/dashboard/creating-gates)** > > Configure gates in the dashboard > **[Migration Guide](/getting-started/migrate)** > > Method-by-method replacements for the deprecated inference surface