Skip to main content
Verlon relays realtime WebSocket sessions to the provider. Your app keeps the provider’s own SDK and protocol: change the base URL and the key, and the session runs through Verlon, with its cost in your logs and under your spend limits. The registry lists every model served this way with "type": "realtime", dated versions included. Gates work on every path. Put a gate’s id where the model goes (the URL’s model, Live’s session.start, Gemini’s setup, the transcription model, or the model a client secret is minted for); the gate’s model serves, and the session is logged under the gate. A model that speaks a different protocol from the path is refused with realtime_provider_not_served, and the message names the path to use.

OpenAI voice

A speech-to-speech conversation on OpenAI’s Realtime protocol. Use the openai package, version 7.53 or later, with its ws peer dependency:
With turn_detection: null you decide when a turn ends, as above. Leave it out and OpenAI’s voice detection answers on its own when the speaker pauses. The WebSocket handshake answers with x-verlon-served-model (the model that serves, after any gate) and x-verlon-session-id (the session’s id in Verlon, for support).

From a browser

A browser can’t set an Authorization header on a WebSocket, and your Verlon key must never reach it. Instead, your server mints a client secret, and the browser opens the session with it.
1

Mint a client secret on your server

This is POST https://api.verlon.ai/v1/realtime/client_secrets, the same shape as OpenAI’s endpoint. Minting needs a Verlon API key.
2

Open the session in the browser

With the OpenAI SDK:
Or with a plain WebSocket, offering the secret as a subprotocol:
Then send audio and read the answer, as on the server.
How a client secret behaves:
  • The session is already set up. It opens with the session the secret was minted with (session.updated arrives first), so the browser sends only audio.
  • It works once. A second connection with the same secret gets 401 invalid_client_secret.
  • It expires. The default is 10 minutes. Set expires_after: { anchor: 'created_at', seconds } when minting to choose between 10 seconds and 2 hours.
  • It’s bound to its model or gate. Mint with a model id or a gate’s id, and name the same one, or none, when connecting. A secret minted for a gate opens the session through that gate.
  • It’s checked like your key. Your key’s checks run again when the secret is used: if the key has been revoked or your account suspended since minting, the session is refused.
  • It opens /v1/realtime only: OpenAI voice, xAI voice and OpenAI transcription. Open OpenAI Live, Gemini Live and Mistral sessions from your server.

xAI voice

xAI’s voice agent, grok-voice-3.0, speaks OpenAI’s Realtime protocol at the same endpoint, so use the same client as OpenAI voice and name the xAI model:
Things to know:
  • Wait for session.updated. Send the audio after xAI confirms the session, as above. Audio sent before the session is applied may not be heard as you meant.
  • Voices are xAI’s, such as Ara, not OpenAI’s.
  • It bills by time, not tokens. xAI reports no token usage, so an xAI voice session bills its per-minute rate for every second it’s open, silence included. Close the socket when the conversation ends.
  • From a browser, mint a client secret with model: 'grok-voice-3.0' and the session above, and open it as for OpenAI voice. The minted session is applied before the browser’s first message, so the browser can send audio as soon as session.updated arrives.

OpenAI Live

gpt-live-1 holds a full-duplex conversation: it listens and speaks at the same time, so there are no turns to commit. It speaks its own protocol, which the openai package doesn’t yet wrap for WebSockets, so use any WebSocket client:
Things to know:
  • session.start comes first. If it doesn’t arrive within 10 seconds, the session gets an error event and closes. Wait for session.started before sending audio.
  • Send audio in real time. Audio faster than 1.2× real time, or a burst over 5 seconds, is dropped with an input_audio_rate_limit_exceeded error.
  • Output audio never stops. session.output_audio.delta streams continuously, silence included, so play it as a stream rather than waiting for it to end.
  • End with session.close. You get session.closed with the final usage, then the socket closes.

Gemini Live

Use Google’s @google/genai SDK with its base URL pointed at Verlon and your Verlon key as the API key:
Things to know:
  • Set apiVersion: 'v1beta'. From version 3.0, @google/genai connects on v1 by default, where Live isn’t served, by Google or by Verlon.
  • gemini-3.8-live-extended-thinking needs a thinking level. Add thinkingConfig: { thinkingLevel: ThinkingLevel.LOW } (or HIGH) to the config, with ThinkingLevel imported from @google/genai. Without it, the session closes as soon as it opens, with 1007 and Google’s reason: “Thinking level must be specified for this model.”
  • Thoughts can arrive as text. gemini-2.5-flash-native-audio-preview-12-2025 sends its thinking as a text part marked thought: true before the audio. Skip those parts if you only want the answer.
  • A turn can end empty. If the speaker pauses mid-question, the model may start to answer, be interrupted, and send turnComplete with no audio. Wait for the turn that carries the answer before closing.
  • gemini-2.5-flash-native-audio-preview-12-2025 is deprecated. It shuts down on March 31, 2027; move to gemini-3.8-live.
  • Server only. Google’s ephemeral tokens aren’t served, so open Gemini Live sessions from your server.
  • Refusals arrive as the close reason. Once the socket is open, a refused session closes with a code and the reason as its text, the way Google’s own server does; there’s no error event. When Google refuses, you get Google’s own code and words: 1007 for a setup it can’t use, 1008 for a request it refuses. When Verlon refuses (a model it doesn’t serve on this path, a spend limit), the code is 1008.

Live translation

gemini-3.5-live-translate-preview listens and speaks the translation as it hears it, with no turns. Name the language to translate into in translationConfig:
Things to know:
  • Stream in real time, and close when you’re done. The translation follows the speech as it comes in. There’s no turnComplete to wait for.
  • Speech already in the target language is silent unless you set echoTargetLanguage: true in translationConfig. The default target is English (en).
  • It needs @google/genai 2.8 or later. Earlier versions drop translationConfig without a word, and the session then translates into English.

OpenAI transcription

Stream speech up and the transcript streams back down as it’s recognized. Open a transcription session with the same client as OpenAI voice:
Things to know:
  • The first event names the model. It must be session.update with session.type: "transcription" and session.audio.input.transcription.model. If none arrives within 10 seconds, the session gets an error event and closes.
  • Commit the audio yourself. gpt-live-transcribe doesn’t take server voice detection, so set turn_detection: null and send input_audio_buffer.commit when a stretch of speech ends.
  • From a browser, mint the client secret with the transcription session above (session.type: 'transcription'), and open it with { intent: 'transcription' } or the plain WebSocket from From a browser.

Mistral transcription

Use Mistral’s SDK with its server URL pointed at Verlon:
Mistral has no client secrets, so open Mistral sessions from your server.

Billing and logs

  • What a session costs. Each session bills by what the provider reports:
    • OpenAI voice and Gemini Live: the text and audio tokens in and out, at the model’s rates;
    • OpenAI Live and xAI voice: every second the session is open, including silence;
    • transcription: the seconds of audio transcribed.
  • Whose key. A session runs on your own provider key when you’ve added one, otherwise on Verlon’s.
  • Spend limits. Spend is reserved a minute ahead and renewed as the session runs. If your spend limit or credit can’t cover the next minute, the session closes (code 1008), so it never runs past what you can pay for.
  • Logs. Each session is one entry in Logs: the model, the gate, the cost, and the session’s length as its latency. Audio and transcripts aren’t stored.

Limits

Errors

A session that can’t open is refused before the WebSocket opens. The refusal is an HTTP status with the provider’s error body: OpenAI’s { "error": { "message", "type", "code", "param" } } on /v1/realtime and /v1/live/sessions, Google’s on the Gemini path, and Mistral’s on Mistral’s path. OpenAI Live, Gemini Live and OpenAI transcription read the model from the first message, so their model refusals come after the socket opens: an error event with the same code (OpenAI), then the close. Once a session is open, it ends with a WebSocket close code:
  • OpenAI SDK: the same client for chat, responses and audio
  • xAI, Google and Mistral: their other models through Verlon
  • Gates: route a session through a gate