Live transcription

The realtime_transcription capability uses /v1/realtime for speech-to-text. Soniox is available as soniox/stt-rt-v5; the existing OpenAI adapter uses openai/gpt-live-transcribe. Check /v1/models for capabilities.realtime_transcription (and its compatibility alias live_transcription) and session limits. Models must be published, permitted by project policy, and configured with a provider credential. Google can use the same input and event contract through a future adapter. Conversational audio, translation, Google Live and WebRTC are currently unavailable in Cloud.

Dictate into console inputs

Enable the project's AI API, link its USD billing account, and configure a published transcription model. Project owners see a microphone when focusing any editable text input or textarea across the console, including dialogs and search fields. Soniox is preferred when available. Read-only fields, passwords and non-text controls are excluded. Developers can opt out a field or container with data-voice-input="off".

Click the microphone, allow microphone access, and speak: text appears directly at the cursor or replaces the selected text. Provisional words update as recognition improves. Click Stop or press Escape to finish, then edit or submit normally. Editing the field or moving to another field stops capture. No text is submitted automatically. Sessions are charged to the selected project, using its default tenant. Existing console permissions still apply; members with read access cannot create billable sessions.

The browser connects to Cloud with a one-use session ticket. Permanent Soniox credentials stay on the gateway, which proxies audio and obtains billing evidence.

Try the playground

Select a project and tenant in the AI playground, choose a live transcription model, and click Start microphone. Your browser requests microphone permission. Click Stop to finish and wait for the final account charge. Recording ends after 60 seconds. No automatic reconnect or paid retry occurs. Request history remains available if the connection is interrupted.

Server connections

Connect with a project key that has ai:generate. Use a fresh idempotency key for each intended session; reuse it only to inspect the same request, never to resume audio.

import WebSocket from 'ws';
const socket = new WebSocket(
  'wss://apis.cloud.feeef.org/v1/realtime?model=soniox/stt-rt-v5',
  { headers: {
    Authorization: `Bearer ${process.env.CLOUD_API_KEY}`,
    'Idempotency-Key': crypto.randomUUID(),
  } },
);
socket.on('message', raw => {
  const event = JSON.parse(raw);
  // Wait for feeef.session.ready before sending PCM audio.
  console.log(event);
});

After feeef.session.ready, send text JSON events:

{"type":"input_audio_buffer.append","audio":"BASE64_PCM16_LE_MONO_24000_HZ"}

Commit a turn with {"type":"input_audio_buffer.commit"} after at least 100 ms of audio. End input with {"type":"feeef.session.close"} and keep reading until feeef.session.completed. Soniox emits feeef.transcription.updated with item_id, text, confirmed_text and provisional_text: replace that item's visible text on every event. Final tokens arrive once; provisional tokens can be revised. Both adapters emit conversation.item.input_audio_transcription.completed; OpenAI also emits append-only .delta events. Completion order may differ from submission order; match item_id. Cloud pins the upstream session configuration and rejects client session.update events.

Browser authorization

Browsers cannot set WebSocket bearer headers. Your trusted server creates a request using POST /v1/ai/live-sessions with {"model":"openai/gpt-live-transcribe","type":"transcription"}, the normal project headers, and an idempotency key. It returns task, ticket, expires_at and websocket_url. Pass the short-lived ticket to the browser:

const socket = new WebSocket(session.websocket_url, [
  'feeef.live', `feeef.ticket.${session.ticket}`,
]);

The ticket expires 60 seconds after admission and authorizes only that request. It is bound to its project, tenant, billing account and credential. Connection and submission recheck project access, credential revocation, rate limits and model policy. Do not log tickets or expose permanent Cloud/provider keys to browsers. There is no resume operation. A created request that is never connected is recovered without an upstream submission.

Limits and billing

Cloud accepts mono PCM16 little-endian audio at 24 kHz, at most 60 seconds of submitted audio, 90 seconds wall time and 120 commits. Frames are capped at 64 KiB and 100 per second; pending audio buffers are bounded. Use short audio chunks and commit periodically. For OpenAI, a final fragment below 100 ms is padded to its provider minimum and contributes to usage. Soniox flushes its pending audio without padding.

Soniox bills actual tokens: the reviewed rates are $2 per million input audio tokens and $4 per million input/output text tokens. Cloud retrieves the exact session's usage log using its server-owned request reference; audio duration and transcript character counts are never substituted for token evidence. No funds are reserved upfront; actual usage is charged after provider evidence arrives, and the balance may go negative. OpenAI uses its reviewed $0.017 per audio minute rate and provider-reported fractional duration; with 10% markup, 60 provider seconds cost $0.0187.

AI usage is postpaid without upfront reservations. Both adapters use exact decimal arithmetic and the pricing/margin snapshot taken at admission. Settlement rounds upward to one micro-dollar. Missing final evidence remains pending reconciliation; disconnecting does not discard usage or trigger another paid stream.

Operator setup

Configure AI_PROVIDER_KEY_SONIOX on the gateway, or add a Soniox provider credential in AI administration. The key needs realtime Speech-to-Text and Usage logs read permissions. Install the reviewed default models to publish soniox/stt-rt-v5 (or publish an equivalent reviewed token price card), and activate AI on the deployment. Then enable AI for each project that should have voice input. Do not put the Soniox key in browser environment variables.

If usage logs are delayed, the gateway makes bounded read-only lookup attempts and keeps billing pending reconciliation when evidence is unavailable. No paid stream is replayed. Operators can reconcile using a normalized cloud.live_transcription document containing the original feeef_request_id, supplier id, complete: true, exact usage counters, and the matching supplier log in soniox_receipt.

See Soniox realtime transcription, usage logs, and pricing. The OpenAI transcription guide describes its ASR events. File transcription and conversational Realtime use different contracts.