OpenAI added three new audio models to its Realtime API: GPT‑Realtime‑2, GPT‑Realtime‑Translate, and GPT‑Realtime‑Whisper. The headline model, GPT‑Realtime‑2, is an end‑to‑end speech‑to‑speech system that lets developers adjust a "reasoning effort" parameter to trade off latency against reasoning capability.
What the new models do
- GPT‑Realtime‑2: performs end‑to‑end audio in and audio out processing (reasoning included), accepting text, audio, and image inputs (up to 128,000 tokens) and producing text and audio outputs (up to 32,000 tokens). Measured time to first audio ranges from 1.12 seconds at minimal reasoning to 2.33 seconds at high reasoning.
- GPT‑Realtime‑Translate: speech translation from over 70 input languages into 13 output languages; accepts audio in (up to 16,000 tokens) and outputs audio (up to 2,000 tokens).
- GPT‑Realtime‑Whisper: speech‑to‑text transcription; accepts text/audio in (up to 16,000 tokens) and produces text out (up to 2,000 tokens).
GPT‑Realtime‑2 features
- Five discrete reasoning effort levels: minimal, low, medium, high, xhigh; higher levels increase latency and consumption of reasoning tokens.
- Parallel tool calls and optional narration of those tool calls using spoken phrases (e.g., "checking your calendar").
- Optional preambles such as "let me check that" so users can see the model is working while it reasons.
- Graceful handling of requests it cannot complete (it will notify the user, rather than remain silent).
- Tone (attitude) control and function calling support.
Performance and benchmarks
- GPT‑Realtime‑2 led the Scale AI Audio MultiChallenge audio‑output leaderboard and performed well on Artificial Analysis Conversational Dynamics, though it lagged on the Artificial Analysis Speech Reasoning leaderboard.
- On Artificial Analysis Big Bench Audio, GPT‑Realtime‑2 set to high reasoning scored 96.6%, tied with Google’s Gemini 3.1 Flash Live Preview at high reasoning; it trailed Step‑Audio R1.1 Realtime (97.6%) and Grok Voice Think Fast 1.0 (97.1%). At minimal reasoning it fell to 71.8%.
- On Artificial Analysis Conversational Dynamics, GPT‑Realtime‑2 at minimal reasoning led with 96.1%; at high reasoning it scored 95.3% and trailed GPT‑Realtime‑1.5 and GPT Realtime Mini (95.7%).
- On 𝜏‑Voice (three customer‑service domains), GPT‑Realtime‑2 led the airline domain with 63%, but its overall score across the three domains was 39.8% — behind Grok Voice Think Fast 1.0 (52.1%) and ahead of Gemini 3.1 Flash Live Preview set to high (37.7%).
- Latency comparison: Step‑Audio R1.1 Realtime requires 1.51 seconds to first audio, Grok Voice Think Fast 1.0 requires 1.25 seconds, while GPT‑Realtime‑2 at high reasoning requires 2.33 seconds — meaning the two faster models deliver first audio sooner.
- Even at xhigh reasoning, GPT‑Realtime‑2’s pass rate on the Scale AI Audio MultiChallenge was below 50%, indicating that reliable multi‑turn spoken dialogue remains difficult for current models.
Availability and pricing
- The models are available through the OpenAI Realtime API.
- Pricing published by OpenAI:
- GPT‑Realtime‑2: $32 / $0.40 / $64 per million input / cached / output audio tokens; $4 / $0.40 / $24 per million input / cached / output text tokens; $5 / $0.50 / $24 per million input / cached image tokens.
- GPT‑Realtime‑Translate: $0.034 per minute.
- GPT‑Realtime‑Whisper: $0.017 per minute.
Undisclosed details
OpenAI has not disclosed parameter counts, model architectures, or the training data and methods used for these models.
Why this matters
Voice agents have typically favored simpler, lower‑latency interactions because deeper reasoning increases response time. GPT‑Realtime‑2’s configurable reasoning effort gives developers explicit control over that tradeoff: minimal settings for faster turn‑taking in live conversations, and higher settings for interactions that can tolerate more latency in exchange for stronger reasoning. This flexibility may broaden the set of tasks voice agents can handle without reverting to text‑based processing.
Conclusion
GPT‑Realtime‑2 advances end‑to‑end speech‑to‑speech reasoning by exposing a tunable latency‑vs‑reasoning knob and adding features such as narrated tool calls and tone control. Benchmarks show substantial improvements in some conversational metrics, but higher reasoning settings bring notable latency and the overall pass rates on multi‑turn spoken dialogue challenges indicate further work is needed for consistently reliable performance.
