Alibaba’s Qwen Audio 3.0 tops the speech leaderboard. The free repo bolted to it is the part that changes how you work.
Every voice assistant you have ever used works like a walkie-talkie. You talk, you stop, you wait for the beep. Alibaba just shipped something that works like a phone call, and then attached a coworker to the other end of it.
Verdict up front. Qwen Audio 3.0 Realtime Plus is currently the top native speech-to-speech model on the Artificial Analysis Speech-to-Speech Index at 84.1%. That part is real. It is also not the fastest, not the cheapest once you count reasoning tokens, and the models are not open weights. The thing actually worth your Saturday is not the model at all. It is qwen-audio-agent, the Apache 2.0 runtime Alibaba open-sourced on GitHub that lets you talk to an agent while it works. Install it, give it an hour, learn what conversational software feels like. Do not rip out a production voice stack over this. Not yet.
The walkie-talkie problem
Think about how a walkie-talkie actually works. One channel, one direction at a time. You press the button, you say your piece, you release, and you say “over” so the other person knows it is safe to talk. If you both press at once, nobody hears anything.
That is half duplex. And that is every voice assistant you have ever used, wearing a very convincing phone costume.
The reason is architectural. The classic voice stack is three separate boxes bolted together. A speech recognizer turns your audio into text. A language model reads that text and writes a reply. A speech synthesizer reads the reply out loud. Somewhere between those boxes, your tone, your pauses, your sigh, and your “hmm, wait” all get thrown in the garbage. The transcript survives. Everything that made it a conversation does not.
The handoff between boxes also needs a trigger, and that trigger is usually silence. The system waits for you to shut up for about 800 milliseconds and then decides you must be finished.
Which produces the single most universal AI experience of the last three years.
You: “Hey, can you pull up the invoice from, uh...” Assistant: “Certainly! Here are your recent inv...” You: “I WASN’T DONE.” Assistant: (already three sentences into the wrong invoice)
We have all yelled at a speaker. Some of us have yelled at a speaker in a car, alone, at a red light, while the driver in the next lane watched the whole thing happen. Not naming names.
Stage two, the phone call
A phone call is full duplex. Both ends of the line are open at once. You can interrupt. You can say “mm-hmm” without derailing anything. You can trail off mid-sentence and the other person waits, because they can hear from the shape of your voice that you are still thinking.
Qwen Audio 3.0 Realtime is built as a native speech-to-speech model. Audio goes in, audio comes out, no transcript relay race in the middle. Alibaba’s docs describe a WebSocket full-duplex connection with streaming input and streaming output, and the model also supports AOQ and WebRTC for client-side integration.
The feature that makes this land is called smart_turn. Rather than watching for silence, the model combines acoustic perception with semantic understanding to decide whether you have actually finished your thought. Alibaba’s documentation says filler sounds like “uh” and “hmm” do not interrupt the conversation. There is also a speaker enhancement option where you hand it a sample of the target user’s voice so it locks onto that one person and tunes out everybody else in the room.
The benchmark backs the claim. On Artificial Analysis’s Full Duplex Bench subset, which measures pause handling, turn taking, interruption handling, and backchannel handling, Qwen Audio 3.0 Realtime Plus scores 98.4%. Top of the board. The runner-up is its own Flash sibling at 96.9%.
So the phone call works. Now for the part that matters more.
Stage three, the colleague with hands
A phone call is still just a phone call. The person on the other end talks, and talking is all they do.
What changes here is that the voice can now reach out and touch things. Qwen Audio 3.0 Realtime supports function calling directly from the spoken conversation, and Alibaba’s launch materials describe native tool use through FunctionCall, MCP, APIs, and knowledge bases. The path stops being voice to answer. It becomes voice to reasoning to tool to action to result to spoken answer.
Then there is the repo.
qwen-audio-agent is an open-source runtime from the speech team at Alibaba’s Tongyi Lab. Its README states the design goal plainly: keep the agent talking, working, and present. The architecture splits the conversational front end from the backend agent, so when you hand off a job, the conversation does not freeze while the job runs.
Picture the difference. You say “figure out why auth is failing on the staging branch.” A normal voice assistant goes quiet and eventually returns an answer or a timeout. This one delegates to a coding agent, keeps talking to you, lets you ask what files it is looking at, lets you cancel, and comes back into the conversation when the work is finished.
The backend list is where this gets interesting for anyone already living in a terminal. The README table lists OpenCode, OpenClaw, Qoder, Hermes, CodeBuddy, and Codex. The changelog documents several the table has not caught up to: a Kimi Code backend in 1.1.0, a Claude Code backend in 0.10.0 built on the Zed ACP adapter, and a Qwen Code backend added in 1.8.0 on August 9. There is also a generic ACP stdio entry point, so any agent speaking that protocol plugs in without anyone touching gateway code.
It reuses what those agents already have. Your tools, your MCP servers, your skills, your auth. You are not rebuilding your setup for voice. You are putting a mouth on the setup you already run.
The release velocity is unusual. Version 1.8.3 shipped today, August 12. Versions 1.8.2 and 1.8.1 both shipped yesterday. Recent releases added scheduled reminders, a voice wake word, long-term memory, Windows and Linux desktop builds, and a built-in computer-use MCP that gets injected into every backend session so the agent can click and screenshot even when the backend was never configured for it. That last one can be switched off with an environment variable, and if you are the sort of person who reads permission models before installing things, you will want to know it defaults to on.
Now the money part
Nothing here is as free as the phrase “open source” implies, and this is where most coverage stops paying attention.
The runtime is free, no asterisk. Apache 2.0, installed with npm, no strings. It needs Node 22.22.2 or 24.15.0 and up.
The models are not open weights. Qwen Audio 3.0 Realtime and Qwen Audio 3.0 TTS are hosted services, API only, served through Alibaba Cloud Model Studio. You do not download them. That is a different thing from the Qwen3-TTS line, which is Apache 2.0 with downloadable weights you can self-host, and plenty of writeups blur the two.
Alibaba does provide a free trial quota on Model Studio, so your first evening costs nothing.
After that, the numbers get interesting. Artificial Analysis lists Qwen Audio 3.0 Realtime Plus at $0.03 per hour of audio input and $0.18 per hour of audio output. It lists OpenAI’s GPT-Realtime-2 at $1.15 and $4.61. That looks like a slaughter.
It is not. On the same leaderboard, the blended cost of actually running their 40-question benchmark comes out to $4.42 per hour for Qwen Plus and $4.14 for GPT-Realtime-2 High. Qwen is the more expensive one. The audio is nearly free, and then the reasoning tokens show up with an invoice. Artificial Analysis’s methodology counts audio in, audio out, text in, text out, and separately exposed reasoning tokens, and it excludes tool-call costs entirely. Your real bill tracks how much the model thinks, not how long you talk.
Three more costs that never make it onto a pricing page:
Backend agents bring their own meters. Delegating to Claude Code or Codex or Qwen Code means paying whoever serves that model, stacked on top of the voice layer.
Tool calls sit outside that benchmark figure, and tool calls are the entire point of an agentic voice runtime.
Going fully local does not make it free either. Version 1.3.0 added a Hugging Face speech-to-speech frontend that needs no cloud key, which is great, and which also means you have traded an API bill for a GPU bill plus a stack of VAD, STT, LLM, and TTS components you now maintain yourself.
On the text-to-speech side, Plus is listed at $27.6 per million characters. Simba 3.2, currently tied with it inside the error bars, is listed at $10.0.
Where the story has already drifted
The figures above came off the live leaderboards this morning rather than the July launch posts, and in three places those two things no longer agree.
Time to first audio. Artificial Analysis’s launch announcement clocked Qwen Audio 3.0 Realtime Plus at 4.02 seconds and called it among the slowest models it had measured. Today’s leaderboard lists 1.54 seconds. A lot of the coverage still quotes 4.02. Either number loses to Grok Voice Think Fast 2.0 High from SpaceXAI, formerly xAI, at 0.70 seconds.
Text-to-speech ranking. Alibaba’s July blog says Qwen-Audio-3.0-TTS-Plus is number one on the Artificial Analysis arena, and it was, at roughly 1,236 Elo. As of today Simba 3.2 leads at 1,231 with Qwen second at 1,230. That is a one-point gap with a plus-or-minus 15 confidence interval on both, so the honest read is a statistical tie, and the leaderboard itself assigns both models a rank range of 1 to 2.
Agentic performance. Qwen took the top spot on the Tau Voice agent benchmark at launch with 54.6%. On July 29, Grok Voice Think Fast 2.0 High took it back with 56.5%. Qwen still leads the overall index, 84.1% to 82.9%, and still holds speech reasoning at 99.2% and conversational dynamics at 98.4%.
Qwen is the overall leader. Qwen is not the leader in every category. Anyone telling you it is simply the best voice AI is reading from a press release that is three weeks old.
The weird one on the horizon
One more thing worth knowing about, filed under “not yet, but soon.” Alibaba’s researchers published a technical report for Qwen-Audio-3.0-Gen-Preview at the end of July. Instead of generating a voice, a sound effect, and a music bed as separate jobs you assemble in an editor, it generates the whole audio scene as a single mixed waveform. Dialogue between multiple speakers, ambience, music, and foreground events, all coordinated on one shared timeline. The architecture is a diffusion transformer running over a shared VAE that compresses 48 kHz stereo into 25 Hz latent sequences.
That is a research preview, not an API. But if you have ever tried to fake a café scene by stacking four separate audio generations in a timeline, and quit when the rain refused to sit right underneath the dialogue, you can see exactly where this is going.
So What
Do this Install qwen-audio-agent tonight on the free Model Studio quota and point it at an agent you already use. One hour with it will teach you more about where interfaces are heading than another month of reading about them.
Skip this Replacing a working production voice stack over a 1.2-point index lead. Latency, tool-call billing, and data residency decide that call, not a leaderboard.
Wait on this Qwen Audio 3.0 Gen. It is a paper right now. Watch for an actual API before you design anything around generative audio scenes.
Steal this line “Your assistant isn’t slow, it’s half duplex.”
Sources
Artificial Analysis, Speech to Speech Leaderboard (live index, latency, and price figures)
Artificial Analysis, Text to Speech Arena, Provider Voices (live Elo standings)
Alibaba Cloud Model Studio, Qwen-Audio real-time voice model docs