Gradium has released Phonon, a family of small text‑to‑speech (TTS) models designed to run entirely on users’ devices. Phonon requires no server or network connection: typed text remains on the device and audio synthesis happens locally. The team published model weights and licensing varies: Phonon 31m is available under Apache 2.0, while other models are released under CC BY‑NC.
Why an on‑device voice?
Most voice AI runs in the cloud, sending each utterance to a server. Phonon’s goal is to keep user text local and to work offline. The open weights allow commercial use for the Apache‑licensed 31m model, while the CC BY‑NC models are free for non‑commercial projects. For commercial use of those models, pricing is stated to be user‑based rather than tied to generated speech volume.
Development background and technology
Phonon builds on the Pocket TTS lineage and uses flow matching for audio generation, an approach that differs from many other TTS systems. Flow matching enabled higher quality at very small model sizes, but introduced new training challenges: early training runs were numerically unstable and required several tricks to achieve consistent quality across all five languages.
What was released
- Two model sizes of roughly 65 MB and 75 MB.
- Support for five languages: English, French, German, Spanish and Portuguese.
- More than 100 built‑in voices, plus the ability to create custom voices in Gradium Studio via voice cloning or design.
- Handling of common tricky tokens such as phone numbers, dates and codes.
- Runtimes on phones, laptops, single‑board computers and in a browser tab.
Phonon 31m speaks English and is Apache 2.0 licensed; Phonon 40m covers all five languages under a non‑commercial license.
Demos and early projects
The launch included demos from Gradium and partners:
- An Apple Watch demo (Wi‑Fi off) that walks through a coffee brew step by step.
- An iPhone demo (no signal) that counts reps during exercise.
- Autonomous’s Lamp robot speaking in character with a sorting‑hat gimmick.
- A browser demo turning an offline dino runner into a radio‑show hosted by a character called Rex.
- A sci‑fi game and a Proust reading that run on the player’s own machine.
Hugging Face built the Transformers integration so developers could use Phonon from day one. Early community projects include ExecuTorch (runs Phonon on a visitor’s GPU in the browser), Micdrop (Node.js integration), and multiple Spaces such as smallest‑s2s.
Phonon was downloaded over 200,000 times in its first two weeks and remained trending on Hugging Face.
Performance on the Hugging Face Open TTS Leaderboard
Hugging Face’s Open TTS Leaderboard evaluates models across 23 languages using metrics for:
- intelligibility (word error rate of generated outputs),
- speed (inverse real‑time factor and time‑to‑first‑audio),
- naturalness (UTMOS for English, KOMOS for Korean).
Fixed‑voice and voice‑cloning models are evaluated separately, with an extra speaker similarity metric for cloning. The leaderboard also offers a Listen tab for side‑by‑side audio comparison and community voting.
Phonon placed first for intelligibility in German and Spanish, and ranked in the top three for every language evaluated, competing with models up to 100 times its size. It particularly excelled on speed, topping the CPU streaming benchmark on time‑to‑first‑audio.
Inference and integrations
Gradium built a new inference framework, ptts, specifically for Phonon. ptts aims to deliver strong performance across many devices and supports more than 10 backends, including CPU (Intel, Arm, WebAssembly), GPU (CUDA, Vulkan, Metal, WebGPU) and NPU (Apple Core ML, Snapdragon®).
Phonon is also available in Hugging Face Transformers from v5.21 as a reference PyTorch implementation. While ptts focuses on single‑stream on‑device performance, the Transformers integration targets GPU throughput with batched generation and full‑graph torch.compile for high‑scale synthesis. As a native PyTorch model, Transformers provides a base for integration or research.
Phonon’s architecture is modular on top of Pocket TTS; the changes required to define Phonon are described as roughly 200 lines that capture the differences. The model generates audio latents autoregressively: each step outputs a latent fed back as the next input, and training loss is computed on those latents. In this design latents play the role of tokens for a language model, so an audio codec mapping audio to latents acts as a tokenizer. Transformers ships an audio tokenizer held by the processor alongside the text tokenizer.
A published example shows the model with approximately 30,788,354 parameters and the audio tokenizer with about 10,305,313 parameters.
Voices are represented as tensors of speaker latents rather than raw recordings. Phonon does not encode raw audio on the fly; precomputed speaker latent files are provided in gradium/phonon‑voices.
Prompt format and generation flow
Checkpoints include a chat template that specifies voice positions, text and an audio opening token. Developers write a standard conversation with an empty audio item where the voice should appear. The processor expands the audio placeholder to one position per frame, normalizes and tokenizes the text, and batches inputs for the model. The model.generate call returns audio latents, which the processor decodes into audio files.
Next steps and customization
Phonon can speak any voice: clone a voice from a short recorded sample with the speaker’s consent, or describe a voice in text within Gradium Studio and download the resulting voice file. Gradium also offers to train custom Phonon models with partners: they can reduce model size for tight memory budgets, add new languages (for example Korean, Arabic or Japanese) or fine‑tune on product‑specific vocabularies.
Contact and acknowledgements
Gradium invites users and developers to get in touch about needs and custom work. The company thanks the Hugging Face team and everyone who tried Phonon in its initial two weeks.
Phonon is Gradium’s first open release; users are encouraged to try it and share what they build.
