Google has introduced SL2T (sign‑language‑to‑text), a massively multilingual translation model for sign languages, and is shipping the first consumer features that let Deaf and hard of hearing users dictate by signing. The capability is available on Pixel 11 in Gboard and Live Transcribe, initially supporting American Sign Language (ASL) to English.
What SL2T enables
With SL2T, users can sign instead of typing in contexts such as web search, drafting messages or documents, and asking Gemini to answer questions or perform tasks. In Live Transcribe, participants can sign responses rather than typing replies. According to Google testers, signing in ASL can be faster and more natural than typing in English.
Technical challenges and design choices
Google highlights two central challenges in sign‑language AI: sign languages are independent natural languages with unique grammars, so models must perform true machine translation rather than sequential sign‑to‑word transcription; and sign languages convey meaning through simultaneous, fine‑grained whole‑body movements (hands, arms, torso, head, face), which demands high‑performance computer vision.
SL2T was developed with both cultural sensitivity and large‑scale data. The model was trained on over 100,000 hours of data across more than 50 sign languages, with roughly one quarter of the training data in ASL.
Privacy: landmarks instead of raw video
To protect user privacy, SL2T represents sign input as a sequence of pose landmark coordinates rather than raw camera video. An on‑device MediaPipe Holistic model tracks signer keypoints, and only those geometric coordinates are sent for translation; the original video can be discarded immediately.
Model approach and performance
Rather than translating via intermediate "gloss" annotations, SL2T maps landmark sequences directly to text. The Google announcement states this approach avoids artificial vocabulary limits and lets translation quality scale with data.
On evaluation benchmarks such as FLEURS‑ASL (sd‑test) for ASL→English translation, SL2T achieved a zero‑shot BLEURT score of 70, which Google reports is substantially higher than previously published results. The team also addressed practical concerns including minimizing streaming latency, avoiding hallucination on non‑signing input, supporting left‑handed signers (about 10% of signers), and improving one‑handed signing performance used when holding a smartphone.
Community collaboration and governance
Google emphasizes that the project was built with the Deaf community. The initiative involved Sam Sepah, a Deaf Google employee, and data collection, evaluations, and impact assessments were carried out with Deaf partners. To guide responsible deployment, Google formed the AI Sign Language Advisory Committee (AISLAC), which includes global Deaf organizations and subject‑matter experts.
A joint impact report accompanying the SL2T 1.0 release in Gboard and Live Transcribe documents the technology’s capabilities and current limitations; Google says it will continue participatory governance for future major sign‑language releases.
Next steps
Google frames ASL input on phones as an initial milestone. The company says it plans to expand SL2T to additional sign languages, work on sign‑language generation, and develop further AI capabilities. SL2T is available first on Pixel 11 in Gboard and Live Transcribe, with more devices and languages coming, at no additional cost.
Acknowledgements
The announcement credits teams from Google DeepMind and Android. The core SL2T development team includes Garrett Tanzer, Benoit Brard, Elizabeth Clark, Tim Dozat, Sebastian Ebert, Dan Garrette, Manfred Georg, Vicky Holgate, Shankar Kumar, Mohammad Saboorian, Miloš Stanojević, Megh Umekar, John Wieting, Andy Zhang, and Chris Dyer. The Android integration team includes Ausmus Chang, Sai Aditya Chitturu, Dayle Chiu, Anna Chou, Ajay Dudani, Angana Ghosh, Alex Huang, Joanne Kim, Ed Lee, Thomas Lin, James Su, Yanchao Su, and Sharlene Yuan. Additional contributors and early testers are also acknowledged in the release.
