The Open ASR Leaderboard has expanded its private evaluation data through a collaboration with Liva AI, adding two new conversational English accent splits: Egyptian and Nigerian.
- Liva AI Conversational EG — Egyptian: 1.34 hours; conversational, spontaneous; transcriptions are punctuated, cased, and include disfluencies.
- Liva AI Conversational NG — Nigerian: 1.55 hours; conversational, spontaneous; transcriptions are punctuated, cased, and include disfluencies.
These additions continue previous efforts to include private data in the leaderboard. A few months ago the project integrated 11 private splits from Appen and DataOceanAI covering scripted and conversational speech across five accents; later it added an additional private split for Indian-accented English with Voice Arena. The new Liva AI splits increase the amount of private test data available for evaluations.
Why this matters
The stated objectives are to improve accent and language representation in ASR evaluations and to strengthen trust in reported results by introducing private datasets that are not publicly available. Private held-out sets help reduce so-called “benchmaxxing,” where systems are overfit to known public test sets.
Richer accent breakdown in the Private tab
With the new accents, the Open ASR Leaderboard presents a more detailed accent breakdown inspired by Kachru’s Three Circles (the idea is credited to Hume AI). The leaderboard will report:
- US vs. non-US, where the non-US list includes UK, Australia, Canada, India, Nigeria, Egypt.
- Inner Circle (US, UK, Australia, Canada) vs. Outer & Expanding Circles (India, Nigeria, Egypt).
The organizers say they will further split the Outer and Expanding Circles as more accents are added.
About Liva AI
Liva AI collects multimodal interaction data through consumer products used by real people. Because Liva controls both the collection environment and distribution, it can build products aimed at particular model gaps across domains, industries, languages, modalities, and interaction types. The company says this provides access to natural, high-context behavior that emerges in real product use rather than artificially elicited or task-based interactions.
Although the new splits are relatively small in duration, they are positioned as a strategic step toward broader accent coverage and more trustworthy ASR evaluations.
