Treble Technologies and Hugging Face have launched the Far-Field ASR (FFASR) Leaderboard, an open, community-driven benchmark that evaluates automatic speech recognition (ASR) models under realistic far-field acoustic conditions. The leaderboard went live on June 24, 2026, and the project team is inviting submissions, analysis, and community input on future directions.
Why far-field evaluation matters
Speech interfaces are increasingly used outside headsets and smartphones: conference transcription, in-car assistants, humanoid robots, smart glasses and hands-free tools operate in acoustically complex environments where reverberation, background noise, overlapping sounds and microphone distance affect recognition. Traditional near-field benchmarks such as LibriSpeech measure core recognition quality but do not predict performance in these real-room conditions, so a standardized open benchmark for far-field robustness has been missing until now.
Benchmark construction
The FFASR Leaderboard evaluates models across nine acoustic conditions; four of them determine the primary ranking (status as of June 22, 2026):
- Near-field (dry) — clean speech measured in an anechoic chamber
- Far-field high SNR — above 14 dB
- Far-field mid SNR — 8–12 dB
- Far-field low SNR — below 6 dB
The benchmark includes 14 fully furnished rooms with volumes ranging from 20 to 470 m³, covering bathrooms, living rooms with hallways, offices, classrooms and restaurant spaces. Each acoustic scene uses one target speaker (recorded anechoically) and up to three noise sources, including one transient noise (e.g., coughing) and one continuous noise (e.g., HVAC) at three SNR levels.
Acoustic data are generated with Treble Technologies' hybrid simulation engine, which combines a wave-based solver at low-to-mid frequencies with geometrical-acoustics modeling at higher frequencies. That hybrid approach models diffraction, scattering, interference and modal behavior better than simpler simulators, and sim-to-real validation is provided via the Lab Measured and Lab Simulated columns.
The held-out evaluation set contains 2,000 anechoic speech samples distributed across the 14 rooms and three SNR tiers, about 8 hours of audio per condition. Whisper-style text normalization is applied consistently, and the audio is not exposed to submitters to prevent test-set contamination.
Metrics and runtime reporting
The leaderboard reports word error rate (WER) and RTFx (audio seconds per inference second) for every submission. RTFx is measured under identical conditions on an NVIDIA L4 GPU. The Analysis tab includes Pareto-front visualizations of average WER against RTFx so teams can inspect accuracy/latency tradeoffs relevant for deployment.
FFASR builds on Treble’s prior work: the Treble10 dataset released last year provided far-field room impulse responses and established the simulation pipeline; FFASR extends that foundation into a standardized evaluation with a held-out test set, consistent normalization and automated scoring.
Early findings
Across submitted models a consistent pattern has emerged: near-field (dry) WER values look comparable to established benchmarks, but far-field WER—especially at low SNR—can be several times higher. Making that degradation visible in a shared leaderboard allows direct comparisons that were previously difficult outside proprietary pipelines.
The Pareto front of average WER versus RTFx highlights different design choices: some models favor speed with reduced accuracy, others prioritize accuracy at the cost of throughput, and a few sit competitively on both axes. Reporting near-field and far-field WER side by side helps distinguish genuinely robust models from those that work well only in clean, close-microphone conditions.
How to submit
From the FFASR Leaderboard Submit tab, paste a Hugging Face model ID and the evaluation runs server-side against the held-out dataset. The pipeline supports Whisper variants, IBM Granite Speech, Cohere Transcribe, Wav2Vec2 and HuBERT CTC heads, SpeechBrain ASR and most other Hub architectures without custom configuration.
For complex inference stacks that combine speech enhancement and ASR, a custom evaluator option allows teams to provide their own evaluate() function; custom evaluators run as Hub Jobs after moderator review. The submission notes field should document preprocessing steps to make results interpretable.
Roadmap
Planned future tracks include multi-talker scenarios, microphone-array evaluation (beamforming and spatial filtering) and echo cancellation. The project team says further development will follow community feedback: if a deployment environment or use case is underrepresented, they invite input.
The FFASR Leaderboard aims to give researchers and practitioners a practical, open tool for measuring far-field robustness and to steer development toward models that perform reliably in real acoustic environments. Contributors are encouraged to submit models, explore the Analysis tab and share suggestions to shape the benchmark’s evolution.
