IBM’s Granite Speech 5.0 models process speech 12,600x faster than real time

IBM has introduced two new English speech recognition models under the Granite Speech 5.0 umbrella, designed to push the boundaries of transcription speed and efficiency. Released on August 25, 2026, the models, Granite Speech 5.0 TurboCTC and a noncommercial variant, each contain just 470 million parameters, yet IBM claims they can process more than 3.5 hours of speech in a single second. This performance is measured as an aggregate throughput exceeding 12,600 RTFx (real-time factor times) on a single NVIDIA H200 GPU, a figure that underscores the models' ability to handle audio at unprecedented speeds. The noncommercial model achieves a 4.85% word error rate (WER) on public English short-form test sets, while the Apache 2.0-licensed version records a 5.00% WER, according to IBM’s vendor-reported results. These scores are unofficial but were generated using Hugging Face’s infrastructure and the OpenASR Leaderboard’s scoring tools, suggesting they may soon be reflected in official rankings once the leaderboard updates its tables.

The two models share identical architectures and sizes but differ in their training data and licensing. The Apache 2.0 model is trained on approximately 60,000 hours of English audio and is cleared for commercial use, while the noncommercial variant incorporates an additional 15,000 hours from datasets like GigaSpeech and SPGI Speech, bringing its total training corpus to nearly 75,000 hours. IBM notes that the extra data provides a modest accuracy boost across most test sets, though it also introduces a slight disadvantage on the chunked Earnings22 test. The noncommercial model’s training data is licensed under CC-BY-NC-SA-4.0, restricting its use to non-commercial applications.

The dramatic speed improvements in Granite Speech 5.0 stem from a fundamental shift in design. Earlier versions of the Granite Speech line combined a Conformer acoustic encoder with a Granite language model using a projector and LoRA adapters, generating text in an autoregressive manner similar to chat models. In contrast, the 5.0 models eliminate the language model entirely, relying instead on a 16-layer Conformer encoder trained with Connectionist Temporal Classification (CTC). This approach maps audio frames directly to output tokens in a single non-autoregressive pass with greedy decoding, drastically reducing computational overhead. Two key optimizations drive the throughput gains: the models emit 12.5 tokens per second, achieved through three stages of 2x temporal subsampling from a 100-frames-per-second log-Mel front end, and they use a vocabulary of 16,384 trained subword units (SentencePiece for the noncommercial model and BPE for the Apache one). These changes reduce the frame rate to a quarter of its original value while maintaining accuracy, enabling the models to process audio more than 20 times faster than their predecessors.

This trade-off prioritizes transcription speed and efficiency over broader capabilities. Without a language model, the new Granite Speech 5.0 models lose features like speech translation and keyword biasing that were available in earlier versions. However, IBM positions them as ideal for enterprise speech-to-text applications where low latency and high throughput are critical, even if they sacrifice some versatility. The models are designed to run efficiently on laptops and edge hardware, making them suitable for real-time transcription tasks in environments where computational resources may be limited.

Independent benchmarks provide early validation of the models’ performance. On the FFASR Leaderboard, which evaluates recognition of noisy, reverberant speech, the noncommercial Granite Speech 5.0 model ranks fifth in accuracy as of August 25, 2026, while the Apache model sits in ninth place. Crucially, both models are the fastest entries on the leaderboard, with throughput measured on a single NVIDIA L4 GPU. The FFASR Leaderboard tests models under challenging conditions, including noisy, reverberant, and moving-source audio at various signal-to-noise ratios, scenarios where far-field recognition typically degrades. These results suggest that Granite Speech 5.0 can maintain strong performance even in less-than-ideal acoustic environments.

The headline throughput figure of 12,600 RTFx warrants careful interpretation. It reflects batched-inference performance on a high-end datacenter GPU, not the speed a user would experience on a laptop running a WebGPU streaming demo. Similarly, the reported WER figures apply only to short-form English and are based on vendor-provided data that has not yet been officially incorporated into the OpenASR Leaderboard’s rankings. IBM’s announcement acknowledges these limitations, framing the results as strong vendor-reported metrics pending further validation. The company also highlights that the training data for both models consists entirely of publicly available datasets, with synthetic additions totaling 2,740 hours. This synthetic data includes 2,500 hours of multi-speaker audio created by concatenating single-speaker segments and 240 hours of utterances generated by OpenAI’s open-weight gpt-oss models, synthesized using StyleTTS2 to target challenging numerical and address formats that often trip up speech recognizers. Training was conducted over 10 days on 8 NVIDIA H100 GPUs using IBM’s Blue Vela cluster.

For developers and researchers, the Granite Speech 5.0 models are now available on Hugging Face, with native support in the transformers library. They can be installed from source until the next official release. A browser-based streaming demo is also accessible in Chrome and Edge, offering a hands-on way to evaluate the models’ performance. These releases underscore IBM’s commitment to advancing open-source speech recognition, providing tools that balance speed, accuracy, and accessibility for a wide range of applications.

The launch of Granite Speech 5.0 arrives at a time when the demand for real-time transcription services is growing across industries like healthcare, customer service, and media. While commercial transcription services already dominate the market, IBM’s focus on open-source models with exceptional throughput could democratize access to high-performance speech recognition. However, the models’ limitations, such as their lack of speech translation and keyword biasing, mean they are best suited for specific use cases rather than general-purpose applications. As the models undergo further testing and independent validation, their true impact on the field will become clearer, particularly in edge and low-latency scenarios where speed is paramount.