Here is a scenario every voice-AI team eventually hits. The demo is flawless on clean, native-accented English; then a real user speaks with an accent, over a phone line, in a noisy room, and the transcript falls apart. The instinct is to fine-tune the model on that accent. For a single-region product that instinct can even work -- but for a global product it is a trap, and the real answer is counter-intuitive: breadth, not specialisation.
Where accent errors actually come from
Speech recognition maps sound to words using patterns learned from training data. When a model has heard little speech resembling a given speaker -- their accent, cadence, or the audio conditions -- its guesses drift toward whatever it has heard most. Small models fail hardest here, because they simply have not absorbed enough acoustic diversity to generalise.
The garbled transcript is not a bug; it is a model operating outside the distribution it was trained on. This is why the same small model can transcribe a clean studio recording perfectly and then mangle a real phone call. The pipeline is fine. The model has just never really heard a voice like the one in front of it.
The fine-tuning trap
Fine-tuning to a specific accent does raise accuracy for that accent. But a model has finite capacity, and specialising it pulls that capacity toward the target and away from everything else. For a single-market product this can be a fair trade. For a product used across the globe it is actively harmful: you improve one accent while quietly degrading the dozens of others your users actually have.
You cannot fine-tune your way to universal coverage one accent at a time -- you run out of capacity and introduce regressions faster than you fix them. Picture it as a curve: a model tuned to one accent shows a tall spike on its target and troughs everywhere else, while a large model trained on diverse data shows a broad, even plateau, lower than the specialist's peak but far higher across the range a global product must serve.
Breadth beats specialisation
For a worldwide user base, accuracy comes from scale and data diversity, not specialisation. A model that has heard millions of hours of varied speech generalises to accents it was never explicitly trained on, because it has learned the underlying structure of how humans produce sound rather than memorising one group's patterns.
The practical implication is blunt. The fix for failing on accents is usually a bigger, more diverse model -- not a narrower, tuned one. For a global product the model you want is the broadest one you can run, not the one most sharpened to a single voice.
If you tune, tune for domain -- and let the system carry accuracy
There is a place for adaptation, but it is the domain, not the accent. Biasing a model toward your vocabulary -- product names, industry jargon, the terms that appear in your calls -- helps every user regardless of how they speak, because those words are hard for any general model. Tuning toward one accent helps some users at the expense of others. The first compounds; the second cannibalises.
The most durable answer is to stop treating the raw transcript as the finished product. When a language model sits downstream, interpreting and correcting the text in context, the transcript no longer has to be perfect for every accent. Accuracy becomes a property of the system, not of one model straining to be all things to all speakers.
For teams building globally, this is the mindset shift we push hardest: resist the urge to specialise the model to whichever accent failed in the last demo, reach for breadth, tune only for your domain, and let the system carry accuracy across every market you serve.