Real-time voice AI has a cost problem that batch AI does not. A summarisation tool runs when you ask it to; a live call assistant runs continuously, for every participant, for the entire length of every conversation. Get the cost architecture wrong and the bill grows in lockstep with your success. Get it right and cost grows with value delivered, not with raw minutes of audio -- and the difference is a design decision, not a vendor discount.
Know where the money actually is
Teams usually fear the wrong cost. There are two very different drivers in a live-AI stack. Transcription is continuous -- it never stops while a call is live -- but it is cheap per unit. Language-model understanding is occasional -- it fires on specific moments -- but expensive per call.
The instinct is to worry about transcription because it runs constantly. The reality is that a large model called continuously on long context is what quietly dominates the bill at scale. If your cost grows with every minute of audio, you are paying for silence, small talk and everything that never needed a model.
The principle: decouple cost from audio-minutes
The single most important cost principle for live AI is to put the continuous work on cheap, owned compute and put the expensive work behind triggers, so it fires occasionally rather than every second. Continuous transcription runs locally or at the edge, where the marginal cost of another minute is essentially zero.
The heavy language model runs only when a moment warrants it, so spend scales with meaningful events rather than with the clock. This is the same insight that drives the two-speed architecture, viewed through the lens of money instead of latency.
Why full-cloud is fine to start and painful to scale
A fully cloud-based stack is the right way to begin: zero infrastructure, elastic, and genuinely cheap at low volume, with some transcription services costing only cents per hour. The problem is the shape of the curve, not the starting point. Per-minute pricing scales linearly and never bends.
At a handful of users it is negligible; at a sales floor running thousands of concurrent hours a month, the same linear curve becomes the largest line in the budget, precisely when you can least afford surprises.
A hybrid design changes the shape. Local or edge transcription turns the continuous cost into a fixed one you have already paid for. The only variable cost left is the occasional language-model call, and because it is trigger-gated it scales with the number of moments that actually needed intelligence, not with total talk-time. The result is a much flatter curve that stays affordable as volume grows.
Controlling the part that remains
Once transcription is cheap, the language model is the cost to manage, and the same techniques that reduce its latency reduce its cost: trigger-gating so it runs rarely, model tiering so most work uses a small cheap model, short outputs, and caching the stable parts of prompts. Cost and latency turn out to be the same engineering problem viewed from two angles, and the two-speed architecture solves both at once.
We design voice-AI economics before writing the pipeline, because the cost model is architectural. Decide up front what runs continuously and what runs on triggers, and you build a product that gets cheaper per call as it grows -- instead of one that quietly becomes unaffordable at exactly the moment it succeeds.