The problem
Moving a voice assistant onto one device puts speech recognition, language-model inference, and speech synthesis on the same hardware. The conversation can stay local, but the application now owns memory contention, turn detection, and the ability to interrupt a response.
I built EdgeVox around that constraint. This case study distills the decisions and failure modes in my firsthand build account. It focuses on the voice pipeline rather than repeating the project's full feature list or claiming a new benchmark.
A streaming system with a separate stop path
The normal path moves from microphone audio to a transcript, then through a local language model and speech synthesis. Those stages must cooperate while competing for the same machine. Synthesizing a completed sentence while the model continues generating avoids waiting for the entire answer before producing speech.
This is a conceptual view of the voice loop, not a timing diagram. For a robot, the reactive stop path must also remain independent of deliberation: stopping an action cannot depend on a language model composing a reply. That separation is an architectural choice, not a certification of safety on any particular robot.
Decisions and alternatives
Cloud APIs would move much of the resource management outside the device, at the cost of network dependence and sending conversation data elsewhere. Keeping the pipeline local means selecting components against the combined memory budget, not picking each model independently for its quality score.
I kept speech and model backends behind interfaces so configurations could change without rewriting the conversation loop. The ROS2 bridge stays optional. It lets robotics integrations use topics and actions while a desktop installation avoids inheriting the entire robotics stack.
Cancellation has to reach the decoder as well as the speaker. Stopping playback alone leaves the language model consuming resources for a reply nobody will hear. The implementation described in the build account passes cancellation through to model generation and rearms the interrupt path for the next turn.
The failures that informed the design
Voice activity detection can end a turn too soon or leave an awkward pause. The agent's own speech can also be mistaken for new input, which is why echo handling matters outside a quiet demonstration.
A separate transcription-cleanup model was another lesson from the build: better results for an isolated stage did not translate into a better composed pipeline. The extra stage introduced incompatible behavior at the boundary. Evaluating the full chain mattered more than the standalone score.
What shipped, and what remains conditional
EdgeVox provides an open-source local voice pipeline with an optional ROS2 bridge. The repository documents the current components, configuration, and examples. The useful result is an inspectable system whose stages and cancellation behavior can be adapted to a target device.
Responsiveness depends on hardware, model choices, prompt length, audio conditions, and whether the models are already loaded. This case study makes no universal latency claim. A credible evaluation should report the complete configuration, measure time to first audio, and exercise repeated interruptions as well as an uninterrupted answer.
The lesson
An offline assistant needs the discipline of a real-time application, even when its language model is probabilistic. Make resource ownership and cancellation explicit, keep optional integrations optional, and evaluate the conversation people actually experience.
Continue with the full EdgeVox build account and the measured comparison of small-model agent strategies.