“Tell me this month’s sales.” The AI says it will check, then goes quiet. A few seconds later, the user asks again: “Are you still there?” The same query may now run twice, and the earlier result might start playing over the newer question.
A good voice agent needs more than a natural-sounding voice. It must listen when the user speaks, explain necessary waits, and call a task complete only when completion is verified. The same challenge appears in language practice, customer support, work briefings and voice-controlled creative tools.
This design guide draws on official technical documentation checked on October 23, 2026. It is not a measured product review or a case study of an application we deployed. The six images—the cover and five illustrations—are generated conceptual visuals, not screenshots of a working service or performance measurements.
1. The first reply and task completion run on different clocks
Google’s October 19 ADK guide explains how an agent can keep a conversation going while a slow tool runs. In its sales-report demonstration, Google reports reducing the time to the first spoken words from 14.4 seconds to 1.2 seconds. Those are vendor-reported results from a particular configuration—not evidence that the underlying query finished that much sooner. Backend processing time remains a separate measure.
Without that distinction, a system that quickly says “I’ll check that for you” can be mistaken for one that completes work quickly. Measure separate intervals from the end of the user’s utterance to acknowledgment, evidence of actual progress, and the verified final result. Do not collapse them into one response-time number.
The initial acknowledgment should say what the agent understood and what it intends to do. It must not claim that a query is already running when it has not started, or announce success merely because a calculation is in progress. A single statement of verified status is more useful than small talk that conceals a loading delay.
2. Separate the conversation path from the work path
ADK’s live-tool documentation distinguishes ordinary tools, background execution and streaming tools that provide intermediate results. If tool execution blocks the conversation, a prompt telling the agent to “keep talking helpfully” cannot fix the architecture. The application needs an actual event and execution design.
A useful mental model is to give one path responsibility for receiving the user’s questions and another for queries, transformations or generation. These paths do not necessarily mean two products or two models. A single application can maintain conversation state and task state separately.
The table below is AIZIGOO’s proposed minimum state ledger. It is an operating agreement for developers and product teams, not an API schema ready to implement.
| State | What to tell the user | Evidence to retain |
|---|---|---|
| Accepted | Understood request and scope | Task ID and request version |
| Running | Current verified stage | Execution events and stage timestamps |
| Checking | Result received; validation pending | Original result and check records |
| Complete | Verified result and remaining limitations | Result ID, sources and completion time |
| Canceled or unresolved | Whether work stopped, or why status is unknown | Cancellation response and current server state |
If the user changes a condition, increment the request version. Do not present the old result as an answer to the revised request. When a question is repeated, first decide whether to check the existing task or create a new one. For actions that change external state, such as sending a message or making a reservation, linking the task ID to an idempotency key can help prevent duplicate execution.
3. Report progress only when something actually changes
Repeating “Still processing” every few seconds does not build trust. Useful updates are tied to real events: a file opened, a query returned, or a result awaiting validation. If progress cannot be quantified, do not invent a percentage. State the verified stage and when the next status check will happen.
When to speak a result is another design decision. ADK describes WHEN_IDLE for delivery during a natural speaking gap, INTERRUPT for an immediate response, and SILENT for updating context without announcing it. Check support against the model, SDK and session configuration. These are not interchangeable controls available in every voice application.
| Type of result | Suggested delivery | What to avoid |
|---|---|---|
| Routine report or lookup | After the user finishes speaking | Reading figures over the middle of a sentence |
| Failure that needs immediate attention | Briefly signal its importance, then explain | Continuing the conversation while hiding the failure |
| Information needed only for a later answer | Update state quietly | Forcing an unrelated intermediate result into the conversation |
| Speaking or interview practice | Leave room for the user to think | Treating every silence as a failure |
These recommendations are AIZIGOO’s design proposals, not measured comparisons of product defaults. Natural interaction does not mean filling every gap. It means intervening when the intervention is useful.
4. “Wait” and “Cancel that task” are different requests
An interruption can mean two different things. “Wait, I have another question” may ask the agent to stop audio playback. “Cancel the report generation” asks it to stop background work. Handling the first does not prove that the second has happened.
Google’s October guide explicitly states that a user interruption does not automatically cancel a running tool. A separate cancellation path is needed, such as stop_streaming for an ADK streaming tool. Even that does not mean an action already submitted to an external server—or a message already sent—has been reversed. The application must check the resulting state.
OpenAI’s Realtime interruption and truncation documentation also addresses keeping the conversation history consistent with the part of a response the user actually heard. Responsibilities differ between WebRTC or SIP and WebSocket connections. Stopping the sound alone is not the whole process. If the agent assumes the user heard an explanation that never played, its next answer can lose context.
Keep audio playback state separate from work state. When scope is unclear, the agent can ask once: “I’ve stopped the explanation. Should I stop the lookup too?” If the cancellation response has not arrived, say “I’ve requested cancellation and am checking the status,” not “Canceled.”
5. A fluent voice is not a substitute for authorization
“Yes, go ahead” means different things depending on the preceding explanation. Consent to a lookup must not be expanded into permission to send an email or confirm a reservation. Before taking a consequential action, show the target, scope and important changes—such as the amount or recipient—and obtain confirmation for that specific action.
ADK’s guardrails documentation covers independent checks at the input, response and tool stages, alongside instructions. Because speech unfolds in real time, do not assume that checking a transcript afterward prevents every problematic utterance before it is spoken. Put a separate authorization and parameter-checking boundary before tool execution.
Begin a pilot with simulated orders, sample documents or read-only lookups. Tell users whether recordings or transcripts are retained and explain the retention policy. Keep only the minimum information necessary. Verify the requester and task scope so that a background voice on a call, or text embedded in an external document, is not mistaken for an instruction to act.
6. A 30-minute pilot: test state, not just the voice
The following is a small evaluation procedure proposed by AIZIGOO. We did not call a live voice API to execute these tests for this article, and this is not an official benchmark. Reading the same script in text alone will not reveal microphone, speech-detection or overlapping-speech failures. ADK’s voice evaluation documentation likewise emphasizes reviewing the actual audio path and the conversation as a whole.
Use the first five minutes to fix one read-only task and its expected outcome. Spend the next 15 minutes repeating the scenarios below on the same device and network. In the final ten minutes, compare consented conversation recordings with task logs and classify the failures. Where practical, try the same script separately in Korean, English and Japanese. Equivalent translations do not guarantee equivalent voice behavior.
| Test scenario | Evidence of a pass | First place to investigate a failure |
|---|---|---|
| Deliberately slow lookup | Acknowledgment and final result remain distinct | Task-start and result events |
| User speaks over the agent | Audio stops and the new question is accepted | Speech detection and playback buffer |
| User requests task cancellation | Server state confirms whether work stopped | Cancellation path and follow-up requests |
| User changes a condition while waiting | Old output is not mistaken for the new answer | Task ID and request version |
| Retry after a disconnection | Current state checked without duplicate execution | Result lookup and idempotency |
| Long silence while the user thinks | No unnecessary pressure to answer | Conversation mode and end-of-turn detection |
Rather than recording a single average response time, track first-response and final-completion time distributions separately. Also count incorrectly interrupted utterances, duplicate executions and false completion announcements. Cost per successful conversation should include retries, external tools and human review, not just audio and model calls. With a small sample, use the evidence to fix one recurring failure rather than publish a model ranking.
A good voice agent knows when to stay quiet
Natural conversation requires more than a quick opening line. The agent needs to distinguish listening from speaking, set aside an outdated answer when the task changes, and never claim a task was canceled until cancellation is confirmed. Instead of starting with an all-purpose assistant, design an agent that clearly accepts, progresses and verifies one read-only task.
Official sources checked
- Google Developers: Step-by-step guide to building a non-blocking voice agent, October 19, 2026
- ADK: Tools for live agents
- ADK: Guardrails for live agents
- ADK: Evaluation for live agents
- OpenAI: Realtime interruption and truncation
Sources were checked on October 23, 2026. The developer API and ADK concepts in this article should not be confused with settings in consumer applications. Before implementation, recheck the relevant service’s latest documentation for model support, SDK versions, account access, data handling and billing.