Explainers · Explainer · Part 2 of 6
What is a voice agent?
Software that listens, thinks and talks on a phone line or a microphone. How the pieces fit, what it can do for a business, and the rules for calling anyone.
In brief
- A voice agent chains three pieces: speech to text, a language model, and text to speech, connected to a phone line.
- It works best on calls with one right answer: hours, bookings, messages, reminders people asked for.
- It should say it is automated, hand off to a person on request, and only call people who agreed to hear from you.
A voice agent is an AI assistant you talk to out loud. On a phone line, it answers the call, understands what the caller wants, replies in a natural voice and does something useful: books an appointment, takes a message, answers a question from your approved information, or passes the call to a person.
How it works
Under the hood there are three pieces and a connection to the phone:
- Speech to text. The caller's words are turned into text as they speak.
- A language model. It reads the text, decides what to do, and writes a reply. If it needs to check a calendar or look up an order, it calls a tool.
- Text to speech. The reply is spoken back in a chosen voice.
Around those pieces sits a lot of timing work. The assistant has to know when the caller has finished, stop instantly when interrupted, ignore a "mhm" or a cough, and not mistake its own voice coming back through a speakerphone for the caller. We spent weeks on exactly these problems building voice for our own assistant, and wrote up what we learned. They matter more than how pretty the voice sounds.
What it can do for a business
- Answer when you cannot. After hours, at lunch, during a rush. It takes the details so nobody is lost to voicemail.
- Handle the repeated questions. Hours, directions, what you offer, whether you take walk-ins.
- Book and reschedule. If it is connected to your real calendar.
- Summarise every call. Who called, what they wanted, what was done, in a note you can read in seconds.
- Make calls people asked for. Appointment reminders, "your order is ready", a follow-up on a quote the customer requested.
Exhibit 1Good first jobs, and jobs to keep for people
| Give it | Keep for a person |
|---|---|
| Opening hours and directions | Complaints and upset callers |
| Simple bookings and changes | Anything that needs a diagnosis or advice |
| Taking a message after hours | Prices for custom work |
| Reminders the customer agreed to | Payments over the phone |
Picture a call to a plumbing company at nine at night. The assistant answers with the company name and says it is the automated assistant. The caller has a leaking water heater. The assistant asks for the address and a callback number, reads the number back, explains that a technician will call first thing in the morning, and offers the emergency line if water is spreading. Two minutes later the owner has a text summary. Nothing in that call needed judgment the owner had not written down in advance.
The rules for calling people
Answering your own line is one thing. Calling out is another. Calls made with an AI or artificial voice need the person's consent first under federal rules, and many states add their own. So we only build outbound calls to people who asked to hear from you: existing customers who booked, ordered or requested a quote. We do not do AI cold calling, and we turn down projects that need it.
Recording has rules too. Florida, for example, requires everyone on the call to agree to a recording, so a recorded line should say so at the start. The assistant should also say plainly that it is automated. This is a plain-language summary, not legal advice; check with your attorney for anything regulated.
A good voice agent knows two things: what it is allowed to say, and when to get a person.
What to have ready
Most of the setup is writing down what your best receptionist already knows: the common questions and their answers, your booking rules, which calls go straight to a person, and what the assistant must never say. The better that sheet, the better the agent. Our guide to what an AI receptionist should handle walks through it.
When you are ready to build, see phone answering agents for inbound calls, after-hours and overflow for the cautious first step, and reminder and follow-up calls for outbound calls with consent.
