Building a Knowledge Base Your AI Phone Agent Can Use

When an AI phone agent gives a wrong answer, the instinct is to blame the model. Usually the real cause is simpler: the agent was never told the right answer, or was told something vague that it filled in on its own. The knowledge base, the collection of facts and answers the agent draws on, decides most of what your callers hear.

Here's how to build one that works on the phone.

Start from real questions, not your website

Your website FAQ is written for people browsing, and it often answers questions nobody calls about. Start from what callers actually ask:

• Pull a few weeks of call reasons from your phone system or CRM
• Ask the people who answer the phones for the top 20 questions they hear
• Read a sample of call recordings or transcripts if you have them
• Check support emails and chat logs for questions that would also come by phone

Group the questions by topic and count them. The top 20 to 30 usually cover most routine calls.

Write answers the way you'd say them

Answers will be spoken, so write them for the ear:

Before (website style):

Our standard business hours are Monday through Friday from 8:00 AM to 5:00 PM, with extended Saturday availability from 9:00 AM to 1:00 PM, excluding public holidays (see holiday schedule).

After (spoken style):

We're open Monday to Friday, 8 to 5, and Saturday mornings from 9 to 1. We're closed Sundays and public holidays.

Rules of thumb:

• Short sentences, one fact per sentence
• Put the direct answer first, details after
• Avoid tables, links and "see above"
• Write numbers and times the way they're said

Spell out conditions

Vague answers produce confident guesses. Compare:

Vague:

We offer free delivery.

Specific:

Delivery is free on orders over $50 within 25 miles of our store. Under $50, or farther away, delivery is $15. We don't deliver on Sundays.

If the answer depends on something, say what it depends on and what to do when the agent can't tell. "Pricing depends on the size of the job. The agent should not quote prices for repairs and should offer to book an estimate instead."

Say what the agent doesn't know

This is the most overlooked part. List the topics the agent should not answer, and what it should do instead:

• Don't quote prices for custom work. Offer to book a free estimate.
• Don't give medical, legal or tax advice. Offer a callback from staff.
• Don't confirm stock levels. Offer to have the store call back.
• If a question isn't covered here, say you'll pass it to the team and take a message.

An explicit "I don't know, here's what happens next" is far better than an improvised answer. We cover this in how to write a call script an AI agent can follow.

Structure it simply

You don't need special software to start. A shared document or spreadsheet works, with one entry per topic:

The owner and date columns matter. Information goes stale, and someone has to be responsible for each piece.

Separate stable facts from live data

Some information changes constantly: appointment availability, order status, account balances. That doesn't belong in a knowledge base. The agent should look it up live through an integration with your systems at the time of the call.

The knowledge base holds what's stable for weeks or months: policies, hours, services, prices for standard items, procedures.

Keep it current

A knowledge base is never finished. A routine that works:

• Weekly at first: read transcripts of calls where the agent said "I'll pass that to the team" or transferred. Each one is a candidate new entry.
• When anything changes: hours, prices, staff, policies, holiday closures. Add "update the agent" to the checklist for these changes.
• Monthly: each owner reviews their entries and updates the last-checked date.

Holiday hours deserve special mention. They're the most common source of wrong answers because they're temporary and easy to forget.

Test it like a caller

After building or changing the knowledge base, test it:

• Ask each common question in two or three different ways
• Ask follow-ups ("what about Saturdays?" after asking about hours)
• Ask questions that are deliberately not covered and check the agent declines properly
• Ask about edge cases in your conditions ("I'm 30 miles away and my order is $60")

You can try this kind of test on a language model using our free AI chat tool: paste a few entries as instructions, then ask questions as a caller would.

Example entries

Good knowledge base entries are short, specific and written for speech. A few examples for a veterinary clinic:

Topic: Opening hours Answer: We're open Monday to Friday, 8 to 6, and Saturday 9 to 1. We're closed Sundays. Conditions: For emergencies outside these hours, give the after-hours emergency hospital number: 555-0144. Owner: Practice manager

Topic: New patient visits Answer: New patient exams take about 30 minutes. Please bring any previous vaccination records. If you don't have them, we can request them from your previous vet with your permission. Conditions: Book with any vet. Exotic pets (reptiles, birds) only with Dr. Lee. Owner: Front desk lead

Topic: Prices Answer: A standard wellness exam is $65. Vaccines and other treatments are extra, and the vet will go over costs before doing anything. Conditions: Don't quote prices for surgery, dental work or treatment plans. Offer a callback from the front desk. Owner: Practice manager

Notice how each entry tells the agent what to say, and also what not to say.

Handling conflicting information

Conflicts creep in when several people maintain the knowledge base or when it's assembled from different sources. A website says you close at 5, a policy document says 6. The agent may say either.

Prevent this by:

• Having one source of truth per topic. If hours live in the knowledge base, the website should be updated from it, not the other way around.
• Searching for duplicates before adding a new entry.
• Reviewing changes with the topic owner before publishing.
• Testing after updates by asking the agent the questions you just changed.

When to use tools instead of text

Some questions sound like knowledge base material but are better answered by a live lookup:

• "Do you have any appointments Thursday?" (calendar)
• "Is my prescription ready?" (pharmacy system)
• "What's my balance?" (billing system)
• "Is the store open on the holiday?" (could be either; if it changes often, use a tool)

If the answer changes more than once a month, consider whether a live data source is better than an entry someone has to remember to update.

Measuring knowledge base quality

Track a few simple numbers each month:

• Unanswered questions: calls where the agent said it didn't have the information
• Wrong answers found in review: with the entry that caused them
• Stale entries: entries whose last-checked date is older than your review cycle
• Coverage: share of call reasons with a matching entry

Unanswered questions are the best source of new entries. Wrong answers point to entries that need rewriting. Stale entries point to an ownership problem.

Frequently asked questions

How big should the knowledge base be?

Big enough to cover the questions callers actually ask. For a focused agent, 30 to 100 entries is common. Quality matters far more than size.

Can I just give the agent my whole website?

You can use it as a source, but expect problems: outdated pages, marketing language and contradictions. A curated knowledge base gives much more reliable answers.

Who should own the knowledge base?

One person should coordinate it, with a named owner for each topic area. Without clear owners, entries go stale.

Written by the Voxvencer editorial team. We build and run AI voice agents for call centers and small businesses, and we write about what we see on real phone lines. Questions or corrections: info@voxvencer.com.