What happens when an AI chatbot cannot answer a question is the most important thing about it, and there are only three possibilities. It invents something plausible. It quietly pushes the visitor at a contact form without explaining why. Or it says it cannot support an answer from your content and hands the question to a person with the conversation attached. Only the third is acceptable, and which one you get is a product decision rather than a limitation of the model.
The mechanism behind the third is a confidence floor: every retrieved passage comes back with a similarity score, and the system declines below a threshold instead of generating anyway.
What are the three things it can do?
Three behaviours, and every product on the market picks one as its default. Knowing which one you have bought takes thirty seconds and almost nobody spends them.
| Behaviour | What the visitor sees | What it costs you |
|---|---|---|
| Invent | A fluent, confident, wrong answer | A return, a complaint, and a customer who no longer believes you |
| Silently deflect | A contact form, no explanation | Mild annoyance and a support ticket you could have avoided |
| Decline and hand over | An honest sentence and a route to a person | Nothing, and usually some goodwill |
The first row is the expensive one and it is expensive in a specific way. A wrong answer does not fail immediately; it succeeds, the customer acts on it, and the cost arrives a week later as a return, a chargeback or a review. By then nobody connects the two.
The second row is underrated as a failure. A deflection with no explanation reads as the system not wanting to help, when what actually happened is that it could not. One sentence of honesty converts the same outcome into a reasonable experience.
How does it know it does not know?
By scoring, not by understanding. This is worth being precise about, because the marketing language around it suggests something more mysterious than what happens.
Your pages are split into passages and each one is converted into a numeric representation by an embedding model. When a question arrives it is converted the same way, and the system measures how close the question is to each passage. What comes back is a ranked list with a similarity score attached to every entry.
That score is the whole basis for knowing. If the best passage scores well, your content probably contains the answer. If the best passage scores poorly, nothing on your site is close to the question, and any answer generated from those passages will be built from the model’s general knowledge rather than from you.
A confidence floor is a threshold on that number. Above it, answer. Below it, decline. OpenAI’s retrieval guide covers the mechanism if you are assembling this yourself.
Setting the floor is a real trade rather than a solved problem. Too high and it refuses questions it could have handled, which reads as unhelpful. Too low and it invents, which is worse. It should be visible and adjustable, and a product that cannot tell you what its floor is almost certainly does not have one.
What should a good refusal say?
Four things, and most refusals in the wild get two of them at best.
| What the refusal should do | The bad version | The good version |
|---|---|---|
| Name the limit honestly | “I’m not sure I understand” | “I can’t find that on this site” |
| Offer a real route | A dead end, or a generic form | A named next step, with the context carried |
| Keep what was said | The customer starts again | The conversation travels with the handoff |
| Stay in the visitor’s language | An English fallback on a Dutch page | Refusing in the language they asked in |
Row one matters more than it looks. “I’m not sure I understand” blames the customer for a phrasing problem they do not have, and the honest version, that the answer is not on this site, is also more useful: it tells them to ask a person rather than rephrase.
Row three is the difference between a tolerable experience and the worst one this category produces. Somebody who has explained their situation and is then asked to explain it again has been made worse off than if there had been no chat window at all.
Row four gets missed on multilingual sites constantly, because the fallback message is usually written once, in English, by somebody who was thinking about the happy path.
How do you test refusal before launch?
By asking questions your site genuinely does not answer, which is a deliberate act because every instinct pushes you to test the things it does.
Be plausible rather than absurd. Asking a chatbot the meaning of life gets a refusal from almost anything, which tells you nothing. Ask whether you ship to a country you do not ship to. Ask about the warranty on a product line you do not carry. Ask for a discount code that does not exist. These are questions a real customer would ask and your site does not answer, and that is exactly the gap where invention lives.
Run five of those, and score each one on three things. Did it decline. Did the refusal say something useful. Did it offer a person. A system that passes all three on all five is safe to put in front of customers on day one, whatever else is imperfect.
There is a second reason to care about refusals beyond avoiding harm: they are the most honest content roadmap you will ever get. Every refusal is a customer who asked something your site does not cover, recorded with their exact wording. Export a month of them, group them, and the top three groups are the pages to write next, ranked by real demand rather than by keyword tools. Most teams treat the refusal log as a failure report and delete it. It is closer to a research budget somebody else paid for, and the shops that read it stop having the same refusals within a quarter.
Then do it again after every content change, because the floor’s behaviour shifts as your index grows. A question that had no good passage in March may find a mediocre one in June, and mediocre is where the trouble lives. You can try ours on the demo, and the mechanism is written up on how it works.
Why do most products answer anyway?
Because refusing looks bad in a demo and in a dashboard, and both of those are what products are optimised for.
A demo is a controlled question asked of prepared content. Everything answers well. Nobody in a sales call asks the thing the site does not cover, so the behaviour that matters never gets seen, and a product that declines honestly in that moment looks weaker than one that improvises.
The dashboard problem is worse because it persists after purchase. If deflection rate is the headline number, every refusal looks like a failure, and the pressure is to tune the system to be more confident rather than more correct. That is a direct incentive to lower the floor, and lowering the floor is invisible in every number anybody reports.
Some of this is also just age. Plenty of chat products grew out of scripted flow builders and live chat, where there was no model and no invention was possible, and bolted a model onto the fall-through case. That architecture has no natural place to put a floor. It is not dishonesty, it is history, but it means the question is worth asking of any product regardless of how modern the interface looks. The market context is in 826,000 WordPress sites and almost no AI, and the free option most people evaluate first is AI Engine, at 100,000 installs on the plugin API.
When is answering anyway the right call?
Rarely, but not never, and pretending otherwise would be its own kind of dishonesty.
Two cases hold up. When the answer is genuinely general knowledge and carries no cost if it is slightly off, explaining what a size chart is, or what a tracking number looks like, a sensible general answer is better than a refusal. The test is whether being wrong costs anybody anything.
And when a partial answer plus an honest gap is more useful than nothing. “We ship within the EU, and I cannot confirm whether that includes Norway, so let me pass you to someone” is better than silence, because it moves the customer forward while being clear about the edge.
What does not hold up is the tempting middle: answering with a hedge like “generally” or “usually” to cover a gap. Hedged invention is still invention, and the hedge is read as politeness rather than as uncertainty. If the passage is not there, say the passage is not there.
Worth noting that products with human live chat attached, like Tidio, have an easier version of this problem: a refusal has somewhere obvious to go. If you have no humans behind the widget, your refusal has to be much better written, because it is the whole experience rather than a transition. For the category overall, start with what an AI chatbot for a website actually is.
What owners ask us most
What happens when an AI chatbot cannot answer a question?
One of three things. It invents something plausible, which is the common default and the worst outcome. It quietly redirects to a contact form without saying why, which frustrates people. Or it says it cannot support an answer from the site’s content and hands the question to a person with the conversation attached. Only the third is acceptable, and which one you get is a product decision rather than a model limitation.
Why do AI chatbots make things up?
Because generating text is what a language model does, and nothing stops it when retrieval comes back empty. Ask a question your site does not cover and the retrieval step returns weak passages; unless something checks how weak they are, those get sent to the model anyway and it fills the gap from general knowledge. The invention is not a malfunction, it is the default behaviour of a system with no floor.
What is a confidence floor?
A threshold on how well the retrieved passages match the question. Every passage comes back with a similarity score, and a floor says that below a certain score the system declines instead of generating. Setting it is a trade: too high and it refuses things it could have answered, too low and it invents. It should be visible and adjustable, and a product that will not tell you it has one probably does not.
How do I test whether a chatbot will invent answers?
Ask it something your site genuinely does not cover, and be specific rather than absurd. Not the meaning of life, but whether you ship to a country you do not ship to, or the warranty on a product line you do not sell. Absurd questions get refused by almost anything. Plausible questions your content does not answer are where a system either declines honestly or invents fluently.
Is it bad if the chatbot refuses a lot?
It depends entirely on what it refuses. Frequent refusals on questions your site does answer means retrieval is misconfigured or your content is unfindable, and that is worth fixing. Frequent refusals on questions your site does not answer is the system working correctly and telling you which pages to write. Read a sample before treating the rate as a problem.
Should the chatbot always cite its source?
Yes, and it is the cheapest quality mechanism available. An answer with the page attached can be checked in one click by the visitor and by you; an answer without one is a claim you have to take on trust. Citation also constrains the system: a product that has to name the page it used is a product that cannot comfortably answer from nowhere.
