Help AI Features Admit Uncertainty

Ask an AI feature a question it cannot really answer, and most of them will answer anyway. Smoothly, in full sentences, with the same confidence they use when they are right. That moment, the one where the feature is out of its depth and does not show it, is where a customer quietly decides whether to trust the thing. It is almost always the part nobody designed.

The trouble is that the model underneath does not help you here. Left to its own habits, it would rather sound sure than admit it does not know.

AI can sound certain when it is wrong

There is a reason an AI feature tends to answer confidently even when it is wrong. The models underneath are trained and scored in a way that rewards a confident guess over admitting it does not know. Researchers at OpenAI made the point directly in 2025: the standard way models are marked hands out more credit for guessing than for saying "I am not sure", so the model learns to guess. The same finding has since been picked up in the wider research literature.

So a confident wrong answer is not a rare glitch in your feature. It is closer to the house style of the thing underneath it, and your design has to push against that grain rather than assume good behaviour.

This matters because a confident wrong answer is expensive in a way a visible "I am not sure" is not. When a customer catches the feature being sure and wrong even once, they do not just distrust that answer. They quietly stop trusting the next ten, including the ones that were right. A feature that shows the edge of what it knows keeps its credibility for the answers that are actually solid.

Show the source behind the answer

The first instinct, when you want a feature to look careful, is to bolt a confidence number onto every answer. Resist it. A "92% confident" label looks reassuring and tells the customer almost nothing they can use. Most people cannot read a confidence figure the way it is meant, and a precise-looking number implies a precision the model does not actually have. It dresses a guess up as a measurement.

What works better is showing where the answer came from. An assistant that says "your delivery page says orders placed before 2pm ship the same day", and shows that line, is doing something a percentage cannot: it lets the person check for themselves. When the answer is drawn from the customer's own content, the source is right there to be judged, and the customer does the last step of verifying it without being asked to. That is a far stronger signal of reliability than any badge, because it is checkable rather than asserted.

Not knowing is a useful answer

The harder state to design is the one where the feature genuinely cannot answer. The lazy version makes something up. The slightly less lazy version throws a red error that reads as if the feature is broken. Neither is what you want.

A feature that handles this well treats "I do not have that" as a normal, useful reply rather than a failure. It says what it cannot do in plain words, and it points somewhere real: a page, a phone number, a person who can help. The aim is for the customer to leave that moment with a next step, not a shrug and not a fabrication.

In our own online art shop, Modern Printworks, the on-site assistant answers from the real product catalogue. The decision that takes the most care is not how it answers when it can. It is what happens when a shopper asks about something the catalogue has no data for, a bespoke framing option, a price that does not exist yet. The feature is built to say it does not have that and move the shopper towards someone who does, rather than invent a confident spec the gallery then has to honour or apologise for. Across the AI features we ship, that is the pattern: show the source when there is one, and step back to a person at the edge of what the feature can stand behind.

Explain what makes the answer uncertain

There is an over-correction that does almost as much damage as false confidence: drowning every answer in disclaimers. A feature that ends each reply with "AI can make mistakes, please verify everything" has taught the customer to ignore the warning by the third time they see it. A warning that is always on is the same as no warning at all.

Doubt is only useful when it is specific. Flag the part of the answer that is shaky, the figure that came from a weaker source, the detail the feature is inferring rather than reading, and leave the solid part alone. "The opening hours below are from your main site; the bank holiday times I am less sure about" is worth ten generic disclaimers, because it tells the customer exactly where to look twice. Specific doubt reads as care. Blanket doubt reads as a legal footer.

Good design cannot remove model uncertainty

Here is the limitation worth being clear about, because it shapes everything above. You cannot get a reliable sense of certainty out of the model on its own. It can be completely sure and wrong, and quietly right about something it hedged. Any design that promises a calibrated confidence reading is promising something the technology cannot really back.

The levers that do work sit around the model, not inside it: what it is allowed to answer from, how narrow its job is, and whether the source is visible. A tightly scoped feature that answers from one catalogue knows where its knowledge ends. A general assistant pointed at everything does not, which is a large part of why the narrow ones are easier to trust. If you want a feature to behave well at the edge of what it knows, give it a clear edge in the first place.

Give uncertain answers a safe next step

All of which comes down to one question worth answering before any AI feature gets built: what does it do when it does not know? If the real answer is "it answers anyway", you have a demo, not something to put in front of customers. If you can describe the not-sure state as clearly as the good one, where it shows its working, where it declines, and where it hands over, then the feature is actually designed rather than just demonstrated.

We build these features inside the same applications we host and support, so we are the ones a customer reaches when an answer was confidently wrong. It is hard to treat the "I am not sure" moment as an afterthought when you are the one who has to make it right afterwards. If you are adding an AI feature to your site or product and you want a second pair of eyes on how it behaves when it is unsure, that is the sort of thing we are happy to look at on a short call.