Your support bot is confidently telling customers the wrong thing, and the instinct is to fix the bot. Change the model, rewrite the system prompt, add a rule telling it not to make things up.
That almost never works, because the bot isn't the problem. It isn't inventing out of nowhere. It's answering from your articles, and where they're silent, it guesses. It found a section that didn't say what it left out, and it filled the gap the way anything does when the answer is missing.
So the fix is upstream, in the content. Here's the order.
See it
Collect what it's actually getting wrong
Not what you think it gets wrong. Pull about 20 real conversations, mostly ones where the answer was bad, plus a few at random from the ones that looked fine, and write down the question, the answer it gave, and what the answer should have been. Guessing at this stage sends you off fixing the wrong articles.
Diagnose it
Find the cause, which is one of 3 things
Nearly every wrong answer comes from the answer not being in any article, or being in an article whose section can't stand on its own, or 2 articles saying different things. Each has a different fix, and until you know which one you're looking at you'll fix the wrong thing.
Fix it
Check that your names aren't the problem
If your product calls something a workspace, your docs call it an account, and your billing page calls it an organization, your bot has to guess which one a customer means. It usually guesses wrong, and this cause is invisible until you look for it.
Keep it fixed
Stop it drifting back
Content fixed once goes wrong again with the next release. A bot that's right today and unreviewed in 6 months is a bot that's wrong again, and nobody will tell you.
If you only do one thing this week, do the first stage. About 20 real conversations will tell you more about your help center than any audit, and it costs you an afternoon. Reading what your assistant tells people is taught in Course 05, Part 8, and the pages written for the assistant alone in the course Content nobody is supposed to see.
Start with what it's getting wrong →