Skip to content

AI Implementation

How to Reduce AI Hallucinations: A Plain Guide.

JK
11 min read

AI sometimes makes things up. It sounds sure while doing it. You cannot switch this off, but you can cut it down a lot.

The fixes are simple. Give the AI good source material. Ask it to use that material. Let it say 'I don't know'. Test it often. Keep a human in the loop for anything that matters.

This guide walks through each one. No jargon without a plain explanation first.

TL;DR

  • AI hallucinates because it predicts likely words. It does not check facts.
  • The biggest fix is to give it your own trusted sources to answer from.
  • Tell it to say when it does not know. Reward that answer.
  • Test the whole system, not just the AI model. Keep testing after launch.
  • Every fix has a cost. Track how useful the answers are, not just how safe.

What a hallucination is and why it happens

A hallucination is a confident answer that is wrong or made up. Think of a new hire who hates saying 'I'm not sure'. So they guess. Sometimes the guess sounds great and is false.

That is close to how these models work. They are trained to predict the most likely next words. They are not trained to check what is true. Duke University Libraries explains that this is why models still hallucinate in 2026.

There is a second cause. Many tests used to grade AI reward a guess more than an honest 'I don't know'. OpenAI researchers argue that training and testing should reward models for showing uncertainty. So the habit of guessing gets built in.

A third cause is people-pleasing. Duke's experts call it sycophancy. The AI agrees with your idea, even when your idea is wrong. That makes a wrong answer sound like a good one.

What this means for you:

  • Better models help, but they do not end the problem.
  • You need a system around the model.
  • Think of it as a kitchen. The model is the cook. Sources, rules and checks are the recipe, the scales and the taste test.

Fix 1: Give the AI your own sources

This is the biggest lever for most businesses.

Without sources, the AI answers from memory. Memory is fuzzy. With sources, it answers from a page you gave it. It is the difference between an exam from memory and an open-book exam.

The technical name is RAG, short for retrieval-augmented generation (the AI looks up trusted documents first, then writes its answer). A study of cancer information chatbots found that RAG with reliable sources cut hallucination rates and helped the bots admit when they lacked information. The idea carries over to any field where you own the facts.

For a founder, those facts are your frameworks, your course material, your client notes and your policies. Your IP (intellectual property, the know-how only you have) is the best source you can give.

Tips that make it work:

  1. Use clean, current documents. Old files give old answers.
  2. Remove contradictions. If two docs disagree, the AI will pick one.
  3. Split long files into clear sections with titles.
  4. Cover the questions people really ask. Gaps invite guessing.

RAG is not magic. A 2026 review of hallucinations lists it as one tool among many for catching and fixing them. A tool is only as good as what you feed it. So the quality of your source library matters as much as the tool.

Want the build side? Devwiz wrote a plain walk-through of how to build a RAG pipeline. We cover how to organise the material itself in knowledge architecture for AI.

This is also where Claude Code earns its place. Your sources can sit as plain files in a project folder, and Claude Code reads them before it answers. No guessing from memory.

Not sure whether to look things up or retrain the model? We wrote a plain comparison in RAG vs fine-tuning.

Fix 2: Write prompts that point to sources

A prompt is the instruction you give the AI. Small changes in the wording change the results.

One simple trick is 'according to' prompting. You tell the AI to answer according to a named source. Researchers found this can raise how much the answer overlaps with chosen sources by 5% to 105%, and sometimes it improves the final result too.

Here is how that looks in practice.

Weak prompt: 'What is our refund policy?'

Better prompt: 'Using only the refund policy document below, explain our refund policy. Quote the line you used.'

More habits that help:

  • Give one clear job per prompt.
  • Say what to do when the answer is missing.
  • Ask for the source of each claim.
  • Ask for the answer in a set format, such as a short list or a table.
  • Give a good example of the answer you want.

Think of it like briefing a new team member. Vague brief, vague work. Clear brief, clear work.

Fix 3: Let the AI say 'I don't know'

This one feels odd. But it is one of the most useful rules you can add.

A good system says when information is missing or confidence is low. It also leaves out any claim it cannot link to a source. Meilisearch's documentation says the same.

Put this in your instructions:

  • If the sources do not answer the question, say so.
  • Do not guess to fill the gap.
  • Point the person to a human or a next step.
  • Give the source for every fact.

Watch out for the people-pleasing habit too. Tell the AI to push back when your question has a wrong assumption. Do not let it just nod along.

When to hand over to a person

Some questions should never be left to the AI alone. Here are good triggers for a handover:

  • Money, refunds or contracts
  • Health, legal or safety topics
  • An upset or high-value client
  • Anything the sources do not cover

We go deeper on this in how AI systems handle edge cases and in our guide to AI agent guardrails.

Fix 4: Narrow the job and the format

A wide-open question invites wide-open guessing. A narrow job leaves less room to drift.

Compare these two jobs:

  • 'Tell me about our clients.' Too wide.
  • 'List the three goals in this client's intake form.' Narrow.

Some more ways to narrow the work:

  1. Split big tasks into small steps. One step finds the facts. The next step writes them up.
  2. Use set formats. If the answer must fill fixed fields, there is less room to invent.
  3. Give the AI the right tools. A calculator for maths. A database for numbers. Do not ask it to remember what a tool can look up.
  4. Pick the right model for the job. Test a few on your own real questions. See our guide to LLM selection.

The kitchen rule applies again. A cook with a measuring jug does not guess the flour.

Fix 5: Keep a human in the loop

No setup is perfect. So decide up front which answers a person must see.

A good rule is to match the check to the risk:

Risk levelExampleCheck
LowDraft social postSpot check now and then
MediumClient emailPerson reads before it sends
HighRefund, legal or medical answerPerson approves every time

This is also the advice in schools. US guidance says AI should support human judgment, and people should review AI output against trusted sources before sharing it. That is a good habit in any business.

People need clear steps for this, or they skip it. Write down who checks what, and how fast.

How to test whether your fixes work

You cannot fix what you do not measure. Many teams skip this step. Do not.

Here is a plain way to do it.

Step 1: Build a test set

Collect real questions with good answers. Microsoft's guidance says to build a representative set of trusted question and answer pairs. Use human-checked answers. Add made-up examples only when you lack real ones.

Start with 30 to 50 questions. Include the tricky ones.

Step 2: Set a baseline

Run the test before you change anything. Write down the score. Now you have something to beat.

Microsoft also says to pick measures tied to your business goals and to use more than one. One of them is groundedness. That means: is the answer backed by the material you supplied, or made up?

Step 3: Check each claim

Break each answer into small claims. Check each one against the source. Your score is the supported claims divided by all claims. Curiosity's rubric describes this method. It also says AI judges are fine for tracking trends, but human labels are better for go or no-go calls.

NIST describes a similar idea. Its evaluation probes compare an agent's claims to reference documents curated by people. They keep a record of the source, the verdict and the reason. NIST looks at three things:

  • Faithfulness: does the source back the claim?
  • Completeness: does the answer carry the source's full message?
  • Sufficiency: is the evidence enough, with no overreach?

If you want a quick way to check whether answers trace back to real sources, an LLM citation checker can show you where claims do not line up.

Step 4: Test the whole system

Do not only test the model. Test the lookup, the tools and the final answer too. A bad lookup gives a bad answer even when the model is fine. Re-run the tests after every change.

Our AI agent testing guide lists 10 scenarios and simple checks you can run. And human-in-the-loop workflows shows where a person should step in.

Step 5: Keep watching after launch

Real users ask odd things. So sample live answers and score them. Dropbox describes a pipeline with regression tests on every change, curated tests in staging, and constant sampling of live traffic. Its engineers also review a slice of each release by hand, 5 to 10% of the regression suite.

Dropbox also splits its release checks into three kinds. Yes or no gates, such as 'is there a citation?'. Number budgets, such as source accuracy and speed. And softer scores, such as tone. A failure on the required checks stops the release.

You do not need a big team to copy this. A weekly sample of 20 live answers, read by one person, is a start.

The trade-offs nobody mentions

Every fix has a price. Be honest about it.

  • Lookups add steps. Every lookup adds some delay and cost. Measure it on your own build.
  • Safer can mean thinner. One 2026 study found that in some tested methods, lower hallucination scores came with less detail and less coverage. So track usefulness alongside accuracy.
  • Rules can make the AI too shy. If it says 'I don't know' to everything, it is useless.
  • Sources go stale. Someone must own the library and update it.

So watch two numbers together. How often is it right? And how often is it helpful? A system that answers nothing never lies, but it never helps either.

Our view on hallucinations

Most hallucination advice is about the model. Swap the model, tweak the prompt, hope for the best. We think that misses the point.

In our work with founder-led education and consulting businesses, the real cause is usually missing context. The AI does not know how you think. So it fills the gap with a plausible guess.

That is why we start with your IP. We pull out your frameworks, your rules of thumb and the calls you make without thinking. Then we build them into an : a connected set of AI employees (small AI workers, each with one job), built with Claude Code. Each one answers from your material and your standards, so your know-how scales without you in every loop.

We also build the checks in from day one. Tests, handover rules and sampling are part of the build, not an add-on. Our approach is designed so your team can deliver to your standard without you in every decision.

We also say this plainly: AI will never be perfect. The goal is a system you can trust, because you can see how it works and test it.

James Killick

How we help

We work with founders and experts earning $1M+ a year. Their know-how is the product. They want the business to run without them in every decision.

This is what we build:

  • The AI Orchestrator OS at $30,000, or 3 x $11,000. We build your system with you, using your own IP as the source of truth.
  • 1 on 1 coaching from $1,950 a month once your system is live.
  • A self-serve platform at $90 a month, to build and manage AI systems from your own expertise.

We prototype with you, hands on, so you see how each part works.

A simple plan you can start this week

You do not need to do it all at once. Here is an order that works.

  1. Pick one task. Choose something common, like answering client questions.
  2. Gather the sources. Collect the five to ten documents that hold the right answers.
  3. Write the rules. Use sources only. Quote the source. Say 'I don't know' when unsure.
  4. Build a test set. 30 questions with checked answers.
  5. Score a baseline. Write down how many claims are supported.
  6. Change one thing. Re-test. Keep what helps.
  7. Add a human check. For high-risk answers, a person approves first.
  8. Sample every week. Read 20 live answers. Fix what you find.

This takes effort. But it is a lot cheaper than one wrong answer sent to a paying client.

Sources

Your next step: if your business earns $1M+ and runs on your know-how, take the assessment and see where your know-how is ready to be built into AI.

FAQ

Frequently asked questions.

JK

James Killick

Founder

The AI Orchestrator. 10+ years building digital products and 200+ apps shipped, now helping $1M+ educators and consultants turn their IP into AI-powered delivery systems.

James Killick founded and runs The AI Orchestrators.

Keep reading

Ready to find out where your biggest AI opportunity is?

Take the assessment. It takes about 5 minutes. You'll get a clear picture of how ready your business is.