# What Is a RAG AI Chatbot? How It Works on a Website

> By Daniel Hartley, Founder, DFYe. Published 2026-10-07. 11 min read. Guides.
> Source: https://dfye.com/blog/what-is-rag-ai-chatbot/
> Language: en

The retrieval step is the whole difference between a bot that knows your site and one that guesses.

**TL;DR.** A RAG chatbot retrieves the most relevant passages from your own pages, then has a language model answer from them, an approach published by Lewis and colleagues in 2020. That is what lets it use current prices, show its source and refuse questions your content does not cover. Quality depends mostly on chunking and clear pages, running costs are small, and AI Engine, with 90,000+ installs, is the established free WordPress option today.

A RAG AI chatbot answers a visitor by first retrieving the most relevant passages from your own content, then handing those passages to a language model with an instruction to answer from them. RAG stands for retrieval-augmented generation, the approach Lewis and colleagues described in their [2020 paper](https://arxiv.org/abs/2005.11401), and it has become the standard way website chatbots stay tied to a specific site. The retrieval step is what makes three things possible that a plain ChatGPT-style bot cannot do reliably: answering from your current prices and policies, showing which page an answer came from, and refusing when your content does not cover the question. I am building one for WordPress in public with DFYe, which is not on sale, so what follows is the mechanism, its costs and its failure modes, not a pitch.

## How does a RAG chatbot work, step by step?

The pipeline has two halves: one that runs when your content changes, and one that runs on every question.

**Indexing, when content changes:**

1. **Collect.** The system reads your pages, posts, product descriptions, help articles or PDFs.
2. **Chunk.** Each document is split into passages, typically a few hundred words each, ideally along headings so a passage means something on its own.
3. **Embed.** Each passage is turned into a vector, a list of numbers that represents its meaning, by an embedding model.
4. **Store.** The vectors go into a vector index, a database built to find the nearest vectors quickly.

**Answering, on every question:**

1. **Embed the question** with the same embedding model.
2. **Retrieve** the passages whose vectors sit closest to the question's vector, usually the top three to ten.
3. **Decide.** If no passage is close enough, the bot should say it does not know or hand over to a person.
4. **Generate.** The model receives the question plus the retrieved passages and an instruction to answer only from them, ideally citing which passage it used.

The [survey of RAG methods by Gao and colleagues](https://arxiv.org/abs/2312.10997) catalogues dozens of refinements to this loop, re-ranking, query rewriting, hybrid keyword and vector search, but the basic shape has not changed since 2020.

## RAG compared with the other ways to make a bot know your site

| Option | Best for | Why it works | Main limit |
| --- | --- | --- | --- |
| RAG over your content | Sites with real content that changes | Answers from current text, can cite and refuse | Quality depends on chunking and retrieval |
| Long context, paste everything | Very small sites | No index to build | Cost grows with every page; models miss facts in long inputs |
| Fine-tuning a model | Tone and format at scale | Changes how the model writes | Does not reliably store facts; retrain on every change |
| Plain model, no site data | Generic chat | Nothing to build | Invents answers about your business |
| Hand-written FAQ bot | Tiny, stable question sets | Fully predictable | Breaks on any question not scripted |

The long-context row deserves a sentence. Pasting a whole site into every prompt is tempting now that models accept large inputs, but the [Lost in the Middle study](https://arxiv.org/abs/2307.03172) found that models use information at the start and end of a long input more reliably than information in the middle. Retrieval sends a small, relevant slice instead, which is cheaper and easier for the model to use.

## Why chunking decides answer quality

Most bad RAG answers are retrieval failures, not model failures, and most retrieval failures start with chunking.

A passage that is cut mid-sentence, or that mixes the shipping policy with the returns policy, retrieves badly and reads badly. A passage that follows the page's own structure, one heading and the text under it, retrieves cleanly because it means one thing. That is why the way a site is written matters to its chatbot: pages with clear headings and self-contained sections produce good passages, and walls of text produce bad ones.

Three chunking rules hold up in practice. Split on headings first and on length second. Keep the page title and heading with every passage, so "14 days" arrives as "Returns policy: 14 days" rather than a bare number. And keep tables intact, because half a table is worse than none.

## What should a RAG chatbot do when retrieval finds nothing?

Refuse, clearly, and offer a route to a person. This is the behaviour that separates a trustworthy site bot from a liability.

When the nearest passages score below a threshold, the honest answer is "I could not find that on this site", followed by a contact option. A bot that answers anyway falls back on the model's general knowledge, which is exactly where invented shipping times and imaginary discount codes come from. We wrote a longer piece on [what should happen when a chatbot cannot answer](/blog/what-happens-when-ai-chatbot-cannot-answer-question/), because the refusal path is where most of the design work actually sits.

Setting the threshold is a trade-off you tune on real questions. Too strict and the bot refuses things it could answer; too loose and it answers things it should not. Log both refusals and answers, and read a sample every week.

## What does RAG need from your content?

| Content property | Why retrieval needs it | What to check |
| --- | --- | --- |
| Clear headings | Passages split on structure retrieve cleanly | Every page has H2s that describe their section |
| Facts in text, not images | Embeddings read text only | Prices, hours and policies are not only in banners |
| One topic per section | Mixed sections retrieve for the wrong questions | Shipping and returns are separate sections |
| Current information | The bot repeats what the page says | Old campaign pages are removed or marked |
| Consistent numbers | Two pages with two prices produce a coin flip | One source of truth per fact |
| Accessible pages | The indexer must be able to read them | No login walls or script-only content on pages you want used |

These properties are also, almost line for line, a list of what makes a page good for search engines and for AI answer engines. A site that is easy for its own chatbot to read is easy for everyone else's.

## How do you test a RAG chatbot before it goes live?

Write the test set before you look at any answers. Collect thirty to fifty real questions from your inbox, contact form and sales calls, and add ten questions your site deliberately does not answer, such as a competitor's opening hours or a product you do not sell.

Then run every question and mark three things: did it retrieve the right passage, did the answer match that passage, and did it refuse the ten it should refuse. Retrieval misses point to chunking or to content gaps. Answers that drift from a correct passage point to the instruction or the model. Questions answered that should have been refused point to the threshold.

Keep the set and rerun it after every content change of any size and after any change of model. A RAG bot has no single setting that makes it right; it has a test set that tells you whether today's configuration still is.

## How much does a RAG chatbot cost to run?

The raw usage is small for most sites. Embedding a few hundred pages is a one-off cost that, at the per-token prices model providers publish, typically comes to cents or a few dollars. Each answer then costs the question, the retrieved passages and the reply in model tokens, usually a fraction of a cent to a few cents depending on the model.

The expensive part is everything around the pipeline: hosting the index, keeping it in sync as content changes, logging, analytics, the widget, and the time spent reviewing answers. Hosted chatbot products bundle those into a subscription, which is why their prices sit far above raw usage. Our breakdown of [what an AI chatbot costs](/blog/how-much-ai-chatbot-cost/) goes through the line items.

## Which WordPress options use RAG today?

On WordPress, the most established option is AI Engine, which is free, mature and shows 90,000+ active installations on its [WordPress.org listing](https://wordpress.org/plugins/ai-engine/) as read on 14 October 2026. Its Pro version adds knowledge bases built from your content and PDFs with automatic chunking, and supports vector databases including Chroma, Qdrant and Pinecone, as well as OpenAI's vector store. If you want RAG on WordPress today, it is the first thing to try.

Hosted products such as Chatbase take a different route: you connect your site or documents to their platform and embed their widget, and the index lives with them. That is less work and less control; our [Chatbase alternatives comparison](/blog/chatbase-alternative/) covers when each shape fits.

DFYe, the plugin I am building, is a third shape: everything runs inside WordPress, close to what people mean by a [self-hosted chatbot](/blog/self-hosted-wordpress-ai-chatbot/), passages come from the site's own pages, and the citation is shown with the answer. It does not ship yet, so it is not an option for your site today, and I would rather say that plainly than pretend otherwise.

## When is a RAG chatbot the wrong answer?

When the site has almost no content. A five page brochure site gives retrieval nothing to work with, and a clear contact page or a short FAQ is cheaper and more honest than a bot that refuses most questions.

When the questions are about live account data. "Where is my order?" or "why was I charged twice?" need secure lookups in your shop or billing system, not passages from public pages. RAG covers the policy half of those conversations and none of the data half; our post on [training a chatbot on your own data](/blog/how-do-train-ai-chatbot-own-data/) covers where that line falls.

And when nobody will read the logs. A RAG bot without review drifts quietly: content changes, a page is renamed, a policy is updated on one page and not another. If there is no time for a weekly look at what it said, a well written help page will serve visitors better.

## Key takeaways

A RAG chatbot retrieves passages from your own content and has a model answer from them, which is what lets it stay current, cite its source and refuse what your site does not cover. Retrieval quality, and therefore answer quality, depends mostly on chunking and on how clearly your pages are written. Set a refusal threshold and tune it on real questions. Running costs are small; the work is in keeping the index current and reading the logs. AI Engine is the established free way to do this on WordPress today, and a plain FAQ is still the better answer for very small sites.

## Quick answers

### What is a RAG AI chatbot?

A RAG chatbot answers by first retrieving relevant passages from your own content, such as pages, help articles or PDFs, and then asking a language model to write the answer from those passages. RAG stands for retrieval-augmented generation, a method described by Lewis and colleagues in 2020. The retrieval step is what lets the bot cite your pages and refuse questions your content does not cover.

### Is RAG better than fine-tuning a model on my website?

For a website chatbot, almost always. Fine-tuning changes how a model writes, not what facts it can reliably quote, and it has to be redone whenever content changes. RAG reads the current pages at question time, can show which passage it used, and updates when you republish. Fine-tuning makes sense for tone or format at scale, rarely for keeping prices and policies correct.

### Does RAG stop a chatbot from hallucinating?

It reduces hallucination but does not stop it, and anyone promising zero is overselling. A model can still misread a retrieved passage, blend two passages, or answer from general knowledge when retrieval finds nothing useful. The fixes are a refusal rule when no passage scores well enough, citations shown to the visitor, and a review of logged answers, which is the part most setups skip.

### What does a RAG chatbot cost to run on a small website?

Mostly model usage, and it is usually small. Embedding a few hundred pages is a one-time cost of cents to a few dollars at current API prices, and each answer costs a fraction of a cent to a few cents depending on the model and how much retrieved text you send. Hosted chatbot subscriptions cost far more than the raw usage, because you are paying for the product around it.

### When is RAG not worth it?

When the site has very little content, such as a five page brochure site, or when nearly every question needs live account data rather than published text. In the first case a short FAQ or a contact form is cheaper and more honest. In the second, the bot needs secure integrations with your systems, and retrieval from pages solves only a small part of the problem.

## Sources

- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (Lewis et al., 2020)](https://arxiv.org/abs/2005.11401)
- [Retrieval-Augmented Generation for Large Language Models: A Survey (Gao et al.)](https://arxiv.org/abs/2312.10997)
- [Lost in the Middle: How Language Models Use Long Contexts (Liu et al.)](https://arxiv.org/abs/2307.03172)
- [AI Engine plugin listing (WordPress.org), read 14 October 2026](https://wordpress.org/plugins/ai-engine/)

---
*Published by [DFYe](https://dfye.com/). Free to read, index, quote and cite with attribution and a link.*