Skip to content

Topic hub · 1 guide

What the bot reads, and what it must not

A chatbot that reads the wrong pages answers the wrong questions. This hub covers what gets indexed by default (published pages, posts and products), what must never be (noindex, password-protected, drafts), how a passage updates when you edit the page behind it, and why each language on a multilingual site is indexed separately rather than machine-translated.

What this hub covers

  • What gets crawled by default
  • Excluding pages properly
  • Re-indexing on edit
  • Multilingual indexing

Start here

How do I train an AI chatbot on my own data? You mostly don't: rounded dark and brand-coloured blocks sliding from a loose stack on the left into an aligned column on the right
Guides

How do I train an AI chatbot on my own data? You mostly don't

Training teaches a model how to sound. Indexing teaches it what is true about your business. Almost everyone asking the first question actually wants the second.

10 min read

Related topic hubs

Named across this hub: WordPress, WooCommerce, Yoast, noindex, hreflang.

Put a real answer on every page.

Ask this site's own widget a question and watch it cite the page, or refuse. It is the same plugin, running on the same content you are reading.

Join the launch list Try the live demo