How to make your website readable by AI search

When someone asks an AI assistant to recommend a studio, a plumber or a plant hire firm, it answers from what it can read. If it can't read your site cleanly, it describes you badly — or not at all.

Let the right crawlers in

AI services each use their own crawlers — GPTBot and OAI-SearchBot for OpenAI, ClaudeBot for Anthropic, PerplexityBot, Google-Extended and others. Your robots.txt decides which of them may read the site. Many sites block them by accident with a blanket rule; check yours says what you mean.

Give them a clean copy

A web page is mostly layout, scripts and navigation; the words are a small part of it. An llms.txt file at the root of your site is an emerging convention: a short Markdown index of your important pages, with a summary of each. Going further, each page can have a plain Markdown version alongside the normal one.

Our own site does both: franksdigital.com/llms.txt lists every page, and each page has a Markdown twin — /services/web-design.md, for example. They're built from the same content as the pages, so they never drift out of date.

Say who you are in structured data

Structured data (schema.org JSON-LD) states facts about your business in a form machines don't have to guess at: your name, what you offer, where you work, how to contact you, your questions and answers. Search engines have used it for years; AI systems read it too.

Write plainly

The rest is good writing. Say what you do, for whom and where, in plain sentences near the top of the page. Use real examples and real numbers. A page that a person can summarise in one line is a page an AI can summarise accurately.

Got a project like this?

Drop your email and a line about it. We'll come back with ideas, timings and a quote — usually within one working day, wherever you are.

Or email [email protected]

Already working with us?

Something gone wrong on a live site? Email us and we'll get straight on it. [email protected]

By submitting, you agree that we'll store your email address (and any message) along with your IP address, which we use to estimate your general location, and may contact or market to you about our services. See our Privacy Policy.