In this guide · 8 sections
Short answer
For most small business sites, llms.txt is optional and low priority. Google says no AI text files are needed for AI Overviews or AI Mode, and OpenAI asks you to allow its search crawler in robots.txt instead. It takes under an hour, so add one only after crawling, indexing and clear service pages are sorted.
What is llms.txt?
The llms.txt proposal was published by Jeremy Howard on 3 September 2024 and updated to version 2 on 10 August 2026. It suggests a Markdown file at the root of a website (or under a folder like /docs/llms.txt) that gives large language models and AI agents a short, clean overview of the site, with links to the pages that hold the detail.
The idea is practical: web pages are full of menus, scripts and pop-ups, and an AI agent with a limited context window wastes effort stripping them out. A curated text file points the agent straight to the useful content. Version 2 also suggests offering a clean Markdown copy of important pages (for example page.md next to page.html) and linking to it with standard link tags.
What goes in an llms.txt file
The specification is short. Only the first line is required; everything else is optional but recommended, and must appear in this order:
- An H1 title with the name of the site or business (the only required part).
- A blockquote with a one or two sentence summary.
- Plain paragraphs or lists with key context, such as what you sell and where you operate.
- H2 sections with link lists, each item a Markdown link plus an optional note after a colon.
- An "Optional" H2 section for secondary links an agent can skip when space is tight.
For a small service business, a useful file might look like this in plain words: the business name as the heading; a summary line such as "Aircon servicing and repair for homes and offices in Singapore"; a few lines on service area, opening hours and how to book; then a "Services" section linking each service page, a "Prices" section linking the price list, and an "Optional" section linking the blog. Keep every link pointing at a live page.
Do Google, OpenAI and others use llms.txt?
This is the question that decides whether it is worth your time. Here is what the vendors' own documentation says as of October 2026.
| Who | What their documentation says | What actually controls visibility |
|---|---|---|
| Google (AI Overviews, AI Mode) | You don't need new machine-readable files, AI text files or markup to appear in AI features | Normal crawling, indexing and snippet controls |
| OpenAI (ChatGPT search) | Recommends allowing OAI-SearchBot in robots.txt; no mention of llms.txt for search | robots.txt rules for OAI-SearchBot |
| Anthropic (Claude) | Documents three robots controlled through robots.txt | robots.txt rules for ClaudeBot, Claude-SearchBot, Claude-User |
| The llms.txt site | Says AI labs publish llms.txt files for their own developer docs | Agents may read it when helping a user |
Google's AI features guide is explicit: there is no special schema and no AI text file needed, and normal SEO best practice applies. OpenAI's crawler documentation describes robots.txt tokens, not llms.txt. The llms.txt site itself says the file is used most heavily for software documentation, where coding agents follow it to find API references, and that OpenAI, Anthropic and Google publish such files for their developer docs. Publishing a file for your own docs is not the same as using other people's files to rank them.
llms.txt vs robots.txt vs sitemap.xml
The three files are often confused. They do different jobs, and llms.txt cannot replace either of the other two.
| File | Purpose | Read by | Can it block crawlers? |
|---|---|---|---|
| robots.txt | Tells crawlers what they may and may not fetch | Search and AI crawlers that respect it | It asks; well-behaved bots follow it |
| sitemap.xml | Lists every URL you want indexed | Search engines | No |
| llms.txt | Curated summary and reading list for AI agents | Agents that choose to look for it | No |
If your real worry is AI companies using your content for training, llms.txt does nothing for you. That is a robots.txt and firewall question, covered in our guide to blocking AI crawlers with robots.txt.
Effort vs benefit for a small business
Adding llms.txt is cheap, which is the main argument for it. The argument against is opportunity cost: most small business sites that are invisible in AI answers have a more basic problem.
Pros
- Takes under an hour to write and upload for a typical 10 to 30 page site
- Harmless to search rankings
- May help AI agents that a customer sends to your site find prices and service pages faster
- Some platforms and plugins can generate a first draft
Cons
- No search engine says it uses the file for ranking or citations
- Another file to keep in sync; stale prices in it can mislead an agent
- Does not stop any crawler or protect your content
- Often sold as an "AI SEO" add-on at a price far above the work involved
Do these first, in order: make sure Google can crawl and index your pages, allow the search crawlers you want in robots.txt, write each service page so it answers the customer's question plainly, and keep your Google Business Profile and structured data accurate. Our guide to showing up in ChatGPT and Google AI Overviews walks through each of these.
How to add llms.txt to your site
If you have done the basics and want one anyway, here is the short version.
Write the file in Markdown
Start with an H1 business name, a one-line summary in a blockquote, two or three lines of context, then H2 sections listing your most useful pages with a short note each.
Link only to live, accurate pages
Services, prices, service area, booking or contact page, policies. Leave out tag pages, logins and anything you would not want quoted.
Upload it to the site root
On WordPress, upload via your host's file manager or an SEO plugin that supports it. On Shopify, Wix or Squarespace, check the platform's help first; some don't allow arbitrary root files.
Check it loads
Open yoursite.com/llms.txt in a browser. It should show plain text, return status 200 and not redirect to the home page.
Put it on your update list
When you change prices or services, update the file in the same session. A wrong price in an AI-friendly file is worse than no file.
Do it yourself or get help?
Writing an llms.txt file is a do-it-yourself job for anyone who can edit their website. Paying a specialist several hundred dollars for one is poor value. The work that moves the needle is checking that your site is crawlable, indexed, fast and clear.
Before you worry about llms.txt
Our Speed & SEO Fix (S$390) checks indexing, robots.txt, page speed, titles and structured data, and can add an llms.txt file at the same time if you want one.
Frequently asked questions
No evidence says it does. Google's guide to AI features states that you don't need new machine-readable files, AI text files or markup to appear in AI Overviews or AI Mode, and that normal SEO practices apply. Google has not said it reads llms.txt for ranking. Spend the time on indexing, page content and your Business Profile first.
No. llms.txt is a reading list for AI agents, not an access rule. To ask AI crawlers not to collect your content, add rules for tokens like GPTBot, ClaudeBot and Google-Extended in robots.txt, and use your CDN or firewall bot settings if you want enforcement, because robots.txt compliance is voluntary.
No. It is a community proposal first published on llmstxt.org in September 2024 by Jeremy Howard and revised as version 2 in August 2026. It is not a W3C or IETF standard, and no search engine has adopted it as a ranking input. Some documentation platforms generate it automatically.
Version 2 of the proposal says the file can sit at the site root, yoursite.com/llms.txt, or under any path such as /docs/llms.txt, where it covers the pages under that path. If more than one file applies, agents should use the most specific one. For a small business site, one file at the root is enough.
It is optional. First check whether your platform lets you place a file at the site root; not all hosted builders do. Then weigh the small effort against the lack of any documented ranking benefit. Making product and service pages clear, crawlable and accurate matters far more for AI answers and normal search.