llms.txt: what it does and does not do
By Ismael · Reviewed
llms.txt is a proposed plain-text file at a site root that gives AI agents a curated map of a site’s useful pages and a short description of the site. SiteBienfait publishes one because agents increasingly read sites directly. The specification, published by Jeremy Howard on 2024-09-03 and modified on 2026-08-10, describes a proposal to help agents use a website.
What is llms.txt?
The specification proposes a Markdown file at /llms.txt containing a title, a one-line summary and a set of links grouped by section. Each link names a page that matters and says in a sentence why. A larger companion file at /llms-full.txt can carry the full text of those pages.
The framing in the specification is important: this is a proposal for agents, not a directive for search engines. It was written for tools that fetch a site and need to know where to start, the same way a library’s README tells a developer which docs to read first.
Does Google Search use llms.txt?
No. Google Search Central, updated 2026-07-10, is unusually direct on this point: site owners do not need to create new machine-readable files, AI text files, markup or Markdown to appear in Google Search, including its generative AI capabilities.
That does not make the file harmful. It makes it unnecessary for Google’s own crawler and features. Treating it as a prerequisite for AI visibility is where sites waste effort.
Do other answer engines read it?
Some agent tooling does; the specification exists because those readers wanted a predictable entry point. But no major answer engine documents llms.txt as a ranking or citation input, and none promises that a file changes what appears in a response.
The honest position is that llms.txt is a convenience: if an agent reads it, it saves the agent time. Whether a given agent reads it is outside the site’s control. Two other things are firmly inside the site’s control, and they matter more: crawl access for the documented bots, and pages whose facts are readable without JavaScript.
Should a small business site publish one?
Publish it if the cost is one hour and the maintenance is a line in the release checklist. Do not publish it if it duplicates a sitemap that nobody updates, or if it becomes another place where a stale price or an abandoned service lingers.
A practical rule: write it only when the site has reference content worth pointing at, such as guides, pricing, security notes or documentation. A five-page brochure site gains little, because a crawler can read five pages quickly.
What should go in the file?
A title, one sentence describing the site in the same words used in the Organization schema and the about page, and short sections. Product pages, pricing, security and guides are the usual sections. Link with absolute URLs, one line each, with a phrase describing what the reader will find.
Entity consistency matters more than the file. The canonical description of SiteBienfait is repeated word for word in the schema, the about page and llms.txt, as the marketing plan requires, because a model that finds three different descriptions of one business has to guess which is true.
What should be left out?
Do not put anything in the file that contradicts the HTML. Do not list pages that redirect, are noindexed or require a login. Do not use the file to describe offers that do not exist; a stale llms.txt is worse than none, because it directs an agent to a dead end.
That is the same discipline a redirect map needs. The redirect map checker verifies that every URL a file points at still resolves, before the file ships. The wider context for this work is in what GEO is and what it cannot control.
SiteBienfait’s GEO route treats llms.txt as one optional artefact in a change set, alongside answer blocks, entity facts and crawler access, and states plainly that citation is not promised.
Sources
- Jeremy Howard, The /llms.txt file specification, published 2024-09-03, last modified 2026-08-10.
- Google Search Central, Optimizing for generative AI features on Google Search, last updated 2026-07-10.
- Google Search Central, AI features and your website, last updated 2025-12-10.
Questions
01 What is llms.txt?
A proposed plain-text file at a site root that lists the pages an AI agent should read, with a short description of the site. The specification was published on 2024-09-03 and last modified on 2026-08-10 by its author, Jeremy Howard.
02 Does Google Search use llms.txt?
No. Google Search Central, updated 2026-07-10, states that site owners do not need to create new machine-readable files, AI text files, markup or Markdown to appear in Google Search, including its generative AI capabilities.
03 Will publishing llms.txt get a site cited by ChatGPT?
No answer engine documents that llms.txt affects retrieval or citation. It is useful as a convenience for agents that choose to read it, not as a switch that turns on citations.
04 Should a small business site have an llms.txt file?
It is optional and cheap to maintain. It makes sense when the site publishes useful reference material that agents may fetch, and it is harmless when the same facts already exist in crawlable HTML.