Why we don't score llms.txt
AIOScan reports whether you have an llms.txt file and awards zero points either way. If a tool or agency is scoring you on it, that tells you something about the tool, because the measured evidence is unambiguous: AI systems do not read the file. This page lays out that evidence so you can check it yourself.
What does the measured evidence show?
Two independent studies, one enormous and one surgical, reach the same conclusion. Ahrefs analyzed server logs and bot traffic across 137,000 domains and found that of the roughly 38,000 domains with a valid llms.txt file, 97% received zero requests for it in an entire month. No AI bots, no humans, nothing.
Otterly ran the surgical version: a correctly implemented llms.txt on a monitored site for 90 days, during which the site logged over 62,100 AI bot visits. Exactly 84 of those requests touched the llms.txt file, about 0.1%, while the average content page drew roughly 265 bot visits. The file underperformed an ordinary page by a factor of three.
Credit where due: Otterly sells AI search monitoring, and publishing a null result about a tactic their industry hypes is exactly the kind of honesty we want to reward with a citation.
Why don’t AI systems need the file?
Because they answer through live search. When you ask ChatGPT a question, it rewrites your prompt into search queries, retrieves ranked results, and synthesizes from pages it fetches. There is no step where a model detours to your llms.txt to ask what you’d like it to read.
Pedro Dias, formerly of Google’s search quality team, explains the design intent: llms.txt was aimed at agentic browsing, an AI-driven browser landing on a JavaScript app it can’t parse. At the retrieval layer it plays no part, and it contributes nothing to training because there is no content in it to train on. Even Google’s John Mueller has described it as not being for search.
Where did llms.txt come from, and why did it spread?
The proposal came from Answer.AI in 2024: a clean markdown map of your site at a predictable address, so a model could skip your HTML noise. It’s a tidy idea, which is exactly why it outran its evidence. It is also easy to check, easy to sell, and sounds plausible in a sales call.
A deliverable that costs the vendor nothing and does nothing for you is the perfect unit of AI SEO grift, and “sells llms.txt as a core deliverable” has become a useful tell when evaluating agencies.
Should you delete yours?
No. It’s harmless, and we keep one ourselves. It costs nothing to leave in place, and if adoption ever genuinely changes, we’ll change this scoring in the open with the new evidence linked, the same way the current evidence is linked above. Just don’t spend money or attention on it, and be skeptical of anyone who invoices for it.
What should you do instead?
The things with measured effects. Make sure no firewall is silently blocking AI crawlers, since that failure mode is real and common. Put your answer in the first 100 words, where almost 40% of AI citations originate. And phrase headings as the questions people ask, because those are what AI’s generated sub-queries match.
Rescan after each change and watch points move on checks that actually carry them.
See where your site stands. The free scan takes about fifteen seconds and shows every fix.
Run a free AI visibility scan