Does llms.txt do anything? 97 percent of the files Ahrefs found got zero requests

Ahrefs checked 137,210 domains in June 2026. Twenty-eight percent had published an llms.txt. Ninety-seven percent of those files received zero requests.

Of the three percent that saw any traffic at all, the Ahrefs study found 96 percent of it was bots, and only 1.1 percent came from AI retrieval bots.

Google says the same thing from the other direction. The AI optimization guide, updated on 10 July 2026, states that Google Search does not use llms.txt and that it neither harms nor helps. Gary Illyes has confirmed there are no plans to support it.

Now the awkward part.

The llms.txt guide on this site tells you to write one, and I wrote it.

That guide hedged correctly on support and still landed on "add one anyway, it costs twenty minutes." The study above is the evidence that was missing when I published it, and this post is the correction rather than a defence.

Nobody needs another round of is it read or is it not.

So here is what changes on Monday.

Step 1: take the one case Mueller conceded seriously

John Mueller granted a narrow exception, and it is the only part of the llms.txt argument still standing.

An agent that is already on your site, dispatched there by a person with a task, may use the file to find the right page faster. Not to discover you. To navigate you.

That distinction is everything.

Discovery is a crawler deciding your page is worth fetching in the first place, which is a ranking and retrieval problem that a text file at your root has no influence over. Navigation is a machine that already has your domain and needs to find the booking page in under three fetches.

So the question to ask is whether agents have any reason to be on your site with a task at all. A restaurant with an online booking flow, a clinic with an appointment form, a shop with a cart: those are sites an agent may be sent to. A brochure site with a contact form is not.

If that is you, the file is a signpost worth having.

If it is not, the file is a twenty minute hobby.

Step 2: if you ship one anyway, ship the useful version

Most published llms.txt files are a table of contents nobody asked for.

If the only realistic reader is an agent trying to finish a task, write it for that reader and nothing else.

Lead with classification. One sentence that lets a machine place you: what type of business, where, at what level, for whom.

Then list the action pages before the reading pages. Booking, ordering, appointments, contact, pricing, in that order, each with a one-line description of what the page does rather than what it is about. An agent needs the verb.

Then state the constraints in prose. Hours, price band, capacity, service area, cancellation terms, what you refuse to do. These are the facts a task gets stuck on, and they are the facts most business sites bury in an image or a PDF.

And keep the link list short.

Six pages that answer something beat forty that exist.

Skip llms-full.txt unless your site is genuinely large. The companion file concatenates the content of every linked page into one document, which makes sense for a documentation set of four hundred pages and no sense for a nine-page services site whose entire text would fit in a single fetch of the homepage.

The same goes for generating it from a plugin and never looking at it again. A file written by a script that walks your sitemap reproduces your navigation, and your navigation is the thing the format was invented to strip out.

One rule that overrides all of the above: keep it current or delete it. A stale llms.txt is the single version of this file that can genuinely hurt you, because it presents wrong facts in the cleanest, most confident format on your whole domain.

Step 3: check whether anything has ever fetched yours

This is the step nobody in the llms.txt debate performs, and it settles the question for your site in about four minutes.

Pull the request count first.

grep -hic "llms\.txt" /var/log/nginx/access.log*

Then find out who they were.

grep -hi "llms\.txt" /var/log/nginx/access.log* \
  | grep -oiE '(GPTBot|OAI-SearchBot|ChatGPT-User|ClaudeBot|Claude-SearchBot|PerplexityBot|Bingbot|Googlebot|Google-Extended)' \
  | sort | uniq -c | sort -rn

Read the result honestly.

Your own browser hits count as zero. SEO tools and uptime monitors count as zero. What you are looking for is an answering crawler or an agent user-agent, and in most cases you will not find one.

If your site sits behind Cloudflare, the equivalent evidence is AI Crawl Control, which logs which AI services requested your content. If you are on a host that does not expose raw access logs, a redirect from /llms.txt to /llms.txt?v=1 with the query string tracked in analytics will get you a rough count without touching the file's contents.

The point of running this is not to win an argument. It is to replace a belief with a number for your own domain, which is the same discipline we apply to every other GEO claim at NovaTechRay.

Step 4: spend the attention on the thing that is fetched

Here is the uncomfortable comparison.

Your llms.txt has a known audience of roughly nobody, while your homepage is fetched by the answering crawlers whether you like it or not.

So the work that pays sits on the pages that are already being read.

State what you are, where you are and who you serve in the paragraph directly after the H1, with no warm-up sentence. Make sure the page renders that text server-side rather than assembling it in the browser. Confirm that the answering crawlers are not blocked at the edge, which is a separate question from robots.txt and covered in the Cloudflare decision tree.

None of that is new advice. It is unglamorous, it does not fit on a proposal as a novel deliverable, and it is the reason llms.txt got so much attention in the first place: it was the only item on the list that looked new.

I would rather hand a client a boring correction than an interesting file.

The short version

• Assume llms.txt does nothing for discovery, because Ahrefs found 97 percent of published files got zero requests.
• Keep Google's line in front of you: the AI optimization guide says it neither harms nor helps.
• Ship the file only if agents have a real task to complete on your site, which is the case Mueller conceded.
• Write it for a machine finishing a task: classification first, action pages next, constraints in prose.
• Grep your access log for the path before you argue about it, and count only answering crawlers and agents.
• Delete it rather than let it go stale, because wrong facts in a clean format travel.
• Move the attention to the paragraph under your H1, which is fetched every time.

Sorting evidence from invoice lines is most of what NovaTechRay does at novatechray.com.