llms.txt — what it is, whether to make one and how to do it properly

llms.txt is a text file placed at the root of a site that gives language models a condensed, structured description of its content. The idea is the same as robots.txt and the sitemap: an agreed address where a machine finds what matters without wading through the whole site. The difference is the audience, because llms.txt is written for an AI rather than a search engine. Here is what belongs in the file, how to build one and what to realistically expect from it.

What llms.txt is

llms.txt is a markdown file at yourdomain.com/llms.txt. It typically contains a short description of what the site is, the most important pages as links with descriptions, and contact details. The proposal appeared in 2024 as a solution to a practical problem: web pages are full of navigation, scripts and repetition that make it laborious for a language model to dig out what matters. A clean text summary is easier reading for a model.

The structure is free-form but settling down: a top-level heading, a quoted summary, and then lists by section in the form [page name](address): description. No code, no tricks, just clear text.

What good it does — an honest assessment

Put plainly: llms.txt is promising but unestablished. No major AI operator has publicly committed to reading it, unlike robots.txt, which every reputable crawler respects. Some crawlers already fetch the file, which you can see in server logs, but there is no guarantee about how much weight it carries.

So why make one anyway? Three reasons:

• The cost is close to zero. An hour of work, and no maintenance if you generate it automatically. The asymmetry is appealing: a small input, potentially a significant benefit.
• It forces you to crystallise. Writing your company and services into five clear lines shows you which pages actually describe what you offer and which do not. That clarity benefits everything else you write.
• Early adopters win if the standard catches on. If llms.txt becomes a convention like robots.txt, those already prepared benefit first. If it does not, you lost an hour.

How to build a working llms.txt

Start with the summary. Two to four lines: who you are, what you do, for whom, and what sets you apart. Write facts, not marketing lines. The file is source material for a model, and marketing language is noise to it.

List the pages that matter. Service pages, pricing, tools, your best guides. One line each: name, address, a one-sentence description of what is on the page. Leave out what you do not want in answers, because llms.txt is curation, not a sitemap.

Add contact and basic details. Email, phone, location. These are exactly what people ask an AI, and exactly where models make mistakes when the information is scattered.

Keep it current. An outdated llms.txt is worse than none, because it feeds a model wrong information directly. The best solution is to generate the file automatically from the site’s content, so new articles and changed details update themselves.

What llms.txt does not do

It is not a shortcut to visibility. The file helps a model understand your site, but it does not make your content worth quoting and it does not produce mentions that would not otherwise happen. If the site has no content that answers questions, llms.txt is a cover page for an empty book. The order of priority is clear: content first, then technical accessibility, with llms.txt as part of the latter.

How it differs from robots.txt and the sitemap

Three files in the site root, three different jobs. The confusion is common, so the distinction is worth keeping straight.

• robots.txt says what may be fetched. It is a permission list: where a bot may go and where it may not. It says nothing about the content, and every well-behaved bot respects it.
• The sitemap says what exists. A machine-readable list of every URL with update times. It is complete but unsorted: a sitemap does not separate the important from the irrelevant.
• llms.txt says what matters and why. It is curated and descriptive. Where a sitemap lists three hundred addresses, llms.txt picks fifteen and describes in a sentence what each one holds.

Curation is the entire point. Copy the sitemap into the file and you have a worse sitemap that helps no model at all.

What a finished file looks like

An illustrative extract. Notice how each line states both the address and what is found there, and how the descriptions are factual rather than promotional.

# Example Ltd
> Helsinki-based accounting firm for small businesses. Bookkeeping, payroll and tax advice. Founded 2011, 12 employees.

## Services
[Bookkeeping](https://example.fi/bookkeeping): Monthly bookkeeping with pricing examples, suited to companies of 1 to 50 people.

## Contact
Email: sales@example.fi | Phone: +358 9 123 4567 | Address: Mannerheimintie 1, Helsinki

One to two pages is enough. If the file grows to tens of kilobytes, the curation has failed and the essentials are buried again.

Keeping the file current

A hand-maintained llms.txt goes stale in three months. The pattern is consistent: the file is written once in a burst of enthusiasm and looked at again a year later, by which time something has changed.

In WordPress this is solved with a theme function that listens on the file’s address and assembles the content straight from the database: company details from settings, service pages from the page hierarchy and the newest articles from posts. When a new article appears, it is in the file the same day without anyone remembering to do anything.

Two practical details. Serve the file as text/plain, or browsers and some bots will treat it wrongly. And if the site is multilingual, keep the languages separate: a mixed-language file confuses the model’s picture of who the service is for.

Who actually fetches the file

Rather than argue about it, read your server logs. They tell you directly who requests /llms.txt and how often. On our own site the file is fetched regularly by at least the ChatGPT and Perplexity crawlers, plus a handful of smaller research indexers. The volumes are small next to ordinary page requests, but they are not zero.

This does not prove the contents influence answers. A fetch proves only that the address is checked. But that is more than can be said for most proposed standards, and it explains why keeping the file current is not wasted effort.

Four common mistakes

The file is a copy of the sitemap. Three hundred URLs without descriptions help no model. Curation is the only thing that separates llms.txt from a sitemap.

The text is marketing copy. Phrases like market leader are worthless to a model because they cannot be verified. Write measurable facts: founding year, headcount, service area, services by name.

Contact details are missing. One of the most common questions put to an AI is how to reach a company. If it is not in the file, the model finds it elsewhere or guesses.

The file is forgotten. Services change, prices change, the contact person changes. Automatic generation handles this; manual upkeep does not.

When llms.txt is worth doing

Order matters. If your site does not yet have content answering your customers’ questions, write that first. A language model cannot quote what does not exist, and a tidy index to an empty warehouse helps nobody.

If the content is there but the technical side is not, fix indexability first. A bot that cannot reach your pages gains nothing from a summary either. Only once both are in place is llms.txt a natural finishing touch.

In practice this puts llms.txt near the end of the AI visibility checklist rather than the start. It is cheap to make, but it replaces nothing that comes before it.

Frequently asked questions

What is llms.txt?

A markdown file placed at the root of a site that describes the site’s content concisely for language models, in much the same way robots.txt instructs search engine crawlers.

Do AI systems read llms.txt?

Some crawlers already fetch it, but no major operator has publicly committed to using it. It is a promising proposal rather than an established standard, which is why the effort should be small and automated.

Does llms.txt replace the sitemap?

No. A sitemap lists every page for search engines; llms.txt curates the most important ones and describes them for language models. Both are worth keeping.

What goes into an llms.txt file?

A short factual description of the company, the most important pages as links with descriptions, and contact details. In markdown, without marketing language.

How long should an llms.txt file be?

One to two pages is enough for most companies. If the file grows to tens of kilobytes, curation has failed: the essentials get buried in the mass exactly as they do on the site itself.

Can llms.txt harm search visibility?

No. Google and other search engines do not use the file for rankings, and its presence does not affect indexing. The only risk is that an outdated file feeds a language model incorrect information about your company.

We built ours so that it updates automatically, and you can see the result at purodigital.fi/llms.txt. Building AI visibility in full is described on the GEO service page, and you can see where you stand with the AI visibility test.