An llms.txt file is a short Markdown file at the root of a website that tells AI models and agents what the site is and which pages matter most. Jeremy Howard proposed it in September 2024 on llmstxt.org. It is a convention, not a standard that any AI company has committed to, and the evidence that assistants read it is thin. It is still cheap to write. This post covers what goes in the file, who uses it, and an example from our own site.
What is llms.txt?
The idea is simple. A web page is built for people: menus, scripts, cookie banners and layout. A language model reading the page has to dig through all of that to find the text. An llms.txt file gives it a clean summary and a list of links instead.
The proposal on llmstxt.org sets out the format. In order, the file has:
- An H1 with the name of the site or project. This is the only required part.
- A blockquote with a short summary.
- Optional paragraphs with more detail, without headings.
- Optional H2 sections that list links in the form
[name](url): notes.
A section called "Optional" holds links an agent can skip when it needs less context. The proposal also suggests offering a clean Markdown version of each page at the same address with .md added. The file sits at the site root, for example example.com/llms.txt.
Do AI assistants actually use llms.txt?
Here we keep to what we could check.
- Google Search says you don't need it. Google's guidance on AI features in Search states: "You don't need to create new machine readable files, AI text files, or markup to appear in these features." So llms.txt is not a way into AI Overviews or AI Mode.
- Chrome's Lighthouse checks for it. Lighthouse has an llms.txt audit in its agentic browsing category. It flags a server error when fetching the file. A missing file is marked "Not Applicable", and the page says "providing the file is optional at the moment".
- We found no official statement from OpenAI, Anthropic or Google that their assistants read llms.txt on other people's sites when answering questions. Anthropic and OpenAI do publish llms.txt files for their own developer docs, which says more about docs than about search.
So treat llms.txt as a low-cost extra for agents and tools that choose to fetch it. It is not a ranking factor, and it does not replace clear pages, good titles and structured content.
Who benefits most from an llms.txt file?
The file is most useful where an agent or coding assistant needs to read a lot of text quickly:
- Software documentation and APIs, where developers point their tools at the docs.
- Product and service sites with many pages, where a summary helps a reader pick the right one.
- Sites with Markdown versions of pages, so the links in the file lead to clean text.
A small brochure site with five pages gains less. It costs ten minutes, though, so there is little reason not to.
How do you write an llms.txt file?
Keep it short and factual. A model will take your summary at face value, so write what you would want quoted.
- Name and one-line summary. Who you are and what you do, in plain words.
- Key facts. Location, legal name, contact page, languages. Facts a model often gets wrong.
- Sections of links. Services, products, case studies, docs. One line per link, with a short description.
- An "Optional" section for less important pages.
- Check it. Open the file in a browser, follow each link, and make sure it returns a 200, not a server error.
There are llms.txt generators that crawl a site and build the file for you. They save typing, but read the result. Generated descriptions are often vague, and vague is what you are trying to fix.
An llms.txt example from gottdata.com
Our own file is at gottdata.com/llms.txt. It starts with the required H1, # Gott Data, and goes on like this:
> Gott Data is a data and software studio in Reykjavík. We build data pipelines and dashboards, machine learning and AI, custom software and growth systems.
## Services
- [Analytics: Data Analytics, ETL Pipelines & BI Dashboards](https://gottdata.com/analytics): ETL and data pipelines that join every system into one source of truth...
- [Software: Custom Software, Web & SaaS Development](https://gottdata.com/software): Custom software from first sketch to production...
## Data and AI case studies
- [TourDesk Data Foundations](https://gottdata.com/work/tourdesk-data-foundations): Hotels, vendors, bookings and car rentals unified into one governed source of truth...
Between the summary and the services it has a paragraph with our legal name, address and contact page. It also links to an llms-full.txt file with the full text of every service page, case study and post. That second file is our own addition, not part of the llmstxt.org proposal.
We generate both files from the same content as the site, so they never drift out of date. That is the main maintenance advice: build the file from your content, don't hand-edit a copy.
llms.txt is one small part of how a business shows up in AI answers. Growth HQ tracks that visibility next to SEO, ads and content, and you can try it in the live demo.
