Why Your Docs Site Has an /llms.txt File Now
Knowledge publishes a standards-compliant llms.txt for every space — an early but real response to AI agents becoming a meaningful share of documentation traffic.
Mintlify reported in 2026 that nearly half of traffic to documentation sites now comes from AI agents rather than human browsers. Fern, GitBook, Vercel Docs, and Supabase Docs have all started shipping llms.txt as a default on their documentation platforms. Adoption across the web is still early — something like 10% of sites overall — but the direction is clear enough that Knowledge ships it too, and it's worth explaining plainly what it actually is.
What llms.txt actually is
It's a simple, standardized text file — following the convention established at llmstxt.org — that gives an AI agent or crawler a structured index of what's available on a documentation site, in a format designed to be efficient for a model to consume rather than optimized for a human browsing a page. Where a human might click through a navigation sidebar to find what they need, an AI agent can read one file and get an index of your entire published, public documentation at once.
How Knowledge generates it
Every space serves its own /llms.txt at docs/[spaceSlug]/llms.txt, listing every published and public document in that space. The access-control check behind it is exactly the same predicate used for the regular page render — nothing new to filter here. That's worth noting specifically: the file doesn't expose draft content, internal-only documents, or anything an AI agent couldn't already reach by crawling your public docs site page by page. It's the same content, restructured for a different kind of reader.
Why this is early, not universal, and that's fine
With adoption still around 10% of sites broadly, llms.txt isn't yet something every AI product assumes exists — some will read it, many still won't, and that will keep shifting for a while. Shipping it now isn't a claim that every AI system out there is already using it; it's a bet that the traffic-share numbers point the right direction, and that having the file in place costs nothing extra once your documentation access rules already exist to generate it from.
What this means if you maintain docs in Knowledge
Nothing changes about how you write documentation — llms.txt is generated from your existing published, public content automatically, not a separate thing to maintain. If your docs site is getting traffic from AI agents and you were wondering whether they can find their way around it efficiently, this is the mechanism that answers that, quietly, without any extra authoring work on your end.