llms.txt explained

Short summary

llms.txt is a plain text file at a site's root that guides AI agents to important content, similar to how robots.txt works for crawlers. It reduces token costs by letting agents fetch clean markdown instead of parsing HTML noise. The article explains how llms.txt complements (not replaces) sitemaps and robots.txt, and includes a pitch for the author's validator tool at turva.dev.

• •llms.txt provides a curated map of site content for AI agents, reducing token spend and citation errors
• •It complements sitemaps and robots.txt — not a replacement or ranking trick
• •Sites wanting to be legible to AI assistants should publish llms.txt and serve markdown content

Generated with AI, which can make mistakes.