Dev.to
7/11/2026

What an agent pays to read your site
Short summary
AI agents pay token costs to read your website, and heavy HTML pages make those costs high and reliability low. By implementing content negotiation — serving markdown to agents and HTML to browsers — publishers can cut agent token costs by roughly two-thirds. An llms.txt file provides a site map so agents can read structure in one request instead of crawling page by page.
- •Serving markdown via content negotiation cuts agent token costs by ~66% vs HTML
- •An llms.txt file gives agents a site map in a single request
- •Cheaper pages mean agents read full content and represent your facts accurately
Generated with AI, which can make mistakes.
Is this a good recommendation for you?



