Back to feed
Dev.to
Dev.to
7/11/2026
What an agent pays to read your site

What an agent pays to read your site

Short summary

AI agents pay token costs to read your website, and heavy HTML pages make those costs high and reliability low. By implementing content negotiation — serving markdown to agents and HTML to browsers — publishers can cut agent token costs by roughly two-thirds. An llms.txt file provides a site map so agents can read structure in one request instead of crawling page by page.

  • Serving markdown via content negotiation cuts agent token costs by ~66% vs HTML
  • An llms.txt file gives agents a site map in a single request
  • Cheaper pages mean agents read full content and represent your facts accurately

Generated with AI, which can make mistakes.

Is this a good recommendation for you?

Comments

Failed to load comments. Please try again.

Explore more