Back to feed
Dev.to
Dev.to
7/20/2026
We scored major websites on AI readiness. A national newspaper got 33/100.

We scored major websites on AI readiness. A national newspaper got 33/100.

Short summary

AI assistants like ChatGPT, Claude, and Perplexity are becoming a referral channel, but most sites aren't checked for AI readability. A three-point audit—llms.txt presence, AI crawler access in robots.txt, and sitemap availability—scored WordPress.org at 88 and the Irish Times at 33. The author built a paid tool (llms.txt Checker) for batch scoring at ~$5 per 1,000 sites, with fixes typically taking a day's work.

  • Three checks determine AI readability: llms.txt, AI crawler access in robots.txt, and sitemap
  • Irish Times scored 33/100 with GPTBot explicitly blocked; WordPress.org scored 88
  • Fixes are usually a day's work: write llms.txt, audit robots.txt, confirm sitemap

Generated with AI, which can make mistakes.

Is this a good recommendation for you?

Comments

Failed to load comments. Please try again.

Explore more