Back to feed
Dev.to
Dev.to
7/13/2026
The original title is: "I built an open-source MCP server so AI agents can read the web - even sites that block bots"

The original title is: "I built an open-source MCP server so AI agents can read the web - even sites that block bots"

Original: I built an open-source MCP server so AI agents can read the web - even sites that block bots

Short summary

HatFetch is an MIT-licensed MCP server that gives AI agents like Claude and Cursor the ability to read web pages, including JavaScript-rendered sites and those that block bots. It uses an escalation strategy: fast HTTP fetch first, then a stealth headless browser with residential proxies if needed, and optional CAPTCHA solving. The author honestly notes no tool bypasses 100% of anti-bot defenses in 2026.

  • MIT-licensed MCP server for AI agent web access with escalation from HTTP to stealth browser
  • Handles JS-rendered sites, geo-restrictions, and IP-reputation blocks; coin-flip on hardest targets
  • Integrates with Claude Desktop, Cursor, and any MCP-compatible client via npx

Generated with AI, which can make mistakes.

Is this a good recommendation for you?

Comments

Failed to load comments. Please try again.

Explore more