Harvest: An open-source self-healing web scraper with semantic caching and MCP support

Short summary

Harvest is an open-source web scraper that auto-repairs its selectors when websites change, using an LLM to generate new CSS selectors from old and new HTML. It features semantic caching (50-70% token reduction), structural diffing, zero-token scraper generation, and runs as an MCP server for AI agent integration. Limitations include Cloudflare Turnstile blocks and a brand-new community with no established track record.

• •Self-healing scraper uses LLM to regenerate broken CSS selectors when site HTML changes
• •Semantic caching by meaning reduces token costs 50-70% on repeated extraction queries
• •Runs as MCP server; includes comparison against Crawl4AI and Firecrawl

Generated with AI, which can make mistakes.