Harvest: An open-source self-healing web scraper with semantic caching and MCP support

Short summary

Harvest is an open-source, MIT-licensed web scraper that uses LLMs to auto-repair CSS selectors when websites change, caches extraction results by semantic meaning (50-70% token reduction), and runs as an MCP server for AI agents like Claude and Cursor. It includes structural diffing, a script generator for zero-token runtime scraping, and built-in Cloudflare bypass. The project is new with no community yet but offers a compelling feature set compared to paid alternatives like Firecrawl.

• •Open-source self-healing web scraper that auto-regenerates broken CSS selectors via LLM
• •Semantic caching reduces token usage 50-70% by matching meaning not exact words
• •Runs as MCP server for Claude/Cursor; includes comparison table vs Firecrawl and Crawl4AI

Generated with AI, which can make mistakes.