Our Rust file watcher ate 23.6 GB of RAM, and our ignore rules never had a chance

Short summary

A local-first macOS indexer consumed 23.6 GB of RAM because notify-debouncer-full's FileIdMap cache walked the entire watch root including ignored directories, caching 54 million path allocations for files explicitly excluded. Switching to NoCache dropped memory to 1.45 GB at the cost of losing rename stitching. A second bug caused nested watch roots to trigger redundant rescan walks, embedding files twice.

• •FileIdMap cache walks entire watch root ignoring your ignore rules, caching 54M allocations
• •Switching to NoCache cut memory from 24.8 GB to 1.45 GB but loses rename event stitching
• •Nested watch roots caused duplicate rescans and double embeddings — fixed by de-nesting roots

Generated with AI, which can make mistakes.