Three checks that lied to me while building a content extraction API

Short summary

A developer shares three debugging lessons from building a web content extraction API where passing tests gave false confidence. The benchmark included 40% non-article URLs, the sandbox test inherited the app's own --no-sandbox flag, and CI failures were masked by shared Playwright browser caches. The key takeaway: before trusting any check, make it fail on purpose to verify it actually tests what you think it does.

• •Benchmark success rate was inflated by 40% non-article URLs in the test set
• •Sandbox test passed because Playwright injects --no-sandbox by default
• •Rule: before trusting a check, make it fail on purpose to validate it

Generated with AI, which can make mistakes.