The original title is "I Built What I Benchmark. Here's How I Try Not to Cheat."

Short summary

A solo ESLint plugin author details their self-benchmarking process to avoid vendor bias: public fixture suites built before the rules they test, disclosures placed before results, and fully reproducible runs. The article shares two cases where the process caught errors, including a stale Node version that silently zeroed out eslint-plugin-unicorn's score from 22/40 to 0/40. The approach borrows from the OWASP Benchmark Project and Warren Buffett's practice of opening with his own worst calls.

• •Public fixture suite built before rules, with reproducible clone-and-run instructions
• •Disclosures placed before results, not after, to inform rather than cover
• •Two real cases where the audit process caught silent failures in benchmark runs

Generated with AI, which can make mistakes.