Benchmarking 4 AI Code Reviewers: Greptile, CodeRabbit, Sentry Seer, and Cursor BugBot (146 PRs, 679 Findings)

Short summary

A dev team benchmarked four AI code reviewers (CodeRabbit, Sentry Seer, Greptile, Cursor BugBot) over 3.5 weeks on 146 PRs, collecting 679 findings with transparent methodology. Greptile led on precision (92% bug-shaped, zero false positives), CodeRabbit on breadth (281 findings with 68% applicable patches), Seer on critical issues. The open-sourced dataset enables independent verification.

• •Greptile highest on precision: 92% bug-shaped findings, zero false positives, 51 P1 findings
• •CodeRabbit highest volume: 281 findings across 82 PRs with 68% one-click patches
• •Sentry Seer best at critical classification: 6/6 perfect accuracy on critical issues
• •Open-sourced dataset and ingester available for reproducible testing on own codebases

Generated with AI, which can make mistakes.