Reliable, and still wrong

Short summary

A large-scale audit of over half a million AI judgments reveals that AI judges are reliably consistent but often incorrect—a critical flaw because the field has mistakenly treated consistency as proof of trustworthiness. The researchers demonstrate how a judge that always picks option A would score perfect consistency yet be completely wrong. This directly impacts any product that uses AI to evaluate AI.

• •AI judges are consistent but often incorrect—the field conflates reliability with validity
• •A judge that always picks A is perfectly reliable yet completely worthless
• •Directly impacts product decisions relying on AI evaluation systems

Generated with AI, which can make mistakes.