Preferred, Not Safer: Pairwise Preference Is a Poor Proxy for Clinical Safety

Short summary

A study using 26,804 pairwise judgments from 736+ clinicians across 28 countries finds that clinician pairwise preference is a poor proxy for clinical safety in LLM evaluation. Models ranking highly on preference still exhibit substantial clinically meaningful failures on Harmlessness and Accuracy, with failures unevenly distributed across specialties creating hidden 'no-go zones.' The authors propose a clinically adjusted preference ranking that combines pairwise preference with rubric-derived safety feedback.

• •Clinician pairwise preference does not reliably indicate clinical safety in LLM outputs
• •Safety failures are unevenly distributed across medical specialties, invisible in aggregate rankings
• •Proposed clinically adjusted ranking combines preference with rubric-based safety scoring

Generated with AI, which can make mistakes.