Beyond Shapley: An Influence-Based Data Auditing Pipeline for LLM Alignment and Evaluation

Short summary

This paper introduces a scalable, inference-only data valuation pipeline that approximates Shapley values without model retraining, using semantic k-NN graphs and conditional log-likelihood shifts from a reference LLM. Applied to HelpSteer2, it reduced the manual audit search space by 99.1% and uncovered falsely-labeled records. Applied to Anthropic's HH-RLHF, it identified thousands of hidden safety and factual preference inversions, exposing vulnerabilities in current benchmark integrity where capable models are penalized by flawed human labels.

• •Scalable inference-only pipeline approximates Shapley values via k-NN graphs and LLM log-likelihood shifts
• •Reduced HelpSteer2 audit search space by 99.1% while uncovering false labels across diverse failure modes
• •Found thousands of hidden safety and preference inversions in Anthropic HH-RLHF, exposing benchmark integrity flaws

Generated with AI, which can make mistakes.