Back to feed
arXiv cs.CL
arXiv cs.CL
7/1/2026
Using AI Agents to Automate Black-Box Audits of Personalization Algorithms at Scale

Using AI Agents to Automate Black-Box Audits of Personalization Algorithms at Scale

Short summary

Researchers deploy a framework for black-box auditing of personalization algorithms using generative AI agents as synthetic users with realistic personas grounded in survey data. A study of 1,120 agents on X post-2024 election found the algorithmic feed amplifies toxic, polarizing, and right-leaning content versus chronological feeds, with effects varying by user ideology and demographics. Counterfactual analysis reveals how platform-visible signals like age and location affect content delivery in persona-dependent, non-linear ways.

  • •Generative AI agents deployed as synthetic auditors for black-box personalization algorithms
  • •X's algorithmic feed amplifies polarizing and toxic political content versus chronological feed
  • •Demographic effects are persona-dependent; pooled analysis masks significant subgroup variation

Generated with AI, which can make mistakes.

#research-breakthrough#ai-agents#regulation-policy#ai-tools
Read full article at arXiv cs.CL

Is this a good recommendation for you?

Comments

Failed to load comments. Please try again.

Explore more

A protocol for auditing AI agent harnesses
Dev.to
The original title is "Auditing the Audit: Five Failure Modes in Benchmark-Validity Audits"
arXiv cs.LG
What Models Express, Suppress, and Resist: Auditing Open-Weight LLMs with Persona Vectors
arXiv cs.CL
The original title is: "No one reads privacy policies. So I built 6 AI Agents to do it for me."
Dev.to