Exposing the Unsaid: Visualizing Hidden LLM Bias through Stochastic Path Aggregation

Short summary

TreeTracer is a visual analytics tool for detecting hidden biases in LLMs through stochastic path aggregation. It uses perturbation analysis to replace ontology-defined terms, aggregates hundreds of generation branches into syntax-aligned hierarchies, and visualizes comparisons via Sankey diagrams with contrastive inference. Validation on GPT-2 XL and Apertus models revealed representational harms like pronoun suppression and conversational marginalization.

• •TreeTracer visualizes hidden LLM biases through aggregated stochastic generation analysis
• •Uses perturbation analysis and contrastive inference to detect harms in lower-probability generation branches
• •Identified concrete biases: pronoun suppression and conversational marginalization in tested models

Generated with AI, which can make mistakes.