How Much Do Circuits Tell Us? Measuring the Consistency and Specificity of Language Model Circuits

Short summary

A mechanistic interpretability study using edge attribution patching across seven language models found that circuits—sparse subgraphs identified as causally important—are not task-specific. Components critical to one task cause similar performance drops in others, revealing substantial overlap. These findings question whether circuits truly support targeted understanding and intervention on model behavior.

• •Circuits show high within-task component reuse but lack task-specificity across different tasks
• •Ablating components from one task's circuit damages other tasks' performance similarly
• •Substantial overlap between task circuits challenges the circuit framework's interpretability value

Generated with AI, which can make mistakes.