Back to feed
arXiv cs.CL
arXiv cs.CL
7/21/2026
Are Arithmetic Heuristic Neurons Form-Invariant? A Mechanistic Analysis of Symbols, Text, and Code in LLMs

Are Arithmetic Heuristic Neurons Form-Invariant? A Mechanistic Analysis of Symbols, Text, and Code in LLMs

Short summary

This study investigates whether arithmetic heuristic neurons in LLMs are form-invariant across symbolic arithmetic, word problems, and Python code using three Llama-3 models. A compact set of shared neurons is identified across all formats, and transferring their activations from successful to failed executions recovers over 97% of incorrect predictions for addition and subtraction. The findings show that cross-format arithmetic failures stem from different activation states of a shared circuit rather than distinct circuits.

  • •Shared arithmetic heuristic neurons identified across symbolic, text, and code formats in Llama-3
  • •Activation transfer from successful to failed executions recovers >97% of addition/subtraction errors
  • •Cross-format failures arise from activation states, not distinct circuits — arithmetic is form-invariant at neuron level

Generated with AI, which can make mistakes.

#research-breakthrough
Read full article at arXiv cs.CL

Is this a good recommendation for you?

Comments

Failed to load comments. Please try again.

Explore more

Silent Failures in Quantized LLM Reasoning: A Taxonomy-Based Analysis of Hollow Convergence and Failure Mode Shifts
arXiv cs.CL
Formalizing Latent Thoughts: Four Axioms of Thought Representation in LLMs
arXiv cs.CL
The original title is "Some Large Language Models Exhibit Consistent Risk Attitudes"
arXiv CS.AI
Belief or Circuitry? Causal Evidence for In-Context Graph Learning
arXiv CS.AI