Formalizing Latent Thoughts: Four Axioms of Thought Representation in LLMs

Short summary

Researchers introduce an axiomatic evaluation framework for LLM latent thought representations using four axioms (Causality, Minimality, Separability, Stability) independent of benchmark scores. Testing across 23 reasoning tasks and multiple model families, they find no LLM satisfies all axioms simultaneously, and representations encode little information beyond input embeddings. The failure is structural and consistent across model sizes and training methods, indicating fundamental limitations.

• •Four-axiom framework (Causality, Minimality, Separability, Stability) evaluates LLM thought representations independently of downstream performance
• •No tested LLM satisfies all axioms; representations distinguish task type but not questions within same task
• •Structural limitation consistent across dense, distilled, and RL-trained models regardless of scale

Generated with AI, which can make mistakes.