Interpreting Latent CoT Reasoning as Dynamical Systems

Short summary

This paper models latent chain-of-thought (CoT) reasoning as trajectories in representation space and applies dynamical systems analysis to understand how reasoning evolves across latent steps. Using measures like Lyapunov sensitivity and direction consistency, the authors show CODI behaves as a stable attractor while COCONUT is an unstable expanding system. SIM-CoT supervision tightens both behaviors without altering underlying dynamics, providing actionable insights for improving latent reasoning.

• •Models latent CoT token sequences as dynamical system trajectories using Lyapunov sensitivity and direction consistency
• •CODI behaves as a stable attractor; COCONUT as an unstable expanding system — two distinct stability classes
• •SIM-CoT supervision tightens dynamics without changing underlying behavior, offering insights for latent reasoning improvement

Generated with AI, which can make mistakes.