Developmental Trajectories of Situation Modeling and Mentalizing in Transformer Language Models

Short summary

Research tracking how LLMs develop theory-of-mind reasoning across training stages finds false-belief understanding emerges late, depends on model size and training volume, and remains fragile to linguistic cues. Situation modeling precedes but shows surprising incoherence. Findings highlight the need for developmental and stress-testing approaches when evaluating LLM capabilities.

• •False-belief task performance depends on model size and sufficient training volume, emerging late in pretraining
• •Performance improves with post-training (SFT, DPO) but is fragile to linguistic cues like non-factive verbs
• •Situation modeling precedes theory-of-mind reasoning but shows surprising fragility in edge cases

Generated with AI, which can make mistakes.