The original title is "Predictable GRPO: A Closed-Form Model of Training Dynamics"

Short summary

Researchers develop a first-principles mathematical model explaining Group Relative Policy Optimization (GRPO) training dynamics in large language models, replacing empirical curve-fitting with mechanistic predictions of training behavior. The model subsumes prior saturation laws and yields predictions for group-size invariance, stability thresholds, and failure modes. Validated across three models with R² ≥ 0.91, providing diagnostics to distinguish reward hacking, advantage degeneracy, policy concentration, and dynamical instability.

• •Develops closed-form mathematical model of GRPO training dynamics, replacing empirical fitting with first-principles analysis
• •Predicts group-size invariance, stability thresholds, and overdamped-to-oscillatory transitions with independently measurable quantities
• •Validated on three models with R² ≥ 0.91; provides diagnostics to separate failure modes including reward hacking and policy concentration

Generated with AI, which can make mistakes.