On Distinguishing Capability Elicitation from Capability Creation in Post-Training: A Free-Energy Perspective

Short summary

Researchers propose distinguishing capability elicitation (reweighting existing model behaviors) from capability creation (expanding what a model can practically reach) using a free-energy framework. Both SFT and RL primarily reweight pretrained distributions; true capability creation requires search, interaction, tool use, or new information incorporation. This framework shifts focus from SFT vs. RL framing to whether post-training expands the model's behavioral support.

• •Introduces 'accessible support' concept to formalize what behaviors a model can practically produce
• •Shows SFT and RL both operate via distribution reweighting, not fundamentally different mechanisms
• •Argues true capability creation requires expanding reachable behaviors via search, tools, or interaction

Generated with AI, which can make mistakes.