Policy gradient methods have significantly advanced the reasoning capabilities of LLMs, particularly through RL. A key tool in stabilizing these methods is Kullback-Leibler (KL) regularization, which discourages drastic changes…
Off-Policy Reinforcement Learning RL with KL Divergence Yields Superior Reasoning in Large Language Models
