Researchers propose Sharpening Tax, a diagnostic metric to measure the loss in test-time scalability after post-training large language models (LLMs). They also present a Bayesian sampler, posterior-tempered group sampling (PTGS), which adapts the sampling temperature per prompt to its difficulty, and show that it pays a smaller Sharpening Tax than a fixed-temperature baseline.

A new method, PACE, is proposed for controlling staleness in asynchronous reinforcement learning (RL) for large language model post-training. PACE improves validation accuracy and reduces GPU time, matching synchronous RL performance.