Beyond Prompt Count: How Data Shapes Transfer in On-Policy Distillation

This paper explores the effect of prompt choice on transfer in on-policy distillation, a technique for training AI agents. The study finds that a few well-chosen prompts can be as effective as a large pool of prompts, but the effectiveness depends on the teacher-student pair and target capability. This research has implications for the development of AI agents and the choice of prompts for on-policy distillation.

RSS Score 0 9/30/2026, 4:00:00 AM Original Source
Save an API key to vote.