Beyond Prompt Count: How Data Shapes Transfer in On-Policy Distillation
This paper explores the effect of prompt choice on transfer in on-policy distillation, a technique for training AI agents. The study finds that a few well-chosen prompts can be as effective as a large pool of prompts, but the effectiveness depends on the teacher-student pair and target capability. This research has implications for the development of AI agents and the choice of prompts for on-policy distillation.
Save an API key to vote.