This paper introduces a pipeline for generating synthetic data and a self-improvement loop to optimize conversational recommendation agents, improving their quality by 8% and accelerating iteration cycles. The approach has been productionized and demonstrated effectiveness in online A/B tests at Spotify.