NAQD Env: A benchmark for selective withdrawal in language agents
The paper introduces NAQD-Env, a synthetic environment to evaluate language agents' selective withdrawal decisions. It assesses the agents' ability to suspend affected actions, preserve unaffected work, and resume after repair, using a deterministic reference policy. The evaluation results show that current models struggle with selective withdrawal, motivating further research in this area.
Save an API key to vote.