Zero2Repo: Can Coding Agents Build Repositories from Scratch?
A new benchmark, Zero2Repo, has been introduced for evaluating the ability of coding agents to build software repositories from scratch. The benchmark consists of a pipeline that converts real open-source projects into behavioral specifications, reproducible environments, and acceptance tests, and evaluates agent performance by running them in isolated containers and withholding acceptance tests until explicit submission.
Save an API key to vote.