WitnessGym: Benchmarking Coding Agents on the Construction of Bug Witnesses
WitnessGym is a framework for constructing bug-validation benchmarks for coding agents. It injects bugs into test-reached paths of real projects, rebuilds the projects, and retains cases exposed by a construction-time witness. The framework can be extended to additional bug types and languages, and has been evaluated with four coding agent frameworks across various bug types and execution contexts.
Save an API key to vote.