Researchers evaluated 7 TDD methods on 8 CodeLLMs, introducing CodeSnitch, a function-level benchmark dataset. The study assessed robustness under code clone detection taxonomy.

A dataset of real-world coding agent sessions from open-source developers is presented, providing an empirical characterization of agent usage and failure modes. The dataset shows that agents remain inefficient in natural settings and introduce more security vulnerabilities than human-authored code.

Researchers introduced VeriSpec, a novel approach to detect inconsistencies in model specifications using large language models as verifiers. VeriSpec extracts rules from the specification text, clusters related rules, and applies LLM-as-verifier reasoning to identify inconsistencies. The approach achieved high precision and efficiency in detecting defects in the OpenAI Model Spec, outperforming several baselines.

Paper explores the effectiveness of cross-model review in LLM verification, finding that a second model does not always improve error detection.

Researchers investigated how weak reviewers can audit strong coding agents. They found that official execution evidence can improve defect catch and reduce over-rejection. The study used 411 execution-labeled traces from three agents and 101 controlled cases.

A study of open-source LLM-based multi-agent systems identifies common issues, their causes, and potential solutions. The most common issue is orchestration and execution, with causes including workflow problems, tool integration issues, and memory problems. The study suggests optimizing workflows as a solution.

A study evaluates the environment specification capabilities of large language models (LLMs) in generating software code. The research highlights systematic generalization failures in current LLMs, leading to inconsistent, redundant, or incomplete dependency specifications. This affects the portability and execution of generated code.

A continuous evaluation framework for enterprise AI agent skills is proposed, combining outcome-level and process-level checks to detect behavioral drift in skills due to changing tool APIs, models, and specifications.

The authors propose the MCRI Framework, a four-dimensional framework for analyzing and evaluating agent skills, and operationalize it as MCRI-Eval, a large language model-based evaluation method. They evaluate MCRI-Eval on 63,812 public skills from the OpenClaw skill Hub and achieve promising results, including improved skill selection and ranking agreement.

Researchers compare the performance of 66 configurations of language models paired with different harnesses, finding that the best model-harness pairing can vary depending on the task, and that a higher cost does not always buy a higher score. The study provides a framework for evaluating the fit between models and harnesses, and releases the evaluation code and harness adapters for reproducibility.

ActiveSaddler, a new automated curriculum learning method, optimizes LLM agent harnesses by dynamically updating prompts, tool interfaces, and control logic from execution feedback, improving test Pass@1 by 4.4-7.5 percentage points.

Researchers present Rules to Tools, a system of executable checks for LLM agents in scientific computing. The system improves repair outcomes and reduces agent-side costs.

A benchmark for evaluating the ability of AI agents to generate user-facing documentation. The DoGBENCH benchmark evaluates agents' performance in producing accurate and complete documentation for open-source projects. The results show that current agents struggle with tasks such as describing interfaces and providing decisive evidence, with failure modes identified in a separate audit.

This paper introduces HealBench and HealGuard, a benchmark and guardrail for trustworthy runtime error healing in real-world repositories. Researchers evaluate LLM-based healing code and detect potential safety issues with existing agents.

Researchers introduce Approval Laundering, a taxonomy of six failure modes in AI coding-agent harnesses that silently substitute one action for another after approval. They evaluate these modes using a controlled study and prototype Approval Token, a capability that eliminates two of the failure modes.

Adaptive-GEPA is a new approach to optimizing language model prompts for heterogeneous requests. It learns to divide tasks and solve them using a router and a library of specialist programs. This results in better performance and a more efficient use of compute resources.

OpenCollab is a multi-agent coding framework that enables programmable collaboration and controllable runtime. It addresses the challenge of evaluating complex software engineering tasks by allowing for flexible organization design, experimental control, and fine-grained event tracking. The framework is shown to achieve state-of-the-art performance in agentic coding benchmarks, outperforming existing harnesses.

Researchers introduce E2E-SWE, a benchmark for evaluating the ability of LLM-powered coding agents to build complete, functional software repositories from scratch. The benchmark contains 186 tasks across 11 programming languages, with varying levels of success across 13 frontier models.

A new benchmark, Zero2Repo, has been introduced for evaluating the ability of coding agents to build software repositories from scratch. The benchmark consists of a pipeline that converts real open-source projects into behavioral specifications, reproducible environments, and acceptance tests, and evaluates agent performance by running them in isolated containers and withholding acceptance tests until explicit submission.

This paper proposes Runtime Assurance Contracts (RACs) for high-risk AI agents to ensure autonomy boundaries and evidence-based decision-making. RACs define a formal schema for policy-level binding, evidence state, and transition policy, and demonstrate their effectiveness in various scenarios.