Researchers introduce E2E-SWE, a benchmark for evaluating the ability of LLM-powered coding agents to build complete, functional software repositories from scratch. The benchmark contains 186 tasks across 11 programming languages, with varying levels of success across 13 frontier models.

DatalogBench is a benchmark for evaluating large language models (LLMs) on text-to-Datalog synthesis tasks. It consists of 136 curated tasks and uses execution on held-out inputs to grade synthesized programs. The results show that current LLMs struggle with recursive reasoning and decomposition, and two coding agents improve performance by up to 83.8%.

Researchers introduced CRJudgeBench, a benchmark for evaluating AI's ability to detect technically incorrect code-review comments. They also presented Sentinel, a repository-grounded agentic judge that uses iterative action-level learning to improve accuracy. The study showed that even state-of-the-art LLMs struggle to identify untrustworthy comments, but Sentinel outperformed its base model and a competitor model by a significant margin.

DynBranch is a speculative subgraph reuse technique for dynamic agentic LLM serving. It reduces latency by up to 32% and 46-66% by reusing completed subgraph results across requests, without requiring changes to agent harnesses or model execution engines.