This paper proposes Transferable Example Scoring and Selection (TESS), a scalable data-selection framework for training large language models. TESS uses a Pointwise Value Matching objective to improve transferability and generalization.

A new method for selecting synthetic data to fine-tune large language models (LLMs) is proposed, focusing on maximizing the value of synthetic data to the target task. The method, called Training-Aware Target Coverage (TATC), is based on a linear theory that characterizes the tradeoff between the benefits and errors of synthetic data. Experimental results show that TATC outperforms alternative methods on various tasks.

Researchers compared mathematical training data for Large Language Models (LLMs) using two approaches: topic and reasoning approach. They found that using a reasoning approach, where sources share the target's method but change the topic, resulted in better transfer after fine-tuning than using a topic-based approach.

EnvACE is a new agentic reinforcement learning method that internalizes environment dynamics by replacing external environment interaction during training with world rehearsal. This allows for strong and transferable performance across various benchmarks, including FinMCP-Bench.

This paper presents a study on Group Relative Policy Optimization (GRPO) fine-tuning for small language models (SLMs) under a practical compute budget. The study analyzes the effect of group size on policy convergence, training stability, and downstream benchmark performance, and provides practical guidance for GRPO training for SLMs.

dattri-LLM is a unified and efficient library for training data attribution at LLM scale, achieving 3.2x the throughput of the fastest competing library and scaling multiple attribution methods to 110B-parameter models across four H200 GPUs.

AutoLoCo is a new adaptive training framework for Large Language Models (LLMs) that reduces communication frequency between accelerators during training, improving performance and efficiency. It adapts the local interval using scalar training statistics and corrects each outer update.

Researchers introduce Interactive-Policy Distillation (IPD), a method that applies adaptive teacher intervention to the student rollout in on-policy distillation, improving performance and data efficiency. IPD trains a student model on its self-generated trajectories with dense token-level teacher feedback, and demonstrates higher accuracy and efficiency compared to traditional on-policy distillation (OPD).