A new approach, Loki, adapts pretrained models to predict new classes without additional training by using a metric that relates labels via distances. This can improve model performance on zero-shot prediction tasks.

Researchers propose a new approach to mitigating model collapse in iterative fine-tuning using a non-parametric entropy rate estimator. The approach does not require a model or external data and shows significant improvements in text diversity metrics.

A research paper proposes a new method for compressing large language model attentions, called FTC, which improves perplexity on several downstream tasks without requiring fine-tuning or gradient-based recovery.

MatrixReward proposes a reward mechanism for open-ended generation by constructing rewards from a rollout-by-rubric win-rate matrix. Compared to previous methods, MatrixReward achieves an average score of 63.02, outperforming the strongest baseline by approximately 2.0%.

The paper proposes FAER, an auditable full-trajectory replay framework for language models, addressing the gap between selection and learning objectives. It introduces a training-free fixed selector and a learner-aware selector fitted on disjoint calibration blocks.

ShamAN-Q is a sub-1-bit post-training quantization method for large language models (LLMs), building on the NanoQuant method. It uses a tractable dense curvature metric and Kullback-Leibler minimization to fit a Kronecker product to the empirical Fisher information matrix. ShamAN-Q improves perplexity on the WikiText-2 dataset and matches zero-shot accuracy on the Eleuther LM Evaluation Harness.