Researchers investigated the performance of embodied LLMs in a physical robotic setup with varying levels of observation fidelity. They found that LLMs performed best under raw RGB input and worst under perfect ground-truth observations. This suggests that measured performance may not reflect robust problem-solving abilities, but rather the interaction between perceptual errors and reasoning failures.

This paper explores the impact of masking-based token pruning on the robustness of CLIP models. It proposes a pre-deployment diagnostic called the Spurious Inversion Metric (SIM) to predict whether masking helps or hurts worst-group robustness. The study finds that masking can have a significant impact on model performance and introduces a batched GPU segmentation routine to mitigate its drawbacks.

A new method, HammingMark, is proposed for robust and efficient LLM watermarking. It uses the semantic hash of the preceding sentence as a dynamic center and accepts candidates whose hashes fall within its Hamming neighborhood. This approach retains a larger fraction of naturally likely semantic continuations and achieves strong robustness, high detectability, and near-unwatermarked generation quality.

Researchers propose a unified evaluation protocol for robust counterfactual explanations across different types of model changes, including parameter perturbations, retraining on new data, and new architectures. They evaluate six robust methods and two standard baselines on four tabular datasets, finding that relative performance and failure modes vary across change families.