AutoLoCo: Communication Efficient Distributed LLM Training via Adaptive Synchronization

AutoLoCo is a new adaptive training framework for Large Language Models (LLMs) that reduces communication frequency between accelerators during training, improving performance and efficiency. It adapts the local interval using scalar training statistics and corrects each outer update.

RSS Score 0 9/30/2026, 4:00:00 AM Original Source
Save an API key to vote.