Less Uniform Discrete Diffusion is More Powerful and Scalable
Researchers propose a novel framework, Less Uniform Diffusion (LUDI), to improve the scalability of Uniform Diffusion Language Models (UDLMs). LUDI addresses the issues of over-uniform training objectives and condition-target confusion in UDLMs by introducing a less uniform loss and token-level corruption hints. This allows for confidence-based few-step sampling, resulting in cleaner supervision and improved generation capabilities.
Save an API key to vote.