Latest Sessions
View All Sessions
S27 | From Interface to Inference: Eliciting Any-Order Inference from Any-Order Models
Seunggeun Kim and Jaeyeon Kim show that the any-order prediction interface of masked diffusion models does not by itself deliver any-order inference, because fixed-canvas token-level models may know what semantic component should appear without knowing where to place it. They propose two fixes: insertion-based masked diffusion, which relaxes fixed-position commitments and generates across non-contiguous regions, and latent-space masked diffusion, which predicts coarser semantic segments and enables search over latent generation orders. A 7B FlexMDM for Python coding and a 125M LatentMDM for GSM8K both show distinct any-order inference behaviors and improved downstream performance.
S26 | Self-conditioned Flow Map Language Models via Fixed-point Flows
Jaehoon Yoo and Wonjung Kim study self-conditioning in flow-based language models, where the model denoises text while conditioning on its own denoising estimate. They show that self-conditioning solves a fixed-point iteration that bootstraps the learned denoiser, and they use this view to define fixed-point flows, a two-dimensional class of self-conditioned flows spanning the flow process and the fixed-point iteration. Distilling both dimensions yields FMLM★, a flow map language model that outperforms state-of-the-art self-conditioned and few-step models in one- and few-step generation on OpenWebText.
S25 | FLARE: Diffusion for Hybrid Language Model
Yuchen Zhu (Georgia Tech) presents FLARE, a framework that turns a hybrid-attention autoregressive (AR) LLM into one that also runs diffusion-style parallel decoding, so a single checkpoint supports both AR verified decoding and diffusion denoising. The authors find that transfer data quality, more than the loss formulation or attention-mask design, decides how much of the seed checkpoint's capability survives. Starting from strong AR checkpoints with limited post-training data, FLARE matches leading open-source diffusion LLMs across scales and delivers consistent throughput gains in single-GPU concurrent serving.
Featured Videos
View All Videos
How did diffusion LLMs get so fast?
Techniques for accelerating diffusion LLMs, from self-distillation and curriculum learning to KV caching and block diffusion
But How Do Diffusion Language Models Actually Work?
Jia-Bin Huang explores several ideas for applying diffusion models to language modeling
Simple Diffusion Language Models
Quick introduction to Masked Diffusion Language Models (MDLM) by Alexander Rush
About the Reading Group
Diffusion LLMs are faster, more controllable successors to traditional LLMs and are rapidly gaining adoption. This reading group builds a community for exchanging and debating emerging ideas in this space. While our primary focus is discrete diffusion models for language, we also welcome work on other modalities and applications, such as molecular design, drug discovery, and beyond.
Meet the Organizers
Subham Sahoo
Holds a Ph.D. from Cornell Tech, where he specialized in Diffusion Language Models. He has made foundational contributions to the field, with his work deployed at scale by Google, NVIDIA, and ByteDance across language generation and drug discovery.
Justin Deschenaux
PhD student in Machine Learning at EPFL, advised by Prof. Caglar Gulcehre. Previously interned at Apple MLR. His research interests include diffusion language models, fast generative models, and generalization.
