Diffusion LLMs (dLLMs) generate tokens in parallel instead of sequentially. Inception's Mercury 2 just proved this works at production scale: ~1,000 tokens/sec, roughly 10x faster than comparable [[Large Language Model - LLMs]] while hitting competitive quality benchmarks. This matters because every approach to speeding up [[Inference]] so far has been about optimizing the same autoregressive loop. Specialized chips ([[SRAM]] for low-latency decode), serving stack tricks (KV cache, batching, kernel tuning), model compression. All of it accepts the sequential generation constraint and tries to work around it. Mercury 2 rejects the constraint entirely. Think about what that means for the [[Prefill and Decode]] split. The entire reason inference is disaggregating into specialized hardware is because decode is sequential and memory-bound. If diffusion-based generation can parallelize what was previously serial, the hardware optimization calculus shifts. The [[Nvidia-Groq - Inference Disaggregation Play]] thesis assumed autoregressive decode as a given. dLLMs challenge that assumption at the architectural level. The [[Batch Size]] economics flip too. Traditional inference providers optimize utilization by batching many requests together, amortizing weight-loading costs. My note on [[AI Inference Infrastructure]] argued that utilization variance matters more than performance variance. dLLMs add a third variable: what if the model itself is 10x more efficient per request? That changes the whole utilization equation. Where it gets interesting for agents: latency compounds across multi-step workflows. An agent making 20 model calls in sequence turns every millisecond of per-call latency into seconds of total delay. Mercury 2 at 1,000 tok/sec makes multi-step reasoning loops practical in production. This is the unlock that turns agents from demos to deployed systems. It also tightens the [[AI Verification]] loop: faster generation means more room for iterative self-checking within the same latency budget. The generation mechanism itself is worth noting. dLLMs start with noise and iteratively refine, like an editor revising a draft rather than a typewriter producing left-to-right. The [[Architecture of LLM]] note covers how autoregressive models maximize P(sequence) by predicting one token at a time. Diffusion inverts this: generate everything at once, then correct. [[Sparsity x LLMs]] tackled the efficiency problem by reducing computation within the autoregressive framework. dLLMs attack efficiency from the generation paradigm itself. The [[Conviction]] angle here is striking. Inception's founders (Stanford, UCLA, Cornell researchers who co-invented diffusion for images, Flash Attention, and DPO) spent years on this when the entire industry was doubling down on autoregressive scaling. Billions poured into making the typewriter faster. They bet that the typewriter was the wrong tool. That kind of conviction, building against consensus with a fundamentally different technical foundation, is exactly what separates deep tech breakthroughs from incremental optimization. Still early. Quality benchmarks put Mercury 2 in the range of Claude 4.5 Haiku and GPT-5.2 Mini, not frontier. The question is whether diffusion generation follows its own scaling curve, or whether quality hits a ceiling that autoregressive models have already broken through. If dLLMs scale the way diffusion scaled in images and video, the speed advantage compounds as models get bigger. That would restructure [[The AI Stack - Building Blocks]] from the model layer down. Worth watching closely. --- Links: - [[Inference]] | [[Inference is Eating AI Compute]] | [[AI Inference Infrastructure]] - [[Prefill and Decode]] | [[Batch Size]] | [[SRAM]] - [[Architecture of LLM]] | [[Large Language Model - LLMs]] | [[Transformers]] - [[Sparsity x LLMs]] | [[Sparse Transformers]] - [[Nvidia-Groq - Inference Disaggregation Play]] - [[AI Verification]] | [[The AI Stack - Building Blocks]] - [[Conviction]] --- #deeptech #kp #inference