Neuromorphic Diffusion Language Models: Addressing Compute and Memory Bottlenecks via Sparsity and Block Denoising

Short summary

This paper proposes neuromorphic masked diffusion language models (N-MDLMs) that combine block diffusion with spike-based neuromorphic computation to improve inference throughput and energy efficiency. Block diffusion generates multiple tokens per parameter access while spike-induced sparsity skips inactive channels, reducing effective computation. Experiments on translation tasks show substantial energy and throughput gains, even on compute-bound platforms where standard MDLMs offer no advantage over autoregressive LLMs.

• •N-MDLMs combine block diffusion with spike-based neuromorphic computation
• •Spike-induced sparsity reduces parameter traffic and computation
• •Substantial energy efficiency and throughput gains on translation tasks

Generated with AI, which can make mistakes.