ResonatorLM: Causal Resonant Field Mixing for Efficient Long-Context Language Modelin

Short summary

ResonatorLM proposes replacing self-attention with a physics-inspired mechanism that treats token sequences as a driven 1D latent field using causal damped resonator functions. In a 6M parameter matched setting, it achieves 6.47x decode speedup over an optimized transformer at 32K tokens and improves WikiText accuracy from 55.32% to 61.31%. The approach targets the long-context efficiency bottleneck that limits transformers, RNNs, and CNNs alike.

• •ResonatorLM replaces attention with causal damped resonator functions over a 1D latent field
• •6.47x decode speedup vs optimized transformer at 32K tokens in 6M param setting
• •WikiText accuracy improved from 55.32% to 61.31% in matched comparison

Generated with AI, which can make mistakes.