Sakana AI’s Error Diffusion Trains Dale-Compliant Dual-Stream Networks, Reaching 96.7% MNIST and 61.7% CIFAR-10 Without Backpropagation

Short summary

Sakana AI introduces Error Diffusion, a training method that avoids backpropagation's weight transport problem by routing errors through dual-stream excitatory/inhibitory networks compliant with Dale's principle. The approach reaches 96.7% accuracy on MNIST and 61.7% on CIFAR-10, and extends to reinforcement learning. Task-dependent ablations reveal how modulo error routing scales across learning scenarios.

• •Error Diffusion replaces backpropagation with modulo error routing in Dale-compliant dual-stream networks
• •Achieves 96.7% on MNIST and 61.7% on CIFAR-10, also applicable to reinforcement learning
• •Task-dependent ablations provide scaling insights across different learning scenarios

Generated with AI, which can make mistakes.