The original title is: "One "+x" That Made 100-Layer Networks Trainable: ResNet Skip Connections"

Short summary

ResNet fixed the vanishing gradient problem with skip connections: instead of learning full mappings H(x), residual blocks learn F(x) = H(x) - x and add the input back. In backpropagation, residual blocks ensure d(x + F(x))/dx = 1 + F'(x), keeping gradient products near 1 rather than near 0 through 152+ layers. This architectural pattern became foundational to transformers and modern LLMs.

• •Skip connections allow networks to learn residual corrections rather than full mappings
• •Mathematically prevent vanishing gradients by keeping backprop multipliers near 1 instead of 0
• •Now fundamental to transformers, U-Nets, and all modern deep learning

Generated with AI, which can make mistakes.