Back to feed
arXiv cs.LG
arXiv cs.LG
7/21/2026
Orthogonal Gradient Constraints Shape Noisy-Label Memorization Dynamics

Orthogonal Gradient Constraints Shape Noisy-Label Memorization Dynamics

Short summary

OrthoGrad is a geometric optimizer intervention that removes the gradient component parallel to current weights, tested on noisy-label image classification. It improves test accuracy for CNNs on small-data MNIST regimes and reduces corrupted-label fitting, but CIFAR-10 ResNet-18 experiments show it alters memorization trajectories without fully preventing them. The method is regime-dependent, working best when gradients have a nontrivial radial component.

  • •OrthoGrad projects out the gradient component parallel to weights, a geometric regularization approach
  • •Improves CNN test accuracy on small-data MNIST but is regime-dependent, not universally regularizing
  • •CIFAR-10 experiments show altered but not prevented noisy-label memorization

Generated with AI, which can make mistakes.

#research-breakthrough
Read full article at arXiv cs.LG

Is this a good recommendation for you?

Comments

Failed to load comments. Please try again.

Explore more

Gradient Smoothing: Coupling Layer-wise Updates for Improved Optimization
arXiv cs.LG
Regularity-Aware Stochastic MGDA with Adaptive Conflict-Avoidant Update Direction Control
arXiv cs.LG
LiST: Lipschitz Scaling Training for Robust and Calibrated Neural Networks
arXiv cs.LG
A Link between Shock-wave Theory and Symmetry-reduced Stochastic Gradient Descent for Artificial Neural Networks
arXiv cs.LG