MIITA: Memory-Induced Inference-Time Adaptation for Continual Learning with Small Language Models

Short summary

MIITA is a framework for continual learning in small language models that stores supervised experiences as compact correction-direction prototypes retrieved at inference time via semantic and uncertainty cues. It applies retrieved directions through gated temporary hidden-state adaptation, avoiding backbone updates, prompt extensions, or test-time backpropagation. Experiments show consistent performance improvements and forgetting mitigation under fixed memory budgets.

• •Proposes memory-induced inference-time adaptation for SLMs under constrained storage
• •Uses compact correction-direction prototypes with semantic anchors for non-destructive knowledge reuse
• •Demonstrates consistent gains across diverse supervised continual learning settings

Generated with AI, which can make mistakes.