Back to feed
arXiv cs.LG
arXiv cs.LG
8/3/2026
LARA: Lightweight Adapters in the Residual Stream for Composable Adaptation and Alignment

LARA: Lightweight Adapters in the Residual Stream for Composable Adaptation and Alignment

Short summary

LARA introduces lightweight residual-stream adapters that correct hidden states at selected layers without modifying base weights, matching LoRA at equal parameter counts. Its key advantage is a scale parameter enabling smooth interpolation between base and adapted behavior at inference. Multiple behaviors can coexist on one frozen model—seven behaviors fit on a 1.5B model with ~33 MB overhead—and are routed per token, making it suitable for hosting many behaviors on-device.

  • •LARA adapts frozen models via residual-stream corrections, matching LoRA performance
  • •Scale parameter enables graded interpolation between base and adapted behavior
  • •Seven behaviors coexist on one 1.5B model with ~33 MB total overhead

Generated with AI, which can make mistakes.

#research-breakthrough#ai-tools#open-source
Read full article at arXiv cs.LG

Is this a good recommendation for you?

Comments

Failed to load comments. Please try again.

Explore more

The original title is "LoRA: Fine-Tune a Giant Model by Training 1% of It"
Dev.to
Quantized LoRA Adapters for On-Device LLMs: Hot-Swapping Task-Specific Behaviors on Android Without Reloading the Base Model
Dev.to
ReCoLoRA: Spectrum-Aware Recursive Consolidation for Continual LLM Fine-Tuning
arXiv cs.LG
Between Gradient and Natural Gradient: A Continuum of LoRA Initializations
arXiv cs.LG