The original title is "LoRA: Fine-Tune a Giant Model by Training 1% of It"

Short summary

LoRA cuts fine-tuning memory 256× by training only small adapter matrices (2% of parameters) instead of updating all 7B weights, enabling 70B models on single consumer GPUs. One frozen base model is shared across dozens of task-specific adapters (a few MB each), making production multi-variant serving and large-scale community sharing practical. Works best when the needed change is low-rank; entirely novel behaviors may require higher rank or full fine-tuning.

• •LoRA trains only 1% of parameters using low-rank decomposition (ΔW ≈ B·A), reducing fine-tuning memory 256× and enabling consumer-GPU training of 70B models
• •Frozen base model shared across dozens of small task adapters enables efficient production deployment and large-scale community sharing of specialized variants
• •Assumes the required change is low-rank; genuinely novel behaviors far from pretraining may plateau and require higher rank or full fine-tuning

Generated with AI, which can make mistakes.