Low-Rank Adaptation (LoRA) — Concepts

Low-Rank Adaptation · LoRA

unseel.com · Freeze W · Train B·A · rank r

Trainable 100%
Rank r —

State —

Frozen W (d×k)
A (r×k)
B (d×r)
Merged W + BA

Unseel.com · LoRA