LoRA: Steering Billions of Weights by Training a Few Million
LoRA fine-tunes a giant model by training two skinny matrices per weight matrix. Count the 0.39%, watch an adapter train, and fit a run on one GPU.
Fine-tuning: Teaching a Know-It-All to Be Helpful
A base model knows Paris but answers with another quiz question. Fine-tuning fixes it with the same loop on new data. Mask the loss, then watch a model forget.