23.3K
DA5 techniques to fine-tune LLMs, explained visually 🧠
Traditional fine-tuning (adjusting all parameters) is infeasible with LLMs. These models have billions of parameters and are hundreds of GBs in size, and not everyone has access to such computing infrastructure.
Thankfully, today, we have many optimal ways to fine-tune LLMs.
Five popular techniques are depicted below:
1️⃣ LoRA
Add two low-rank matrices, A and B, alongside weight matrices W, which contain the trainable parameters.
Instead of fine-tuning W, adjust the updates in these low-rank matrices.
2️⃣ LoRA-FA
While LoRA considerably decreases total trainable parameters, it still demands high activation memory to update low-rank weights.
LoRA-FA (FA = Frozen-A) freezes matrix A and only updates matrix B.
3️⃣ VeRA
In LoRA, every layer has different A and B matrices, both trained.
In VeRA, matrices A and B are frozen, random, and shared across all layers.
VeRA learns small, layer-specific scaling vectors (b and d) as the only trainable parameters.
4️⃣ Delta-LoRA
In addition to training low-rank matrices, matrix W is also adjusted but not traditionally.
The difference (delta) between the product of A and B in two consecutive training steps is added to W.
5️⃣ LoRA+
In LoRA, both matrices A and B are updated with the same learning rate.
Authors found that setting a higher learning rate for matrix B results in more optimal convergence.
👉 Over to you: Which fine-tuning technique have you tried?
#ai #llm #finetuning
@dailydoseofds_
