Fine-tuning is the process of adapting a computational model trained for one task to perform a different, usually more specific, task. It is a form of transfer learning that reuses knowledge learned from the original training objective. Fine-tuning involves applying additional training to the parameters of a neural network that have been pre-trained. Many variants exist. The additional training can be applied to the entire neural network, or to only a subset of its layers, in which case the layers that are not being fine-tuned are "frozen." A model may also be augmented with "adapters", lightweight modules inserted into the model's architecture that nudge the embedding space for domain adaptation. These contain far fewer parameters than the original model.
Common PEFT Techniques
Parameter-Efficient Fine-Tuning (PEFT) is a family of techniques for adapting a large pretrained model to a new task by training only a small subset of parameters, or a small set of added parameters, instead of updating the entire model. It delivers most of the benefit of full fine-tuning at a fraction of the compute, memory, and storage cost, which is why it's become the default way most teams customize large language models.
- LoRA: Freezes the original model weights and trains small low-rank matrices injected into each layer.
- QLoRA: Combines LoRA with a quantized base model to cut memory requirements further.
- Prompt/Prefix Tuning: Learns a small set of continuous "virtual tokens" prepended to the input instead of touching the model's weights at all.
- Adapters: Inserts small trainable layers between the frozen layers of the base model.
Benefits of PEFT
- Lower Cost: Training a fraction of a model's parameters requires far less GPU memory and compute than full fine-tuning.
- Faster Iteration: Smaller trainable footprints mean quicker experiments and faster time-to-deployment for domain-specific models.
- Multiple Task Adapters: Because the base model stays frozen, teams can swap lightweight PEFT adapters in and out to serve many use cases from a single underlying model.
