A process of further training a pretrained model, typically a large language model, on a curated set of labeled input-output examples so it learns to follow instructions or perform a specific task. It is usually the first tuning stage after pretraining and lays the groundwork for later alignment steps like Reinforcement Learning from Human Feedback (RLHF) or Direct Preference Optimization (DPO).
Where SFT Fits in the LLM Training Pipeline
The LLM training pipeline includes three stages:
- Pretraining: The model learns general language patterns from massive unlabeled text.
- Supervised Fine-Tuning: The model trains on curated (prompt, ideal response) pairs, teaching it to follow instructions rather than just predict the next token.
- Preference Alignment: Techniques like RLHF, Reinforcement Learning from AI Feedback (RLAIF), or DPO further tune the model using human or AI preference signals.
Task Specialization
Businesses use Supervised Fine-Tuning (SFT) to adapt a general-purpose foundation model to a specific voice, domain, or workflow, such as customer support scripts or legal drafting.
Full-Weight vs. Efficient Tuning
Classic SFT updates all of a model's weights; Parameter-Efficient Fine-Tuning (PEFT) methods like Low-Rank Adaptation (LoRA) achieve similar results by updating far fewer parameters.
