Supervised Fine-Tuning
Supervised Fine-Tuning (SFT) is about Fine-Tuning a pre-trained model on labelled demonstration data. Really just standard supervised learning, continuing from the pre-trained weights (Ouyang et al., 2022).
In the LLM Training Pipeline, this can teach the model to respond to user queries using examples of instructions and their desired responses. This form of SFT is called instruction tuning. SFT is the broader term, since supervised fine-tuning can also target other tasks (Raschka, 2026).
Preference tuning, such as RLHF or DPO, can follow SFT. Instead of only learning to imitate demonstrations, it uses feedback about which responses are preferred (Ouyang et al., 2022) (Rafailov et al., 2023).
References
Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike, and Ryan Lowe. Training language models to follow instructions with human feedback. March 2022. arXiv:2203.02155, doi:10.48550/arXiv.2203.02155. ↩ 1 2
Rafael Rafailov, Archit Sharma, Eric Mitchell, Stefano Ermon, Christopher D. Manning, and Chelsea Finn. Direct Preference Optimization: Your Language Model is Secretly a Reward Model. 2023. doi:10.48550/ARXIV.2305.18290. ↩
Sebastian Raschka. Build a Reasoning Model. Manning Publications, Erscheinungsort nicht ermittelbar, 2026. ISBN 978-1-63343-467-7. ↩