Preference fine-tuning
Search documents
X @Avi Chawla
Avi Chawla· 2026-07-31 08:43
4 stages of LLM training, clearly explained:(a popular LLM interview question; bookmark this)→ Pre-training→ Instruction fine-tuning→ Preference fine-tuning→ Reasoning fine-tuningThe visual covers all of them.A base model treats a question as text to be continued, not a request to be answered. Questions in its training data usually have more questions next to them, so that's what the model replies.The stages after pre-training allow changing that behaviour, and each one uses a different training signal to d ...