Part 5: Preference Alignment with DPO (Direct Preference Optimization) & GRPO
← Previous Chapter: Part 4: Knowledge Distillation | Series Hub | Next Chapter: Part 6: vLLM Deployment & Automated Evals → Prerequisite: Read Part 4: Knowledge Distillation from DeepSeek-R1 & Frontier Teachers for Chain-of-Thought reasoning distillation. Answer-first: Direct Preference Optimization (DPO) and Group Relative Policy Optimization (GRPO) supersede unstable 4-model PPO pipelines for SLM alignment. By deriving implicit rewards directly from reference model log-probabilities or computing group-relative advantages without Critic networks, developers enforce 99.8% JSON schema compliance and eliminate hallucinations on single 24GB GPUs with zero reinforcement learning instability. ...