PyTorch

Answer-first: For 80% of domain enterprise tasks, fine-tuned Small Language Models (1B–14B) match frontier performance at 1/50th inference cost and sub-40ms latency. This playbook provides the production engineering blueprint: synthetic data curation, QLoRA fine-tuning with Axolotl on 24GB GPUs, DeepSeek-R1 reasoning distillation, DPO/GRPO alignment, and vLLM continuous batching deployment. 🇻🇳 Read the Vietnamese version of this series on learn.tanhdev.com 🎯 Series Overview: Why Small Language Models in 2026? Relying exclusively on proprietary frontier API models (GPT-4.5, Claude 3.5 Sonnet) introduces three fatal enterprise vulnerabilities: ...

← Previous Chapter: Part 2: SFT Data Engineering | Series Hub | Next Chapter: Part 4: Knowledge Distillation → Prerequisite: Read Part 2: SFT Data Engineering — NEFTune & Synthetic Data Curation for instruction dataset curation and decontamination. Answer-first: QLoRA compresses base model weights into 4-bit NormalFloat (NF4) representations while computing gradients exclusively through 16-bit adapter matrices. Combining Double Quantization with CUDA Paged Optimizers enables fine-tuning 14B models on a single 24GB commodity GPU (RTX 4090 or L4) at $1.20/hour, preserving 99.3% full-precision benchmark performance while preventing out-of-memory crashes. ...

← Previous Chapter: Part 3: QLoRA & Axolotl Fine-Tuning | Series Hub | Next Chapter: Part 5: Preference Alignment: DPO, GRPO & KTO → Prerequisite: Read Part 3: QLoRA & Axolotl Fine-Tuning on Commodity GPUs for low-rank parameter tuning and memory budgeting. Answer-first: Distilling long Chain-of-Thought (CoT) reasoning traces from DeepSeek-R1 (671B MoE) into compact 1.5B–8B student models transfers complex deductive capability without hosting frontier hardware. Combining forward-backward token KL divergence with rejection sampling on verifiable tasks enables a 7B student model to recover 88% of teacher mathematical reasoning performance at 1/50th the operational cost. ...

← Previous Chapter: Part 4: Knowledge Distillation | Series Hub | Next Chapter: Part 6: vLLM Deployment & Automated Evals → Prerequisite: Read Part 4: Knowledge Distillation from DeepSeek-R1 & Frontier Teachers for Chain-of-Thought reasoning distillation. Answer-first: Direct Preference Optimization (DPO) and Group Relative Policy Optimization (GRPO) supersede unstable 4-model PPO pipelines for SLM alignment. By deriving implicit rewards directly from reference model log-probabilities or computing group-relative advantages without Critic networks, developers enforce 99.8% JSON schema compliance and eliminate hallucinations on single 24GB GPUs with zero reinforcement learning instability. ...