大模型的学习笔记：从数学基础、Transformer 架构，到训练对齐与推理工程。每个条目按「核心答案 → 深入原理 → 权衡追问 → 参考」组织，⭐ 标记值得重点深挖的地方。 Notes on large language models: from the maths underneath, through the Transformer, to training, alignment and inference engineering. Each entry runs core answer → how it works → trade-offs → references; ⭐ marks the points worth digging into.
基础架构 · 训练对齐 · 推理优化Architecture · Training & Alignment · Inference
交叉熵与 KL 散度 · Transformer 与 self-attention · RoPE · KV Cache 与 MQA/GQA · MoE · RLHF 与 DPO · LoRA/QLoRA · 量化 · Flash Attention · 推测解码 · Prefill 与 Decode · Scaling Law Cross-entropy and KL divergence · Transformer and self-attention · RoPE · KV cache, MQA/GQA · MoE · RLHF and DPO · LoRA/QLoRA · Quantization · FlashAttention · Speculative decoding · Prefill vs. decode · Scaling laws
通读全文 →Read it →