AI Systems, From First Principles

A first-principles guide to how AI systems run, and how to make them efficient. We build understanding in dependency order — each topic only uses ideas already covered. For every technique, keep asking one question: what does this cost — in latency, memory, and dollars — and how does it make that cheaper? That single thread runs through the whole book.

The spine is inference and serving: how a token is actually produced, why it costs what it costs, and the techniques that cut the bill — KV caches, batching, quantization, and the serving stacks that tie them together. The capstone then turns the lens around: you build a real app on the Claude API and cut its cost with the levers you control from outside the model. Read in order, and after each page answer the Check your understanding questions in your own words. Depth over speed.

Part 0 · Foundations — Why Efficiency Matters

Part 1 · The Hardware Reality

Part 2 · The Transformer, Mechanically

Part 3 · Inference Efficiency Core

Part 4 · Model Compression

Part 5 · Serving Systems

Part 6 · Application-Layer Efficiency

Part 7 · Training & Fine-Tuning Efficiency

Part 8 · Measuring & Operating (LLMOps)

Part 9 · Advanced & Rare Concepts

The senior-level track — the frontier techniques and failure modes that show up once you operate AI systems at scale.

Part 10 · Hands-On Capstone (Claude API)

The capstone — build a real RAG/agent app on the Claude API, measure its cost and latency, then cut both with the levers you control without owning a GPU: caching, model routing, context trimming, and streaming.

Part 11 · Frontier

The efficiency frontier as of 2024–2025 — where the unit of cost stops being one call: agentic systems, reasoning/test-time compute, multimodal & diffusion, embeddings & vector DBs, small language models, the DeepSeek moment, and the 2025 serving frontier.