The DeepSeek Efficiency Moment

Mixture of Experts gave us the central bargain of large-model efficiency: scale total parameters for knowledge, but activate only a slice per token so compute stays cheap. For most of this book that idea is a technique. In late 2024 and early 2025 it became a news event — the clearest public demonstration that efficiency is not a cost-saving footnote but a competitive strategy. This page records what happened, carefully and with dates, because the specifics matter and because much of the surrounding coverage was breathless. Throughout, treat reported figures as reported/claimed, not as audited facts.

Dates and figures are as reported

Everything below is stated as it was publicly reported around the time. Independent verification of training costs, hardware, and benchmark claims is hard, and some figures were contested. Where a number appears, read an implicit “reported” in front of it.

DeepSeek-V3 (December 2024)

In December 2024, the Chinese lab DeepSeek released DeepSeek-V3, a large open-weights Mixture-of-Experts model. The widely reported shape:

  Total parameters   ≈ 671B   (the full MoE checkpoint)
  Active per token   ≈ 37B    (only the routed experts + shared layers run)

This is exactly the MoE distinction from earlier in the book, at frontier scale: roughly 671B of knowledge, ~37B of compute per token. The memory bill tracks the ~671B total (all experts must stay resident); the per-token latency and compute track the ~37B active. That decoupling is what lets an MoE this large be served at a per-token cost closer to a mid-sized dense model.

Two further efficiency claims drew attention:

• FP8 training. DeepSeek reported training V3 largely in FP8 mixed precision. As the numeric-precision page covered, FP8 became a hardware-native format on NVIDIA’s Hopper generation — so training in 8-bit isn’t a stunt, it’s using the silicon’s fastest math units. Doing it stably at this scale is, however, serious engineering.
• A low reported training cost. The accompanying report described a total training compute bill that was a small fraction of what comparable frontier runs were assumed to cost. This figure became the headline — and the most contested number, since it reflected a specific accounting of one run and excluded much of the surrounding research, data, and infrastructure cost.

The signal, stripped of the disputed dollar figure: a competitive frontier-class model, trained and served efficiently through MoE sparsity, low precision, and hard systems work — and released with open weights.

DeepSeek-R1 (January 2025)

In January 2025, DeepSeek followed with DeepSeek-R1, an open-weights reasoning model — one trained to spend extra tokens “thinking” before answering, the test-time-compute idea covered in reasoning and test-time compute. R1 was notable on two fronts:

1. Open-weights reasoning. Strong reasoning behavior had mostly been the province of closed frontier models. R1 put a capable reasoning model into the open, with reported methodology around reinforcement learning for reasoning.
2. Distilled variants. DeepSeek also released smaller models distilled from R1’s reasoning traces — the distillation lever again: bottle a large model’s behavior into sizes others can actually run.

The combination — an efficient large MoE base, plus an open reasoning model, plus distilled small versions — landed as a single coherent statement about how to compete: not by out-spending on compute, but by out-engineering on efficiency.

The market reaction (late January 2025)

The technical releases became a financial story in the last week of January 2025. As DeepSeek’s app and models surged in visibility, equity markets reacted sharply on the thesis that if frontier-grade capability could be produced and served this cheaply, the world might need less compute than it had been buying.

• The reaction was widely reported as one of the largest single-day market-value losses for a US company in history, concentrated in AI-chip and AI-infrastructure stocks.
• The move was driven by narrative, not by any audited cost figure — markets repriced an assumption (“frontier quality requires frontier-scale spend”) faster than anyone could verify the inputs.

What the sell-off did and didn’t prove

A one-day market move proves a change in belief, not a change in physics. The lasting lesson isn’t the stock chart — it’s that a credible demonstration of radical efficiency can reprice an entire industry’s assumptions overnight. Whether cheaper inference reduces total compute demand or increases it (cheaper tokens → far more token usage — the Jevons-paradox argument many made in response) is genuinely unsettled. This page reports the reaction; it does not endorse either forecast.

What it signaled

Set aside the contested numbers and the stock move, and a durable thesis remains — the same one the efficiency-mindset page argues from first principles:

  Old assumption:  frontier quality  ⟸  frontier-scale spend
  The signal:      frontier quality  ⟸  aggressive efficiency
                    (MoE sparsity + low precision + systems engineering
                     + open weights)

Efficiency, in other words, is product strategy. A team that reaches comparable quality at a fraction of the cost doesn’t merely save money — it changes who can compete, what can be open-sourced, and how much compute the market believes it needs. Every lever in this book — MoE, FP8, distillation, careful serving — showed up in this one episode, which is why it’s worth dating precisely: it’s the moment the book’s recurring question (how do we make it cheaper?) stopped being an engineering concern and became a strategic and even macroeconomic one.

Under the hood — why ~671B total but only ~37B active

The arithmetic is the MoE arithmetic from earlier in the book. An MoE layer replaces the dense FFN with many expert FFNs plus a router; for each token the router activates only the top-k experts. Attention and embedding layers are shared, not replicated. So “total” counts every expert across every layer (the full ~671B that must sit in memory), while “active” counts the shared layers plus just the handful of experts a given token is routed through (~37B of actual compute). The model has the parameter count of something enormous and the per-token FLOPs of something far smaller — the exact decoupling that makes a model this large economically serveable. It is also why such a model is hungry on memory: cheap to compute per token, expensive to keep resident.

Check your understanding

1. What are DeepSeek-V3’s reported total and active parameter counts, and which one drives memory cost versus per-token compute cost?
2. Why is “trained largely in FP8” an efficiency claim rather than a gimmick, and what makes doing it at scale hard?
3. Which reported figure about V3 was the most contested, and why should you treat it cautiously?
4. What three things did the R1 release combine, and what single strategic statement did the combination make?
5. The late-January-2025 market reaction proved a change in belief, not in physics. What durable thesis survives once you set aside the disputed numbers and the stock move?

Show answers

1. Reported ~671B total and ~37B active per token. The ~671B total drives memory cost (all experts must stay resident), while the ~37B active drives per-token compute and latency — the standard MoE decoupling at frontier scale.
2. FP8 is a hardware-native format on NVIDIA’s Hopper generation, so 8-bit training uses the chip’s fastest math units — a real efficiency lever, not emulation. It’s hard because low precision can cause overflow/instability during training, so doing it stably across a model this large takes serious systems and numerics engineering.
3. The reported total training-compute cost, which was described as a small fraction of comparable frontier runs. It’s a specific accounting of one run that excluded much surrounding research, data, and infrastructure cost, and was disputed — so it should be read as “reported,” not audited.
4. An efficient large open-weights MoE base (V3), an open-weights reasoning model (R1), and smaller models distilled from R1’s reasoning. Together they stated: you compete by out-engineering on efficiency and releasing openly, not by out-spending on raw compute.
5. That frontier-grade quality can come from aggressive efficiency — MoE sparsity, low-precision (FP8) training, distillation, and strong systems engineering — rather than only from frontier-scale spending. Efficiency is product (and even macroeconomic) strategy, not a cost-saving footnote.