The Cost of Agentic Systems

Prompt & context economy showed that every token in a single prompt is re-billed on every call. An agent takes that fact and multiplies it. Instead of one request, an agent runs a loop: think, call a tool, read the result, think again, call another tool — often ten or twenty times to finish one task. And on each pass it re-sends the entire conversation so far. The single most important thing to understand about agents is that their cost does not grow with the task; it grows with the square of the number of steps. This page is about that multiplication, and how to fight it.

What an agent actually does

The dominant pattern is ReAct — Reason + Act — where the model interleaves natural-language reasoning with tool calls, introduced by Yao and colleagues in 2022. One loop iteration looks like this:

   ┌───────────────────────────────────────────────┐
   │  context = system + task + ALL prior steps     │
   └───────────────────────────────────────────────┘
                        │
                        ▼
              model "thinks" (reasoning)
                        │
                        ▼
              model emits a tool call ──▶ tool runs ──▶ observation
                        │                                   │
                        └────────── append observation ─────┘
                                     loop again

The catch is in the first box. On step 7, the context is not just the task — it is the system prompt, the task, and the full transcript of steps 1 through 6: every thought, every tool call, every tool result. Tool results are often the largest part: a web page, a file, a database response, a stack trace. The agent re-reads its whole history to decide its next move.

The agent tax: why cost grows quadratically

Each step re-sends a context that keeps growing. If every step adds roughly a fixed chunk of tokens (a thought plus an observation), then step n sends about n chunks, and the total input tokens billed across the whole run is the sum 1 + 2 + 3 + ... + N — which scales with N², not N.

By the numbers — the quadratic agent tax

Take a deliberately simple model of one agent run. Say the fixed preamble (system + task) is 1,000 tokens, and each step appends about 500 tokens of new thinking and tool output that all future steps must re-read.

Input tokens billed at step n ≈ 1,000 + 500 × (n − 1).

Summed over a run, with the per-step term being the triangular sum 500 × (0 + 1 + ... + (N−1)) = 500 × N(N−1)/2:

N = 5  steps :  5×1,000 + 500 × (5×4/2)  =  5,000 + 5,000   =  10,000 input tokens
N = 10 steps : 10×1,000 + 500 × (10×9/2) = 10,000 + 22,500  =  32,500 input tokens
N = 20 steps : 20×1,000 + 500 × (20×19/2)= 20,000 + 95,000  = 115,000 input tokens

Doubling the step count from 10 to 20 more than tripled the input bill (32,500 → 115,000). That super-linear growth — input tokens climbing faster than the number of steps — is the agent tax. The work the agent did barely doubled; the tokens you paid for grew with the square. (Output tokens, which carry the ~5× premium, grow only linearly with steps — but the re-sent input is what dominates a long agent run.)

Latency compounds the same way

Tokens are only half the bill. The steps are sequential — step 7 cannot start until step 6’s tool returns — so latency is the sum of every step’s round trip. And because each step’s prompt is longer than the last, each step also prefills more input before it can respond. A 20-step agent feels slow not because any one call is slow, but because you wait for twenty calls in a row, each one heavier than the last. The user experiences the agent tax as a spinner.

It actually happened — ReAct (Yao et al., 2022) and the agent wave

The loop this page describes was formalized in ReAct: Synergizing Reasoning and Acting in Language Models (Shunyu Yao and colleagues, 2022), which showed that interleaving chain-of-thought reasoning with tool actions made models markedly better at multi-step tasks. It became the template for the 2023–2025 wave of agent frameworks and coding agents. But ReAct’s design assumption — re-feed the whole growing trace on every step — is exactly what makes the pattern expensive at scale. The capability and the cost came from the same design choice: the agent’s power is that it remembers everything it has done, and its tax is that it re-reads and re-pays for everything it has done, every step.

Clawing the tax back

The fixes all attack the same thing: stop re-sending and re-paying for context that isn’t earning its place.

Prompt caching across turns

Because each step’s prompt is the previous step’s prompt plus a suffix, the long stable prefix (system prompt, task, early steps) is identical call after call. Prefix / prompt caching lets the provider reuse the already-processed prefix instead of prefilling it from scratch each step — turning the most-repeated, most-expensive part of the agent loop into a cache hit. For agents, where the prefix is re-sent on every iteration, this is the single highest-leverage lever: it directly cancels the largest term in the quadratic.

Context compaction

Don’t carry the full verbatim history forever. Periodically compact it — summarize completed sub-tasks, drop tool outputs you’ve already extracted the answer from, collapse a ten-message detour into one line of “we tried X, it failed because Y.” This caps the per-step context size so the sum stops growing quadratically and flattens toward linear. The discipline is the same as trimming a multi-turn chat: a token that has stopped changing future decisions is pure recurring cost.

Cheaper sub-agents

Not every step of an agent needs the frontier model. A common pattern is a capable orchestrator that plans, delegating narrow sub-tasks (search this, parse that, extract these fields) to a cheap, fast model — the same routing and cascade logic, applied inside one task. A sub-agent also gets its own fresh, small context window, so its work doesn’t bloat the orchestrator’s transcript. You pay frontier prices only for the reasoning that needs it.

Parallel tool calls

When the next several actions don’t depend on each other — fetch three files, query two APIs — issuing them in parallel instead of one per loop iteration collapses several sequential round trips into one. This doesn’t cut token cost, but it directly attacks the latency half of the tax: fewer sequential steps means a shorter wall-clock wait.

Lever	Cuts	How
Prompt caching across turns	input $ + prefill latency	reuse the stable, re-sent prefix
Context compaction	input $ (caps the quadratic)	summarize/drop stale history
Cheaper sub-agents	input + output $	frontier model only where needed
Parallel tool calls	latency	collapse independent steps

The discipline

Before adding a step, a tool, or a verbose tool output to an agent, remember it isn’t paid once — it is re-sent on every remaining step of the run. The throughline question for agents is sharper than for a single call: what does this loop cost in latency, memory, and dollars once you count every re-send — and the answer is almost always “more than you think, growing with the square of the steps.” Cache the prefix, compact the history, delegate the cheap parts, and parallelize what you can.

The architect’s lens

An agent multiplies the prompt-economy problem — adopt the loop knowing its tax:

• Why does it exist? Because some tasks need many think-act-observe steps (ReAct), and the agent’s power is that it remembers everything it has done — re-feeding the whole growing trace on every step.
• What problem does it solve? Multi-step tool use a single call can’t do — but its cost grows with the square of the steps: re-sending an N-step trace bills the triangular sum, so going 10→20 steps more than triples input tokens (32,500→115,000).
• What are the trade-offs? The same design choice that gives capability gives the tax — quadratic input tokens and sequential, ever-heavier prefills the user feels as a spinner; the levers (prefix caching across turns, context compaction, cheap sub-agents, parallel tool calls) claw it back.
• When should I avoid it? When a single call or a fixed pipeline solves the task — don’t pay the agent tax for work that isn’t genuinely iterative and tool-driven.
• What breaks if I remove it? You lose multi-step autonomous tool use; keep the loop but skip the levers and the re-sent prefix — the largest cost term — bills at full price on every iteration.

Check your understanding

1. Why does the total input-token cost of an agent run grow roughly with the square of the number of steps rather than linearly?
2. In the by-the-numbers example, why did going from 10 steps to 20 steps more than triple the input bill?
3. Why does prompt caching across turns help agents so much more than it helps a one-shot call?
4. What does context compaction change about the cost curve, and what is the risk if you compact too aggressively?
5. Parallel tool calls don’t reduce token count — so what part of the agent tax do they attack, and why does that work?

Show answers

1. Each step re-sends the entire growing history, so step n bills about n chunks of context; summing 1 + 2 + ... + N scales with N², making total input tokens grow with the square of the step count.
2. The dominant term is the triangular sum of re-sent context, 500 × N(N−1)/2; that term went from 22,500 tokens at N=10 to 95,000 at N=20 — about 4× — so the total jumped from 32,500 to 115,000, more than tripling.
3. An agent re-sends a long stable prefix (system prompt, task, early steps) on every iteration, so caching the already-processed prefix cancels the most-repeated and largest cost term; a one-shot call sends its prefix only once, so there’s little to reuse.
4. It caps the per-step context size by summarizing or dropping stale history, flattening the cost curve from quadratic toward linear; the risk is discarding a detail the agent later needs, degrading the result to save tokens.
5. They attack the latency half of the tax: independent steps that would otherwise run one-per-loop are issued at once, collapsing several sequential round trips into one and shortening the wall-clock wait, even though the same tokens are still billed.