What Thinking Buys

Multiply 47 by 86 in your head.

Most people can, slowly, and most people lose a number along the way, because the partial results keep slipping out of memory. On paper it's easy. You write 47 × 6 = 282, then 47 × 80 = 3,760, then add. Each line is a result you no longer have to hold.

That's most of what a "reasoning model" does when it thinks, and it explains both why thinking works and where it stops working. Thinking converts extra computation at answer time into accuracy on problems that break into steps the model can check: multi-step math, debugging against an error message, a plan that must satisfy a list of constraints. It buys little when the answer is a fact the model doesn't have, information nobody supplied, or a matter of taste, and past some point it can make answers worse. Every current reasoning model ships with a dial that controls how much it thinks. Knowing which kind of question you're asking is how you set it.

What happens when a model thinks

A reasoning model writes tokens, chunks of text a bit shorter than a word, before it writes its answer. They come out of the same network, one at a time, by the same next-word prediction as everything else; there's no separate reasoning engine underneath. The model has been trained to write a long working draft first, and the product usually hides the draft from you.

Why would writing more help? Each token costs one pass through the network, and each pass does a roughly fixed amount of computation, a fixed number of layers from input to output. A model forced to answer immediately has only the passes that write the answer to do all the work. A model that first writes intermediate results gets thousands of extra passes, and each later pass can read what the earlier ones wrote, because the draft sits in its context window, its working memory, like any other text. Researchers showed in 2024 that, under standard theoretical assumptions, a network of fixed depth answering directly can't carry out long chains of strictly sequential steps, and each written step lets it do one more. The draft is the paper.

The analogy breaks in two places worth knowing. The model can't erase: a wrong line stays on the page and keeps shaping what follows. And what it writes isn't guaranteed to be the route it actually took to the answer.

A second picture covers what arithmetic misses. A chess player tries a line, sees a refutation, abandons it and tries another. Reasoning models do something similar in text. DeepSeek's R1-Zero, trained only with rewards for correct answers, started writing things like "Wait, wait. Wait." and re-examining its first approach, without ever being shown that behavior.

Both pictures share one structure, and it predicts almost everything else in this piece: spending time improves the result only when each step can be tested against something. The rules of arithmetic, the rules of chess, a failing test, a stated constraint.

Thinking pays when each step can be checked against something. Where nothing can be checked, extra steps are just extra text.

The habit comes from training. Models are given problems with checkable answers and rewarded only when the checker agrees, a method called reinforcement learning with verifiable rewards, so whatever kind of working-out tends to precede correct answers gets reinforced (What Training Actually Changes). Over such training, R1-Zero's pass rate on a hard math competition rose from 15.6 to 71.0 percent while its thinking grew, unprompted, from hundreds to thousands of tokens per problem.

Two product details matter. Thinking is now adaptive: the model decides per request whether to think and how much, and the effort setting shifts its willingness to think without guaranteeing any count. And you pay for the whole draft while seeing a summary at most. Both Anthropic and OpenAI bill thinking tokens as output, the expensive kind.

Where it pays and where it doesn't

Plot accuracy against thinking and the curve takes a different shape for each kind of question.

Scroll sideways to see the whole figure

Checkable multi-step work shows the textbook curve. Anthropic reported that Claude 3.7 Sonnet's accuracy on competition math improved with the logarithm of the thinking it was allowed. That shape has a direct consequence for budgets: each doubling buys about the same gain, so going from 1,000 to 2,000 thinking tokens buys roughly what going from 16,000 to 32,000 buys, at one sixteenth of the price.

There's a useful refinement. Extra answer-time computation helps most on problems the model sometimes gets right. On problems where a small model already had some success, research in 2024 found that extra computation let it beat a model 14 times larger at equal total computation. On problems it already solves, there's nothing to gain. On problems far beyond it, more drafts mostly produce more wrong drafts. Thinking amplifies an ability the model partly has.

Closed-book recall is less obvious than it looks. In one 2025 study of 14 reasoning models answering factual questions without search, switching thinking on did help some: GPT-5 went from 20.1 to 41.4 percent on a trivia benchmark, plausibly because a few hundred tokens let it surface related facts, cross-check them and decline when unsure. Beyond that, more thinking didn't reliably raise accuracy and often raised hallucinations. GPT-5 mini's hallucination rate rose by more than 15 points as its reasoning grew from about 300 to 3,300 tokens. Computation over a fixed model can't add a fact the weights don't contain; longer reasoning mainly made models more willing to attempt an answer.

Missing information is flatter still. In the same study, GPT-5 mini on a research benchmark without search spent nearly 20 times more reasoning for no gain in accuracy. Given a search tool, more reasoning did pay. When the answer depends on something outside both the context and the weights, the fix is to supply the fact or a way to get it (When a Tool Beats Another Thought). In a company the missing fact is usually mundane: the terms of this customer's contract, a decision taken in last week's meeting.

Writing and judgment have no clean measurement, so this part is reasoning from the mechanism. Parts of a writing task are checkable (a word limit, the points the brief requires, numbers that must match between paragraphs), and thinking should help there. Voice, emphasis and taste have no checker the model can consult mid-draft, so extra drafts have nothing to converge on.

Thinking amplifies what a model can partly do. It cannot supply a fact the model was never given.

When more thinking makes it worse

Three mechanisms make a longer draft a worse answer.

Overthinking easy problems. Asked "what is 2 plus 3?", reasoning models used on average 1,953 percent more tokens than conventional models to reach the same answer; one produced 13 separate solutions. Training a model to stop earlier cut its output on a standard math set by about 45 percent with accuracy unchanged. Here the cost is money and time.

Getting pulled off course. Researchers built tasks to find cases where accuracy falls as reasoning grows, and found them. One kind pads a trivial counting question with irrelevant math-puzzle sentences: given five such distractors and left to reason at its natural length, DeepSeek R1 fell from about 70 to 30 percent as its answers got longer. Another rewords a famous paradox until the answer is obvious: "In a room of n people, there's a 50.7% chance at least two share a birthday. Calculate how many rooms there are." The answer is one. These tasks were built to break models, so they show the effect exists, not how often it occurs in ordinary work. That's enough to stop assuming the longest setting is the best one.

Compounding an early error. The model can't erase, so an early wrong assumption becomes a premise that later steps elaborate, sometimes with invented supporting detail.

Anthropic says it plainly in its own documentation for Claude Opus 4.7: on most workloads, max effort "adds significant cost for relatively small quality gains", and on some tasks it can lead to overthinking.

What the dial costs

Artificial Analysis, an independent benchmarking firm, runs the same evaluations on each model at each effort level and publishes the score, the average cost per task and the response time.

Scroll sideways to see the whole figure

The shape repeats across models. On Claude Opus 5.5, going from low to max took the composite score from 42 to 58, multiplied the average cost per task about elevenfold and the response time about fortyfold. The first step, low to medium, added 9 points for 2.4 times the cost. The remaining three steps together added 7 points for another 4.5 times the cost, while average response time went from 28 seconds at medium to 667 at max. On Claude Fable 5.1, the top two settings scored the same, and the top one cost 28 percent more.

The first step up the effort dial usually buys the most. The last steps buy little and cost the most waiting.

One comparison deserves a second look. Claude Haiku 5.5, a small model, scored 43 at max, against 42 for Opus 5.5 at low, at about 40 percent of the cost per task. A small model thinking hard can match a large model thinking little, and more cheaply. It took 292 seconds to respond against 16. Effort and model size substitute for each other on money. On time they don't.

These are one tracker's averages over a mix of tests, so they blur the task differences above. What they support is the shape: steep early gains, a long flat top, and waiting that explodes at the highest settings.

Bring it down to one request. Take a debugging question to Opus 5.5 at its list price: 6,000 tokens of code, error output and instructions in, an 800-token answer out. Assume 1,500 thinking tokens at low, 12,000 at high and 32,000 at max, roughly in line with the measured ratios. The answer costs 7, 28 or 68 cents, and arrives in about half a minute, 2.8 minutes or 5.7 minutes. At max, 94 percent of the bill is thinking nobody reads.

Is high worth 21 extra cents? If it answers this kind of question correctly five points more often than low (an assumption your own tests should replace), it pays for itself whenever a wrong answer costs more than $4.20, which for a bug that wastes half an hour of an engineer's time it plainly does. High to max is a different trade. It adds 40 cents in tokens and about three minutes of waiting. An engineer sitting idle at a loaded cost of $100 an hour burns about $4.90 in those three minutes, twelve times the token difference. For interactive work, the clock often costs more than the tokens. For work nobody waits on, the clock is nearly free.

Why not just set everything to max?

It's the strongest objection, and parts of it are right. Tokens are cheap and getting cheaper, and a wrong answer costs far more than a dollar. For rare, high-stakes, checkable work that nobody waits on (a proof check that runs overnight, the root cause of a production incident, contract clauses compared against a list of requirements), max is a reasonable choice, and its premium is noise next to a missed error.

As a default, the numbers argue against it. Waiting grows faster than cost. Volume turns cents into budgets: an assistant handling 100,000 such questions a day would spend about $40,000 a day more at max than at high in the example above. Agents multiply the setting across the steps of a loop. Some tasks get worse. And the top steps are small. Anthropic's own choice is telling: Opus 5.5 ships with medium as its default, one level below what earlier Opus models used.

One more caution, about reading the draft. In a 2025 study, Anthropic researchers slipped hints into questions and checked whether models that used a hint said so. Claude 3.7 Sonnet mentioned it about 25 percent of the time, DeepSeek R1 about 39 percent. The thinking summary is still the fastest way to find which assumption the model made. Treat it as testimony, which can be wrong and can leave things out (Do Models Mean What Their Reasoning Says?).

What to do

My view is that effort is a budget decision most teams are making by accident, through a default in code. Make it on purpose.

Fix the cheaper problems first. If the task is a lookup, give the model search or the source document. If it's a calculation, give it a code tool. If the answer depends on context only you have, supply it. Each changes what the model can know, which no effort level does.

Start at the default or lower for interactive work, and raise effort one level at a time for checkable multi-step tasks. OpenAI reserves high for hard reasoning, complex debugging and deep planning; Anthropic suggests medium or low for chat.

Sweep on your own cases. Run 30 to 50 representative tasks at two or three levels, several times each, and record accuracy, thinking tokens and response time, including the slow tail. Keep the lowest level your set can't distinguish from the next one up (Build an Evaluation From Your Own Work).

Route by task. A classifier at low, a planning step at high. On Anthropic's API, changing the top-level effort between requests breaks the prompt cache, so keep long conversations at one level.

Leave room for the draft. Thinking counts toward the maximum output length; OpenAI recommends reserving at least 25,000 tokens for reasoning and output when starting out. A response cut off mid-thought is pure cost.

Re-run the sweep when the model changes. Anthropic recalibrated the levels on Sonnet 5.5, so the same setting no longer produces the same amount of thinking. A setting tuned on last quarter's model is a guess on this quarter's.

For managers, the answer comes per kind of work. A team that mostly drafts, summarizes and looks things up will see the bill rise and little else change. A team that debugs, analyzes against constraints and plans should see real accuracy gains, and should mostly worry about people waiting on the slowest settings. Most teams have both, so one company-wide setting is usually the wrong unit of decision, and thinking deserves its own line in cost monitoring (Where the AI Money Goes).

Level names, defaults and prices will change with the next release. The relation underneath should hold, because it follows from what thinking is: more sequential computation over information already present. Where steps can be checked, that computation converges on better answers. Where the information is absent or the standard is taste, there's nothing to converge on, and the extra tokens become cost, delay and sometimes a more confident mistake.

Check yourself

Why does writing a draft first help a model on multi-step problems?

Each token is one pass through the network with a fixed amount of computation. The draft gives thousands of extra passes, and each can read the earlier results, like partial sums on paper.

Why won't more thinking fix a question about a fact the model does not have?

Computation over a fixed model cannot add a fact the weights lack. Longer reasoning mostly made models more willing to attempt an answer, so the fix is to supply the fact or a way to look it up.

Why is the maximum setting a poor default for interactive work?

The score gains flatten at the top while cost and waiting keep climbing. A person waiting often costs more than the tokens.