Why AI Products Have Thinner Margins

If one Salesforce customer uses the product ten times as much as another, Salesforce barely notices. The code was written once. A click runs a few milliseconds of computing. The heavy customer costs a little more to serve, and the price covers it many times over.

If one customer of an AI assistant uses it ten times as much as another, the company notices on its next invoice from the model provider.

That difference is the whole story of AI margins, and it's easy to state: an AI product pays for the model on every request, so its margin falls in a straight line as usage rises. On a flat price, the heaviest users decide whether the product makes money, success makes it worse, and falling token prices only help if the work doesn't grow faster than the price falls. Most of the confusion about whether AI businesses are "good businesses" comes from looking at the average user and the price list, when the answer sits in the tail and in what customers do with cheaper tokens.

What gross margin measures

Gross margin is the share of each revenue dollar left after paying the direct cost of delivering the product: hosting, the support people who keep customers running, payment fees and, in an AI product, the model calls made on the customer's behalf. Accountants call that direct cost the cost of revenue. Research, sales and overhead sit below the line and don't count.

Classic software keeps most of the dollar. In its fiscal year ending January 2026, Salesforce's cost of revenue was 22 percent of $41.5 billion, a gross margin of about 78 percent. AI products usually keep less. Bessemer Venture Partners put its vertical AI portfolio companies at about 65 percent in 2024, and the ten fastest-growing AI startups it surveyed in 2025 at about 25 percent, often below zero.

Software has seen low margins before. Twilio sat around 50 percent for years because it paid phone carriers for every message it sent. What's new is that in an AI product the per-use cost is the product itself, and it swings with each customer's behavior far more than carrier fees ever did.

Where the cost comes from

A language model reads and writes in tokens, chunks of text a bit shorter than a word. Every request runs a large model over the whole prompt and then writes the answer one token at a time, each step a slice of computation on an expensive chip. Providers charge per million tokens, and they charge far more for writing than for reading: on Claude Opus 5.5, $4 per million tokens in and $20 per million out.

Take one ordinary task: the model reads 12,000 tokens (8,000 of standing instructions and documents, 4,000 new) and writes 2,500 (1,500 of hidden reasoning, 1,000 of answer). That costs about ten cents. Output is only 17 percent of the tokens but more than half the bill, because of that fivefold price.

Ten cents sounds like nothing. Now put a flat price on it.

One $30 seat, three kinds of user

Imagine an assistant sold at $30 per seat per month. Everything other than the model (hosting, support, payment fees) costs $5 per seat, so a seat nobody used would earn 83 percent, right in classic software territory. Users come in three kinds:

• Light, 45 percent of seats, 25 tasks a month.
• Typical, 45 percent, 100 tasks.
• Heavy, 10 percent, 800 tasks, about 38 every working day.

Scroll sideways to see the whole figure

The light user costs $2.45 a month in model calls and earns a 75 percent margin. The typical user costs $9.80 and earns 51 percent. The heavy user costs $78.40, more than twice the price of the seat, and earns minus 178 percent.

It's 39 percent.

The reason is simple once you see it. Each extra task a month costs the same ten cents, so margin is a straight line in usage: here, about a third of a margin point per task. A straight line has a useful property. The margin of the whole customer base equals the margin of a user at the average usage, and averages are pulled by the tail. The heavy tenth uses eight times what a typical user does, so it drags the average up to 136 tasks, and it accounts for 59 percent of all model spending while paying 10 percent of the revenue.

On a flat price, margin is a straight line in usage. The average user sets it, and the heaviest tenth sets the average.

Anyone who judges this product by its typical user overestimates the margin by about 12 points. And this example's skew is mild. Ramp, which sees what its business customers spend on AI, reported that in April 2026 the mean company spent 63 times what the median company did.

Success makes it worse

Here is the part that should worry anyone running an AI product on seat pricing.

Suppose the rollout works. Customers find more uses, and the mix shifts to 20 percent light, 55 percent typical and 25 percent heavy. No individual changes behavior; there are simply more people in the heavy group. Model cost per seat nearly doubles, from $13.35 to $25.48, and the blended margin falls from 39 percent to about minus 2.

On a flat price, adoption is a cost. Your customer success team is paid to make customers use the product more, and every time they succeed, your finance team loses money. That conflict is built into the pricing, and no amount of engineering removes it.

What recovers the margin

The levers come in two kinds. Some lower what a task costs. One changes who pays for the tasks that remain.

Route each task to the smallest model that can do it. In this example every task runs on the most expensive model, which is what many products do by default. A task on Claude Haiku 5.5, a model built for classification, extraction and routing, costs one fortieth of the same task on Opus 5.5; Sonnet 5.5 costs half. Send half the tasks to Haiku and split the rest by difficulty, and the average task costs 59 percent less before anything else changes. That's the biggest single gain, and the riskiest. A task sent to a weaker model that gets it wrong costs a rerun and some of the customer's trust, and only the rerun shows up in the margin arithmetic (Routing Work Between Models).

Cache what repeats. Most of a prompt is the same every time: instructions, policies, reference documents. With prompt caching, the provider keeps the processed form of that repeated beginning for a few minutes, and reading it again costs 5 percent of the normal input price. The catch is timing. A heavy user sending requests minutes apart hits the cache nearly every time. A light user who comes back hours later never does, and paying to write a cache nobody reads costs more than not caching. So caching helps precisely where the cost is concentrated.

Think and write less. Reasoning models write hidden working before they answer, and it's billed as output, the expensive kind. Turning down the effort setting and asking for shorter answers trims the biggest line on the bill. Whether answers stay good enough is the real question, and the arithmetic can't answer it.

Wait when you can. Work that doesn't need an answer in seconds, such as overnight digests or bulk summaries, can go through the providers' batch service at half price. An interactive assistant has little of it; a product built around overnight work has a lot.

Price for the tail. The cost levers lower the slope of the line. Pricing decides who pays for what's left, and it's the only lever that touches revenue. A hard cap on heavy users protects the margin and punishes your best customers. An allowance with usage-based overage keeps them and charges for the excess: include 300 tasks, charge ten cents for each one after. The heavy user now pays $60 instead of $30 and, after the cost levers, costs under $12.

Stack all of it and the example seat goes from 39 percent to about 77 percent, with model calls down to 8 percent of revenue. The levers don't add up neatly, because each one shrinks the cost the next one acts on. And the end state rests on two assumptions worth naming: that your tasks really do separate into easy and hard, and that heavy users will accept a price that follows their use. Anthropic's weekly limits on Claude Code and Cursor's move to usage-based credits, both in 2025, were the industry learning the second lesson in public (Capability, Product and Business Model).

Cost levers lower what each task costs. Pricing decides who pays for the tasks that remain.

When people are part of the cost

Some AI products sell a finished result, and people do part of the work behind it. Take a support agent priced per outcome, at the $0.99 Intercom lists for its Fin agent, and charge only for conversations the agent resolves. Assume each conversation costs about 11 cents in model calls, the agent resolves 60 percent, and a reviewer checks 8 percent of conversations for four minutes each at $36 an hour.

Unresolved conversations cost money and earn nothing, so the cost per resolution comes to 50 cents and the margin to about 50 percent. People are 64 percent of that cost. A tenfold fall in token prices lifts the margin only to 66 percent. Cutting the review sample to 2 percent lifts it to 74. In this business, the people and the resolution rate move the margin more than the model price does.

Two consequences. An outcome price moves the cost of failure onto the vendor, so the margin tracks how often the agent actually succeeds. And where a company books its reviewers is a choice: put them under operations instead of cost of revenue and the same business reports a prettier gross margin. When someone shows you "software-like" margins for a product with people in the loop, ask which people are inside the number.

"But tokens get ten times cheaper every year"

This is the strongest objection, and part of it is right. The price of a token at a fixed level of quality has been falling by five to ten times a year or more. Cut the example's model price tenfold and leave everything else alone, and the blended margin jumps from 39 percent to 79, above Salesforce.

Scroll sideways to see the whole figure

The trouble is "leave everything else alone". Nobody does.

First, customers don't buy last year's quality at this year's price. They buy the newest models. Among businesses Ramp tracks, token use grew about elevenfold and spending about sixfold between January 2025 and April 2026, which means the price actually paid per token fell only about 1.6 times a year. Premium models went from under 6 percent of AI token spending in June 2025 to 56 percent in April 2026. The price cut that matters to your product is the one on the tier your customers insist on (Where the AI Money Goes).

Second, cheaper tokens get spent. Agents loop, reasoning models think longer, and users find more tasks once each one is cheap, the old Jevons paradox. If the price paid falls 1.6 times while tokens per task and tasks per seat each grow by half, the margin ends up at 22 percent, lower than today. Even a tenfold price fall leaves the margin at 48 percent if tokens per task quadruple and tasks per seat double.

None of this is a forecast. It's arithmetic that shows which way the margin goes: it recovers only if the cost of a task falls faster than tokens per task and tasks per seat grow. The price unit decides who absorbs the growth. On a flat seat, all of it lands on the company. Priced per task, growth in tasks pays for itself and only fatter tasks still hurt. Priced per outcome, the vendor carries both.

And competition takes its share. When costs fall for everyone, customers expect prices to follow. The share of a cost cut that reaches customers is called pass-through, and competition pushes it up (Who Receives the Gain?). A company that cuts its model costs tenfold and passes half on can earn more gross profit per task at a margin percentage that barely moves.

Why investors accept it, and when that's a mistake

Investors fund thin margins because of what the margin buys. Bessemer's ten fastest growers averaged about $40 million of annualized revenue in their first year, and Bessemer reads their low margins as partly a decision to buy distribution. A large customer base on a thin margin can be repriced or re-engineered later. A missing customer base can't be bought back.

That bet is reasonable. It fails in predictable ways: when customers chase the newest model so cost per task barely falls, when competitors pass the savings through, when usage grows on a flat price, when the supplier who sets your costs also builds products that compete with you, and when customers don't stay long enough. A thin margin stretches the time it takes to earn back what it cost to win a customer; on a $40 seat, payback goes from about 16 months at 77 percent to about 33 months at 36.

My judgment is that the margin question separates AI companies into two kinds. A product whose value comes from what the model can't supply on its own, such as access to the customer's data and a place inside their daily work, can raise its price as the models improve, and that cushions the cost. A thin layer over someone else's model can't. Its supplier sets its costs, and its customers can see the price list.

What to do

Compute margin by usage segment. The blended number hides the loss, and the typical user hides the tail. Know what your heaviest tenth costs you.

Count cost per accepted task. Include retries, reruns on a bigger model and human review. Price per token is the wrong unit (Cost per Task, Not Cost per Token).

Match the price unit to what drives cost. A seat price fits a bounded task. An open-ended agent needs an allowance, a meter or a price per outcome.

Route before you optimize anything else, and test that quality holds on your own work before you trust the savings.

Treat the cost levers as a one-time recovery. They lower the line once. Usage growth tilts it back, every year, unless the pricing moves with it.

Check yourself

Why does judging by the typical user overstate the margin?

Margin is a straight line in usage, so the blended margin equals the margin at the average usage, and the heavy tail pulls that average up. In the example the typical user suggests a margin about 12 points too high.

Why can cheaper tokens fail to restore the margin?

Customers move to newer models, so the price they pay falls slowly, and cheaper tokens get spent on longer tasks and more tasks.

What does an allowance with overage change that cost levers do not?

It changes who pays for heavy use. The heavy user pays for the excess, which touches revenue, while cost levers only lower the cost of each task.