When self-hosting an open model beats paying per token.
The break-even math, including the part that argues against us: API prices keep falling, so the durable case is control, with cost as the bonus. Here is the model, built so you can run it with your own numbers.
In this article 4 sections
- Self-hosting beats per-token pricing once a workload is steady and its monthly token volume passes the point where a flat machine costs less than the same tokens on the API. Where that point sits depends on your model, your precision and your utilisation, so the post gives you the arithmetic rather than a number.
- Hosted API prices keep falling. Any pitch built purely on "cheaper than the API" decays; ours does not rest on it.
- A flat monthly machine turns an unpredictable usage bill into a fixed line item.
- Below break-even, the honest answer is to stay on the API for now, and an engineer here will tell you so.
There are two ways to sell GPU hardware. One is to promise it's cheaper than your API bill, which is sometimes true, temporarily, for some workloads, and stops being true every time the API providers cut prices again. The other is to do the math honestly: steady volume past a threshold favours owned hardware, bursty or small volume favours the API, and the durable reasons to own the machine (control, jurisdiction, a model version nobody can retire) hold at any price point.
The two bills, side by side
The API bill is a meter. Every month it is your token volume multiplied by the blended price per token on your invoice, and it moves with usage in both directions. The machine bill is flat. Whatever the machine costs per month, it costs that whether it serves one token or every token it has the throughput for. The entire break-even question is where those two lines cross.
Tokens
Price
API bill
Machine
Price
Break-even
That second line is the whole post. Divide the flat monthly cost of a machine that can serve your model by the price per token you pay today, and you have the monthly volume at which the two bills are equal. Everything else is about whether your workload really reaches that volume, and whether it reaches it steadily.
What moves the line
Model size
A larger model needs more memory and serves fewer tokens per second on the same card, so the machine that runs it costs more and the break-even volume sits higher. A smaller model moves it down. Pick the smallest model that passes your quality bar before you price anything.
Precision and quantisation
Running a model at lower precision shrinks its memory footprint and often raises its throughput, which lets a smaller machine do the job and lowers the break-even. The quality cost of that trade is measurable on your own evaluation set, and it should be measured, not assumed. We wrote up what quantisation does and does not cost in a 70B model really does fit on a 48GB card.
Utilisation
A machine busy a quarter of the time costs four times as much per token as the same machine kept busy. This is pure arithmetic and it is the term most self-hosting pitches skip. Steady, round-the-clock traffic earns a machine; a workload that spikes for two hours a day and sleeps otherwise rents better than it owns, unless the other reasons to own carry it.
The falling API price
Whatever break-even you compute today drifts upward every time the API price per token falls, because the same machine now has to displace cheaper tokens. A decision made on cost alone needs re-checking against each new price list. A decision made on control does not move.
Run it with your numbers
- 01Take the last three months of API invoices and read off two things: tokens per month, and the blended price per token (the bill divided by the tokens). Input and output tokens are priced differently, so blend them in the ratio you actually use.
- 02Note the shape of the volume, not just the total. Three similar months is steady. One month at triple the others is a spike, and a spike should be priced as rented capacity.
- 03Decide the model and the precision that pass your quality bar, on your own prompts. That choice sets which machine is needed, and we size it with you.
- 04Divide the machine's flat monthly cost by your blended price per token. That is your break-even volume. Compare it with your steady monthly volume, with the utilisation you can realistically keep.
- 05If your steady volume sits above break-even with room to spare, the machine wins on cost and on everything else. If it sits below, stay on the API for now and come back when it has grown, or decide on the control reasons alone, with eyes open.
The case that does not decay
The cost case is real for steady, high-volume workloads, and it is the weakest reason to own a machine, because it is the only one a competitor's price cut can take away. The others hold at any price point. Your prompts stay on hardware that is yours, in a jurisdiction you chose. The model version you validated stays available until you decide otherwise, rather than until a vendor retires it. Your busiest day is served at the machine's full throughput, not at a rate limit set by someone else's capacity planning. And the bill is the same number every month, which is a different kind of saving from a cheaper token.
That is why we lead with the arithmetic rather than a promise. If the math says the machine wins, good. If it says the API still wins on cost, the decision becomes about the other reasons, and some workloads should still be on the API. We would rather say that than size a machine to a volume that is not there.
Straight answers
Where is the break-even, roughly?+
It is your machine's flat monthly cost divided by the price per token you pay on the API, and it moves with your model, your precision and your utilisation, so a single number would be wrong for most readers. Steady is the load-bearing word in every version of the answer; a spiky workload rents better than it owns.
Won't falling API prices erase the case entirely?+
They erode the pure cost case, which is why we don't lead with it. They do nothing to the control case: your data staying on your machine, your model version staying available, your bill staying flat regardless of usage.
Will you tell us if self-hosting doesn't add up for us?+
Yes. Bring the invoices and an engineer will run the model with you. A machine sized to a workload that is not there yet helps nobody; we would rather you come back when the volume is.
Does the machine have to be bought, or can it be rented monthly?+
Either, and the arithmetic is the same: a flat monthly figure against a metered one. The point of owning or dedicating a machine is that the cost stops moving with usage, not the accounting treatment.
Bring us your AI API bill
An engineer does the break-even math with your numbers, and says so if the API is still the right answer.
See GPU ServersYou’ve reached the end.
Put the reading to work
What are
you running?
Tell us about the workload, traffic and recovery requirements. Start with the server builder, or talk through the details with an engineer.
Written by the engineers who build and run the infrastructure it describes.Talk to an engineer