Interactive Tool
What does it cost to host a private LLM?
Cloud APIs or your own hardware? This free calculator compares the real cost of hosting a private LLM against cloud LLM pricing from OpenAI, Anthropic, Google and Mistral — including hardware amortisation, electricity, and prompt caching savings.
How the cost to host a private LLM is calculated
Self-hosting has three real costs, and most comparisons ignore two of them. The hardware is the obvious one — anything from a consumer GPU (RTX 4090 or 5090) to a 128 GB unified-memory machine (Mac Studio, AMD Strix Halo, NVIDIA DGX Spark) to a datacentre card (A100, H100), or a rented cloud GPU if you'd rather not own anything. Then electricity: power draw times hours running times your kWh rate, which for an always-on GPU is not a rounding error. Then amortisation: the purchase price only makes sense spread over the months you'll actually use the machine.
The calculator folds all three into a cost per million tokens, so self-hosting can be compared like-for-like against cloud pricing at whatever usage level you choose.
Cloud LLM costs: what you're actually paying
Cloud LLM pricing is per million tokens, split into input and output rates that differ by an order of magnitude across models. The number most estimates get wrong is prompt caching: if your requests reuse the same system prompt or context — and in production, most do — a large share of input tokens bills at the discounted cached rate. The calculator models caching explicitly, because ignoring it makes cloud look more expensive than it is. I've published my own cost breakdown — cutting an agent stack from $1,000 to $100 a month — and this tool uses the same accounting.
The honest answer: utilisation decides
At low or bursty volume, cloud APIs win: no capital outlay, no idle hardware, someone else's electricity. Self-hosting a private LLM wins on cost only once you push enough tokens through the hardware, consistently, to beat the per-token price — or when the driver isn't cost at all but data residency, privacy, or latency. Run your own numbers above; the crossover point is usually further out than the "just buy a GPU" advice suggests.
Common questions
Is it cheaper to self-host an LLM or use cloud APIs?
It depends almost entirely on utilisation. Cloud APIs win at low and bursty volumes because you pay per token with no upfront cost. Self-hosting wins when you push enough tokens through the hardware, every day, to amortise the purchase price and electricity. The crossover point is much higher than most people expect — which is exactly what this calculator works out for your numbers.
How much does it cost to host a private LLM?
Three costs dominate: the hardware (from a consumer GPU like an RTX 4090 up to a datacentre H100, or a unified-memory machine like a Mac Studio, AMD Strix Halo, or NVIDIA DGX Spark), electricity (power draw × hours running × your kWh rate), and amortisation (how many months you spread the purchase over). The calculator combines all three into a cost per million tokens you can compare directly against cloud API pricing.
What do cloud LLM APIs actually cost?
Cloud providers price per million tokens, with separate input and output rates that vary widely by model. Prompt caching changes the picture significantly: if your requests reuse the same system prompt or context, 30–80% of input tokens can be billed at the cached rate. The calculator models caching explicitly, because ignoring it overstates cloud costs.
What hardware do I need to run an LLM locally?
The binding constraint is memory. A 24–32 GB GPU (RTX 4090/5090) runs small and mid-size open models well. 96–128 GB unified-memory machines (Mac Studio, AMD Strix Halo, NVIDIA DGX Spark) fit much larger models at lower power draw. Datacentre cards (A100, H100) or a rented cloud GPU make sense at serious scale. The calculator includes all of these with realistic prices and power figures.