Interactive Tool
What does it cost to host a private LLM?
There are three ways to buy LLM capacity, not two. This free calculator compares frontier APIs (OpenAI, Anthropic, Google, Mistral), open-weight models served via API (Together AI, DeepSeek, Groq), and hosting your own hardware — including hardware amortisation, electricity, and prompt caching savings.
How the cost to host a private LLM is calculated
Self-hosting has three real costs, and most comparisons ignore two of them. The hardware is the obvious one — anything from a consumer GPU (RTX 4090 or 5090) to a 128 GB unified-memory machine (Mac Studio, AMD Strix Halo, NVIDIA DGX Spark) to a datacentre card (A100, H100), or a rented cloud GPU if you'd rather not own anything. Then electricity: power draw times hours running times your kWh rate, which for an always-on GPU is not a rounding error. Then amortisation: the purchase price only makes sense spread over the months you'll actually use the machine.
The calculator folds all three into a cost per million tokens, so self-hosting can be compared like-for-like against cloud pricing at whatever usage level you choose.
Cloud LLM costs: what you're actually paying
Cloud LLM pricing is per million tokens, split into input and output rates that differ by an order of magnitude across models. The number most estimates get wrong is prompt caching: if your requests reuse the same system prompt or context — and in production, most do — a large share of input tokens bills at the discounted cached rate. The calculator models caching explicitly, because ignoring it makes cloud look more expensive than it is. I've published my own cost breakdown — cutting an agent stack from $1,000 to $100 a month — and this tool uses the same accounting.
The middle path: open-weight models via API
The self-host question is usually posed as a binary — frontier API or your own GPU — and the most useful option sits between them. Open-weight models (Llama, DeepSeek, Qwen, Kimi, gpt-oss) can be bought per token from serverless inference providers like Together AI, DeepSeek's own API, Groq, or Fireworks: the same buying experience as OpenAI or Anthropic, at open-model prices, with no servers to run. If procurement or data residency is the blocker, the managed catalogues — AWS Bedrock, Google Vertex AI, Azure AI Foundry — serve the same models inside your existing cloud account at broadly similar rates. And if you need fine-tuning or full control, renting dedicated GPUs (RunPod, Lambda, or a hyperscaler VM) is the bridge to the self-hosted column. The calculator prices the serverless tier as its own column, because for many workloads it beats both extremes.
The honest answer: utilisation decides
At low or bursty volume, per-token APIs win: no capital outlay, no idle hardware, someone else's electricity — and open-weight models via API often win outright on price. Self-hosting a private LLM wins on cost only once you push enough tokens through the hardware, consistently, to beat the per-token price — or when the driver isn't cost at all but data residency, privacy, or latency. Run your own numbers above; the crossover point is usually further out than the "just buy a GPU" advice suggests.
Common questions
Is it cheaper to self-host an LLM or use cloud APIs?
It depends almost entirely on utilisation. Cloud APIs win at low and bursty volumes because you pay per token with no upfront cost. Self-hosting wins when you push enough tokens through the hardware, every day, to amortise the purchase price and electricity. The crossover point is much higher than most people expect — which is exactly what this calculator works out for your numbers.
How much does it cost to host a private LLM?
Three costs dominate: the hardware (from a consumer GPU like an RTX 4090 up to a datacentre H100, or a unified-memory machine like a Mac Studio, AMD Strix Halo, or NVIDIA DGX Spark), electricity (power draw × hours running × your kWh rate), and amortisation (how many months you spread the purchase over). The calculator combines all three into a cost per million tokens you can compare directly against cloud API pricing.
What do cloud LLM APIs actually cost?
Cloud providers price per million tokens, with separate input and output rates that vary widely by model. Prompt caching changes the picture significantly: if your requests reuse the same system prompt or context, 30–80% of input tokens can be billed at the cached rate. The calculator models caching explicitly, because ignoring it overstates cloud costs.
Do I have to self-host to use open-source models?
No — and for most teams it is the wrong first step. Serverless inference APIs like Together AI, DeepSeek and Groq host open-weight models (Llama, DeepSeek, Qwen, Kimi, gpt-oss) on their own hardware and bill per million tokens, exactly like a frontier API. You get open-model prices with no servers, no GPU procurement, and no ops. The calculator includes this tier as its own column, because it is often the cheapest per-token option of all.
What are Amazon Bedrock, Vertex AI and Azure AI Foundry?
Managed model catalogues from the big cloud platforms. They serve the same open-weight models as serverless APIs — per token, at broadly similar rates — but inside your existing corporate cloud account, which matters when procurement, data residency, or security review would otherwise block a new vendor. If your organisation already runs on AWS, GCP or Azure, this is usually the path of least resistance to open-weight economics.
What hardware do I need to run an LLM locally?
The binding constraint is memory. A 24–32 GB GPU (RTX 4090/5090) runs small and mid-size open models well. 96–128 GB unified-memory machines (Mac Studio, AMD Strix Halo, NVIDIA DGX Spark) fit much larger models at lower power draw. Datacentre cards (A100, H100) or a rented cloud GPU make sense at serious scale. The calculator includes all of these with realistic prices and power figures.