An AI architecture whose framework layers cost more than the models they orchestrate while contributing indirection rather than quality, because the decisions that determine quality have been outsourced to the stack.
In one line
The model does the work. The stack mostly moves the work around.
What you see
It started with one orchestration framework, adopted because the demo was slick and everyone else seemed to be using it. Then came the vector database, the agent layer, the observability platform for the agent layer, and a monthly bill that grows faster than usage. When someone finally asks what all of it contributes, the uncomfortable answer is: indirection.
I know this pattern from the inside: my own agent setup once cost $250 a week, and the output was still poor. The fix was not a better framework. It was removing the framework.
What is actually happening
The AI tooling market is young, loud, and incentivised to convince you that calling a model is too hard to do directly. In reality the model APIs are among the simplest interfaces in modern software, and each layer placed between you and them adds cost, latency, failure modes, and a vendor’s roadmap to your critical path.
Teams adopt middleware to avoid engineering decisions, then discover they have outsourced exactly the decisions that determine quality: what context the model sees, how outputs are verified, when to escalate to a human. Those decisions come back anyway, now expressed through a framework’s configuration language instead of your own code.
Why AI changes the constraint
This is constraint migration expressed in architecture. The models collapsed the cost of the hard part, so the constraint moved into coordination: the plumbing between the model and the workflow. Middleware promises to own that coordination and instead multiplies it, adding stages to the very part of the system that was already the constraint. The stack grows precisely where the knot sits, which is why the bill climbs while quality stands still.
What I see in practice
When a team audits its stack layer by layer, the layers that survive the “what breaks if we remove this?” question are almost always the boring ones: the model API, state, and the team’s own verification logic. The expensive ones survive meetings, not audits.
Observed in practice
A real engagement, anonymised to sector and scale. Every client gets that deal.
A tiny charity team, buried in admin, with no budget for new software. The good news: they didn’t need any.
A small charity with a handful of staff, all of them time-poor, all of them drowning in manual admin and a steady stream of inbound enquiries. Much of their operational data sat locked inside a proprietary system they couldn’t easily get at.
So I refused to sell them anything new.
Everything I proposed ran on tools they already pay for. Two practical uses, no extra subscriptions.
First, an assistant that reads inbound enquiries, sorts them, and drafts a reply, with a person approving every message before it sends. Nothing goes out unseen.
Second, a fairness audit of how staff shifts get allocated, so the rota could be checked for bias rather than trusted on instinct.
They came away with a route to claw back hours from admin without a single new tool, and a way to sanity-check something they’d only ever done by feel.
If that sounds like you: a small team with no budget for AI probably doesn’t need one. The tools you already pay for can usually take the admin off your plate, as long as a human stays in the loop.
How to diagnose it
Audit the stack by contribution: for each layer, what would break if it were removed? Strip anything whose honest answer is “the diagram would look less impressive”. Two numbers make the pattern undeniable: the ratio of framework spend to model spend, and the time it would take to swap models. If the first is above one and the second is measured in weeks, you are in the pit.
When this isn’t the pattern. If costs are climbing because usage is genuinely climbing, that is success wearing a scary invoice: check unit economics before cutting anything (the calculator exists for exactly this). And if quality is poor with a thin stack, the problem is context and verification design, not architecture.
What changes
Most production AI systems need a model API, a place to keep state, and disciplined prompts and verification. The money saved by stripping to that is real and immediate; the quality gain from owning your own context and verification logic is usually larger.
You know it’s working when the bill tracks usage, the architecture fits on a whiteboard, and swapping a model is an afternoon, not a migration. The decisions that determine quality — what context the model sees, how outputs are verified, when a human steps in — live in your own code, owned by your own people. That ownership, more than the money, is what compounds.
Related patterns and reading
- Constraint migration — the mechanism: here the constraint moved into coordination, and the stack grew on top of it
- I Cut My AI Agent Costs from $1,000 to $100/mo by Removing the Middleware
- An Executable Playbook for Your AI COO
- LLM Cost Calculator — run your own numbers