Own, rent, cache

Most build-versus-buy analyses for LLM inference start from price per token, conclude that self-hosting is cheaper at volume, and quietly fail eighteen months later when nobody can staff the cluster. This framework inverts the order of the decision. Price is the last input, not the first.

It comes out of a six-month enterprise engagement evaluating a managed inference service against an in-house stack — Triton with TensorRT-LLM, Ollama, and quantized open-weight models — where all of those were real candidates rather than a survey. The recommendation was hybrid routing. What follows is the reusable method, with the cost model parameterized so you can put your own volumes through it.

The framework

Five steps, in this order. The order is the whole argument — running them in any other sequence produces the cheap answer that later becomes expensive.

1. 01

   Classify the workload, and gate it

   Enterprise inference is not one workload, and treating it as one is what makes the cost question unanswerable. Each class gets an explicit permission envelope before it gets a routing decision. An ungated workload cannot be costed, because you do not know its volume.

2. 02

   Place each workload by value and risk

   Two questions, asked independently. How much proprietary value does this workload carry — is the output something a competitor could buy? And how much internal risk does it carry — what is the exposure if it leaks, drifts, or acts wrongly? Neither question mentions money.

3. 03

   Own what is high-value and high-risk; rent the rest

   The model you train is the model that carries the most internal value and the most internal risk. That is the one worth owning the weights, the serving path, and the audit trail for. Everything that is genuinely commodity gets rented, because owning it buys you nothing you could not purchase.

4. 04

   Let the savings fall out of cache and commodity shift

   Cost reduction is a consequence of steps 1 through 3, not a goal pursued on its own. Two mechanisms produce nearly all of it: cache hit rate, which removes volume from the billed path entirely, and shifting the remaining eligible volume onto commodity inference. GPU amortization is a distant third and usually a distraction.

5. 05

   Cap ownership at what you can staff

   In-house inference at enterprise scale is operating a supercomputer. It needs a small number of unusually senior people, and that is a hiring constraint before it is a budget line. The savings from steps 3 and 4 set an ownership budget; if that budget will not fund the team the cluster requires, the self-hosted path is wrong regardless of what the spreadsheet says.

Step 01 — Workload classes and their gates

Four classes, each admitted under a different condition. The gate is the governance artifact and the volume input at the same time, which is why the governance model and the infrastructure decision are one decision rather than two.

Retrieval doubling as the drift reference is the load-bearing detail. If you have no stable workload to measure against, you cannot tell a model regression from a data change, and every later routing decision rests on an unverifiable assumption about output quality.

Step 02–03 — Placement and its routing consequence

The grid is deliberately about value and risk rather than cost or latency. Where a workload sits determines who serves it; the red-team / blue-team cycle is what moves workloads between quadrants over time, which is how the boundary gets steered rather than guessed.

Red team, blue team is the steering mechanism. Placement is a judgement, and a judgement made once decays. Adversarial exercises against the deployed workloads are what produce directional evidence that something has moved — a workload that turns out to leak more than expected moves up in risk, one whose output turns out to be reproducible from public models moves left in value. The grid is re-derived from those results rather than defended.

Step 04–05 — The cost model

This is the arithmetic that falls out of the placement, with every input exposed. The two levers that matter are cache hit rate and the share of remaining volume that is eligible for commodity routing. Move them and watch which outputs actually respond.

The output worth arguing about is not the percentage. It is the ownership budget: the monthly figure the savings make available before self-hosting stops paying for itself, and therefore the size of team those savings can fund. Step 05 exists because that number usually converts to two or three people — and two or three people who can run a production inference cluster is a harder thing to acquire than the budget for them.

What this framework decides, and what it does not

It decides which workloads justify ownership, which get rented, what each class is permitted to do, where the cost actually comes from, and whether the ownership path is staffable. Those five answers are what makes an inference program fundable and auditable rather than a collection of pilots with a cloud bill attached.

It does not decide model selection, evaluation methodology, or serving topology. Those are downstream engineering decisions, and they get easier once the questions above have answers. Nor does it survive contact with an organization that will not gate its workloads — without step 01 there are no volumes, and without volumes the rest is arithmetic on invented numbers.