← Back to the blog
Per-token cost of each model tier: Haiku 1×, Sonnet 20× and Opus 40×

Three LLMs Are Better Than One: Cost Optimization in AI Ops

Using one big model for every incident burns money. Here's how KubeBolt splits the work across model-free rules, Haiku, Sonnet and Opus, and how we measure each stage before picking its model.

The fastest way to build an AI Ops product that loses money on every customer is to pick your smartest model and route everything through it. It feels responsible — Opus-grade reasoning on every alert, no compromises, maximum quality. Then the bill arrives and you discover you just paid frontier-model prices to classify a routine pod restart as “probably fine.”

This is the mistake almost everyone makes first, and it’s exactly the one KubeBolt’s Autopilot engine was designed to avoid. The thesis is simple: almost no step of an incident needs your best model, and much of what you detect has no reason to reach one at all. The trick is knowing which is which before you spend the tokens — and measuring it instead of assuming it.

The naive approach and why it burns money

Kubernetes clusters are event firehoses. A single cluster under moderate stress emits hundreds of Warning events an hour: BackOff, FailedScheduling, OOMKilled, transient probe failures, the pod that restarts once and recovers. If your architecture is “every event goes to a big model,” you’re paying frontier prices for a stream that is overwhelmingly noise and well-known patterns.

Model pricing makes this brutal. The numbers move with every release, so treat any figure as a snapshot. Today’s: per token, Claude Sonnet 5.5 costs 20 times what Claude Haiku 5.5 does, and Claude Opus 5.5, 40 times. And an investigation isn’t one call: it chains several with tools, each carrying the accumulated context. Send it all to the big model and your cost per incident is dominated by the incidents that never needed that much intelligence.

Worse, it’s slower and no more accurate. A frontier model asked “is this pod restart noise?” gives you the same answer as a cheap one, just later and more expensively. We measured it, and we show the numbers below.

How KubeBolt splits the work across its layers

Autopilot is a six-layer engine. Five use a model, and each gets the cheapest one that does its job well. This is the default split:

LayerJobDefault modelPrice per tokenWhen it runs
L1 DetectorRecognize the problem and decide whether it gets innonenothingOn every finding
L2 RouterTriage: real severity and whether it goes onHaiku 5.51×Findings of the 8 cataloged types
L3 InvestigatorThe cause, with tools over several turnsSonnet 5.520×What triage lets through
L4 PlannerThe plan, within a closed catalogSonnet 5.520×When there’s a cause
L5 ExecutorApply the plan with typed actionsSonnet 5.5, or none20×, or nothingOn every approved plan
L6 PostmortemWrite the reportOpus 5.540×Once, at close

Each stage also has a fallback model, and a platform admin can change both.

Layer 1, the Detector, uses no model, and its job is to decide what gets in. It’s the 24 deterministic rules of the Insights Engine, the same one that ships in the open source agent. They don’t resolve anything: they surface the problem and leave it with you, recommendation included. What saves tokens is that Autopilot only opens an incident for the 8 types it knows how to handle with its action catalog. Everything else stays a finding for you, and never reaches a model. The most important layer in a system built on LLMs is the one that calls none.

Layer 2, the Router, is a Haiku 5.5 triage. It classifies the finding, decides its severity from evidence of impact, and decides whether it deserves an investigation. Noise stops here. It’s deliberately the cheapest model doing the cheapest task, because triage is a filter, not an analysis. It runs without extended reasoning: when we tested it with reasoning on, results got worse, because it investigated more than its five turns can hold.

Layer 3, the Investigator, is Sonnet-tier, and this is where the real work happens. It uses tools over several turns, up to fourteen: pulls logs, describes the deployment, queries metrics, cross-checks events and recent deploys. It iterates until it lands on a cause it’s confident in, or flags that a person is needed. Sonnet is the right tier here because investigating takes genuine reasoning and tool handling, but not the absolute frontier.

Layer 4, the Planner, is also Sonnet. An earlier version of this article said hard plans escalated to Opus. They don’t. The Planner picks actions from a closed catalog of eight (six that change the cluster, one that verifies and one that only recommends) and fills in parameters the tool validates. It’s a bounded decision, and Sonnet makes it well.

Layer 5, the Executor, applies the plan. By default it’s a Sonnet agent that can only call the typed actions, in order, and every call goes through a gate written in code: approvals, blocked namespaces, cancellation. On the Enterprise edition it can be code with no model (autopilot.executor: code), and then executing spends no tokens. Either way, no model decides anything that isn’t in the catalog.

Layer 6, the Postmortem, is the only place with Opus. It runs once, when the incident closes, off the critical path. That’s where we happily spend on quality, because it blocks nothing and it’s the report a person actually reads. Opus isn’t the default you optimize down from — it’s the exception, and we put it where its price pays off most.

Routing logic: what decides each step

Moving from one layer to the next isn’t a vibe. Every boundary has an explicit signal:

  • Layer 1 to 2: the finding belongs to one of the 8 cataloged types. If not, it stays a finding, with its recommendation, and costs nothing.
  • Layer 2 to 3: the Router decides it isn’t noise. This is the highest-leverage filter in the system, because it’s the cheap gate that keeps noise from reaching a model twenty times more expensive.
  • Layer 3 to 4: the Investigator reaches a cause. If it concludes a person is needed (because the cause is outside the cluster, for example), the plan touches nothing and the incident goes to that person.
  • Layer 4 to 5: the plan’s confidence clears the threshold for its incident type — 0.7 in general, and 0.85 where a wrong action does more damage. If not, the plan stays a suggestion.

On top sits consumption control. A concurrency cap keeps a bad afternoon from fanning out into dozens of parallel investigations: the rest wait in a queue. And credit spend can be spread across the month, so a day full of incidents doesn’t eat the budget for the days after it; critical incidents can skip that spreading.

The numbers, honestly

Here’s the part where AI Ops vendors usually quote a suspiciously round dollar figure. We did too: the first version of this article said “under a dollar per incident, end to end.” It was an MVP estimate, not a measurement, and we’ve withdrawn it.

What we do have are per-stage measurements. Before changing a stage’s default model, we test it on its own with a harness, against cases recorded in our lab, with a replay server answering the read tools. Two of those measurements:

  • Triage, Haiku 5.5 vs Haiku 4.5 (195 runs over 39 cases): the same rule and severity decisions, half the latency (an 8-second median) and 14 times less cost per run. All 195 runs cost $0.32 — under two tenths of a cent each.
  • Investigator, Sonnet 5.5 vs Sonnet 5 (31 cases): the right cause in every run with both models, but Sonnet 5.5 named the remedy in 100% of cases versus 90%, cost 22% less per run ($0.021) and answered 25% faster (a 10-second median).

Look at the ratio: an investigator run costs about thirteen times a triage run. That’s the gradient at work. Noise gets filtered for a fraction of a cent, and the real money is only spent on what made it past the filter.

These are lab numbers, not production numbers, and they don’t add up to an end-to-end cost: that takes measuring a whole incident, every stage of it, with the same care. When we have it, we’ll publish it. What lasts isn’t any dollar figure — it’s the ratio between tiers and the architecture that exploits it.

Resilience: fallback, brake and a second engine

Splitting work across model tiers solves cost. Availability is a separate problem, and it’s worth being precise about what we have.

Every stage has a fallback model. If the primary fails or times out, the stage retries on the fallback instead of losing the incident. The fallback is the previous version of the same model, proven before the new one arrived.

In front sits a circuit breaker, born from a bad day. One afternoon our provider’s API stopped answering for an hour and a half, and Autopilot kept starting runs as if nothing had happened: every stage burned its timeout, retried on the fallback, burned it again, and each run ended twenty-five minutes later having done nothing. Now, after several consecutive provider-caused failures, the breaker opens and no new run starts. After a while it lets one probe run through: if it succeeds, it closes; if not, it waits again.

And there’s a second engine. Since 2.5, on top of Anthropic’s agent SDK, Autopilot on the Enterprise edition can run on a gateway engine that talks to any OpenAI-compatible endpoint, such as GLM or Kimi, with its own model and fallback per stage. An admin picks it. It isn’t an automatic jump from one provider to another when one goes down; it’s the option of not depending on just one. Kobi Copilot, for its part, runs on KubeBolt Cloud with Anthropic as primary and OpenAI as fallback.

The honest trade-off

None of this is free, and it would be dishonest to pretend otherwise. A single-model architecture is genuinely simpler: one client, one prompt style, one set of failure modes, one thing to reason about. What we’ve described is six stages, per-stage prompts, confidence thresholds that need calibrating, a queue, credit spreading, a circuit breaker, fallback models and two engines. That’s real operational surface: more code to test, more edges to get wrong.

That complexity bites hardest when the split misfires. A Router that’s too aggressive marks a real incident as noise and something slips through; one that’s too timid lets noise into paid investigations and quietly erodes the savings the whole design exists to capture. The thresholds at every boundary drift. That’s why we measure each stage before changing its model, and why it’s ongoing work, not a one-time setup.

So the honest framing is a trade, not a free lunch: you take on engineering complexity in exchange for paying each step the price its job calls for. For a tool you run occasionally, it isn’t worth it — run a good model and move on. For an autonomous system processing every production event around the clock, the cost of not splitting compounds on every incident, and the complexity pays for itself. That’s the regime KubeBolt operates in.

Three LLMs really are better than one. But the sharpest part of the design is the layer that decides, with no LLM at all, what reaches the others. The full picture is in How it works and in the six-layer architecture, and the Layer 1 rules are on GitHub, Apache 2.0, readable end to end.