The fastest way to build an AI Ops product that loses money on every customer is to pick your smartest model and route everything through it. It feels responsible. Opus-class reasoning on every alert, no compromises, maximum quality. Then the bill arrives, and you discover you just paid frontier-model prices to classify a routine pod restart as “probably fine.”

This is the mistake almost everyone makes first, and it’s the one we designed KubeBolt’s Autopilot engine specifically to avoid. The thesis is simple: most incidents don’t need your best model, and a large share don’t need a model at all. The trick is knowing which is which before you spend the tokens.

The naive approach and why it burns money

Kubernetes clusters are event firehoses. A single cluster under mild stress emits hundreds of Warning events an hour: BackOff, FailedScheduling, OOMKilled, transient probe failures, the pod that restarts once and recovers. If your architecture is “every event goes to one big model,” you’re paying frontier prices for a stream that is overwhelmingly noise and well-understood patterns.

The cost structure of frontier reasoning models makes this brutal. The exact numbers move every quarter and vary by provider, so treat any hard figure as illustrative. But the shape is stable and it’s what matters: an Opus-class call can cost on the order of 100x a Haiku-class call for the same token volume, and a deep investigation that burns tens of thousands of tokens on multi-turn tool use costs far more again. Run that on every event and your cost-per-incident is dominated by the incidents that never needed the intelligence in the first place.

Worse, it’s slower and no more accurate. A frontier model asked “is this pod restart noise?” gives you the same answer a cheap model does, just later and pricier. You’ve bought nothing. The waste isn’t a rounding error; it’s the whole bill.

How KubeBolt routes across its layers

Autopilot is a six-layer engine. Only three of those layers touch an LLM, and each layer’s job is to resolve the incident as cheaply as possible or hand it up to the next tier with a clear reason. The LLM proposes; a deterministic executor acts.

Layer	Job	Model tier	Relative cost	When it’s used
L1 Detectors	Match known patterns and remediate	none (deterministic)	~free	Every incident, first
L2 Router	Triage: noise vs investigate vs critical	Haiku-class	baseline (1x)	Anything L1 can’t match
L3 Investigator	Root-cause via multi-turn tool use	Sonnet-class	~a few × baseline	Non-noise incidents
L4 Planner	Write the remediation plan	Sonnet, Opus for hard cases	up to ~100x baseline	Only when a plan is needed
L5 Executor	Apply the plan under guardrails	none (deterministic)	~free	Every approved plan
L6 Postmortem	Write the incident report	Opus-class	high, but async	After resolution, off critical path

Relative-cost figures are illustrative and per-token; real prices shift with every provider update. The multipliers between tiers are what stay stable, and they’re the whole reason the routing exists.

Layer 1, the Detectors, is where the savings really live, and it uses no LLM at all. These are deterministic rules, ported from the same Insights engine that ships in the open-source agent. OOMKilled after a deploy, an image pull failing on a missing tag, a PodDisruptionBudget blocking a drain: these are patterns we already know how to recognize and remediate in plain Go. A well-understood share of real-world Kubernetes incidents fall into known shapes, and Layer 1 resolves them in under 50 milliseconds with a confidence score and zero token spend. The most important layer in a system built on LLMs is the one that doesn’t call one.

Only when Layer 1 can’t confidently match does the event escalate, and even then it climbs the tiers one rung at a time.

Layer 2, the Router, is Haiku-class triage. Its only job is to answer one question in under 200 tokens of output: is this noise, does it investigate, or is it critical? Noise gets marked and dropped. This is deliberately the cheapest possible model doing the cheapest possible task, because triage is a filter, not an analysis. At roughly 100x less than an Opus call, you can afford to run it on everything Layer 1 waved through and still barely move the meter.

Layer 3, the Investigator, is Sonnet-class and this is where real work happens. It runs multi-turn tool use, the same seventeen tools the Copilot exposes, in autonomous mode: pull pod logs, describe the deployment, query a metric, correlate recent events. It iterates until it reaches a hypothesis with confidence above 0.7, or gives up after a bounded number of turns and flags the incident for a human. Sonnet is the right tier here because investigation needs genuine reasoning and tool orchestration, but not the absolute frontier. Most root causes don’t require it.

Layer 4, the Planner, is the only place an Opus-class model earns its keep, and even then only sometimes. If the investigation is low-complexity and single-step, Sonnet writes the plan. Opus is reserved for the genuinely hard cases: multi-step remediations, sophisticated risk trade-offs, plans where getting the ordering wrong makes things worse. This is the inversion most teams miss. Opus isn’t the default you optimize down from; it’s the exception you escalate up to.

Layer 5, the Executor, is deterministic again. It takes the plan and applies it action by action against a closed whitelist of about twenty-five operations, under RBAC, with automatic rollback if verification fails. No LLM decides what runs in your cluster. The model writes a proposal; code with guardrails executes it. That boundary is a safety property, and it’s also a cost property: execution is free of token spend.

Layer 6, the Postmortem, runs Opus asynchronously after resolution. It’s the one place we happily spend on quality because it’s off the critical path, it doesn’t block anything, and a good write-up is worth it.

Routing logic: what decides the escalation

The escalation isn’t vibes. Each boundary has an explicit signal:

Layer 1 → Layer 2: the deterministic detectors don’t match, or match with confidence at or below 0.9. A high-confidence match short-circuits the LLM layers entirely; a weak match still escalates but is passed down as a hint to the investigator.
Layer 2 → Layer 3: the Router classifies the event as investigate or critical. A noise verdict ends the incident. This is the single highest-leverage filter in the system, because it’s the cheap gate that stops noise from ever reaching a mid-tier model.
Layer 3 → Layer 4: the Investigator reaches a hypothesis it’s confident enough to act on. If it can’t clear the confidence bar within its turn budget, it escalates to a human instead of guessing.
Layer 4 model choice: Opus only when the investigation is flagged high-complexity or the plan is multi-step. Otherwise Sonnet writes it.

Layered on top is backpressure, which is really cost control wearing a resilience hat. A per-org semaphore caps concurrent investigations, so a bad afternoon can’t fan out into hundreds of parallel Sonnet sessions. And when a cluster emits more than ~50 events a minute, Autopilot stops investigating individual events entirely: that’s not fifty incidents, it’s one outage. It collapses them into a single “cluster outage” incident and pages a human directly. Investigating each event separately during a real outage would be both useless and ruinously expensive, so we don’t.

The numbers, framed honestly

Here’s the part where AI Ops vendors usually quote a suspiciously precise dollar figure. We won’t, because per-call model prices shift with every provider update and any number I write here is stale by next quarter. What’s durable is the ratio and the architecture that exploits it.

The multipliers are the point. When an Opus-class call costs on the order of 100x a Haiku-class one, the entire economic game is keeping work at the lowest tier that can actually resolve it. A system that resolves a large share of incidents at Layer 1 for zero tokens, filters most of the rest through a Haiku-class gate, handles the survivors with Sonnet, and reserves Opus for a thin slice of genuinely hard plans has a wildly different cost curve than “Opus on everything.” Not a little cheaper. Different order of magnitude.

Concretely, that’s what lets KubeBolt Autopilot resolve real incidents end-to-end in under 90 seconds, at under $1 each including the LLM spend. The single-big-model version of the same product doesn’t just cost more per incident; it costs more on the incidents that needed the least help, which is exactly backwards. Route intelligently and the cheap-and-common incidents cost almost nothing, so your average stays low even when the occasional gnarly one runs the full stack up to Opus.

Resilience: multi-provider failover

Routing across model tiers solves cost. Routing across model providers solves availability, and in production you need both.

Every LLM call in Autopilot goes through an internal AI Gateway that fails over across three providers in order:

1. Anthropic API      (primary)
2. Google Vertex AI   (failover #1)
3. AWS Bedrock        (failover #2)

A circuit breaker marks a provider down for 60 seconds after three consecutive failures, so a regional blip at one provider doesn’t stall your incident pipeline: the gateway rolls to the next and keeps going. Because the same Claude model families (Haiku, Sonnet, Opus) are available across all three surfaces, failover doesn’t change the routing logic or the quality of the answer. The tier stays the same; only the road it takes changes. This is the difference between “our AI Ops broke because Anthropic had a bad five minutes” and a footnote in a log.

The honest tradeoff

None of this is free, and it would be dishonest to pretend otherwise. A single-model architecture is genuinely simpler. One client, one prompt style, one set of failure modes, one thing to reason about. What we’ve described is a routing layer with six stages, per-tier prompts, confidence thresholds you have to tune, a semaphore, an outage detector, a circuit breaker, and a triple-provider gateway. That’s real operational surface area. It’s more code to test, more edges to get wrong, more dashboards to watch.

You feel that complexity most when routing misfires. A Router that’s too aggressive marks a real incident as noise and you miss something; too timid and it escalates noise into paid investigations, quietly eroding the savings the whole design exists to capture. The confidence thresholds at each boundary are knobs, and knobs drift. Getting them right is ongoing work, not a one-time setup, and it’s the kind of work that only shows up in aggregate metrics rather than any single incident.

So the honest framing is a trade, not a free lunch: you take on meaningful engineering complexity to buy an order-of-magnitude cost reduction and provider-level resilience. For a tool you run occasionally, that trade isn’t worth it, run one good model and move on. For an autonomous system chewing through every event in production around the clock, the cost of not routing compounds on every single incident, and the complexity pays for itself many times over. That’s the regime KubeBolt operates in, which is why the routing layer exists.

Three LLMs really are better than one. But the sharpest part of the design is the layer that resolves the most incidents using no LLM at all. If you want the full picture, the architecture is documented in the docs, and the open-source agent that powers Layer 1’s detectors is on GitHub, Apache 2.0, readable end to end.