KubeBolt docs
ENES GitHub

Core Concepts

Cost & Rightsizing (Beta)

Per-cluster cloud spend powered by OpenCost — run-rate, idle cost, and money-priced rightsizing recommendations. How each number is computed, the three ways to connect OpenCost, and how to verify it.

On this page
  1. What the Cost tab shows
  2. How the numbers are computed
  3. How rightsizing decides
  4. When recommendations appear
  5. Requirements
  6. Three ways to connect OpenCost
  7. What each path ships
  8. Verify

When OpenCost data flows from a cluster, a Cost sub-tab (marked Beta) appears on its dashboard, and the spend figures on Home and Fleet start counting it. There is no new backend service to operate: the tab is a lens over cost metric families the agent ships into KubeBolt’s metrics store, scoped per cluster like everything else. KubeBolt does not call OpenCost’s API — OpenCost is a Prometheus exporter, and its metrics travel the same path as the rest.

The tab shows only on a cluster whose cost samples have reached KubeBolt (it checks for node_total_hourly_cost, once a minute). A cluster with no cost feed gets no empty tab.

What the Cost tab shows

The Cost tab: spend per namespace and per workload, priced from OpenCost.
The Cost tab: spend per namespace and per workload, priced from OpenCost. KubeBolt 2.1.1
Rightsizing: each recommendation priced in money, not in millicores.
Rightsizing: each recommendation priced in money, not in millicores. KubeBolt 2.1.1

Kobi reads the same engine through get_right_sizing, so asking it in the chat and reading the Capacity or Cost screen give the same numbers. The recommendations are not recomputed for the conversation.

How the numbers are computed

Every figure is a query over OpenCost’s metrics for the selected cluster.

FigureComputed as
Monthly run-ratesum(node_total_hourly_cost) × 730 — the current hourly cost projected to a month, not month-to-date
AllocatedEach container’s CPU allocation (container_cpu_allocation) × its node’s node_cpu_hourly_cost, plus its memory allocation in GiB (container_memory_allocation_bytes) × the node’s node_ram_hourly_cost
IdleRun-rate minus allocated. It includes node system overhead and the headroom you keep on purpose, so not all of it is reclaimable
NetworkLoad-balancer and egress cost families (kubecost_load_balancer_cost, kubecost_network_*_egress_cost), shown apart from compute
Storagepv_hourly_cost; $0 where OpenCost has no disk price
Rightsizing savingsReclaimable cores × the cluster’s average CPU hourly rate, plus reclaimable GiB × the average RAM rate, × 730
EfficiencyCPU in use ÷ CPU requested, across all namespaces

Prices are the list prices OpenCost resolves for your provider and region, not your invoice: committed-use discounts, reservations and negotiated rates are not in them. Expect a steady gap against billing; the value is the relative picture. On-prem, bare metal and kind there is no price list to detect, and everything reads $0 until you give OpenCost a custom one.

How rightsizing decides

Rightsizing is three fixed rules, not a model, so the same data always gives the same answer. They apply to CPU and memory separately, over the P95 of the last 7 days of each Deployment (its ReplicaSets rolled up), StatefulSet or DaemonSet, sampled at 5-minute resolution:

StateWhen it firesWhat it suggestsSeverity
Near limitThe P95 reaches 80 % of the limit and sits within 50m CPU or 100Mi memory of itA limit of P95 × 1.5critical
Over-provisionedThe P95 is under half the request, and the gap is over 50m CPU or 100Mi memoryA request of P95 × 1.2warning
No specsNo request and no limit, and the P95 is above 50m CPU or 100Mi memorySetting theminfo

The absolute floors keep near-idle controllers quiet, where a percentage means nothing. Suggestions round to 10m and 10Mi. The result is recomputed at most every 5 minutes per cluster; every screen and Kobi read the same answer.

The P95 leaves out the busiest 5 % of the week, about 8 hours, so a short spike doesn’t move it. Use it for requests. For a memory limit, check the max on the workload’s Monitor tab too (Monitor charts on long ranges).

When recommendations appear

Rightsizing waits for 2 days of usage history by default. Below that the P95 has not seen two daily peaks and a quiet stretch reads as the steady state, so there are no recommendations at all:

From then on the recommendations are computed over the history there is; the Capacity card’s caption says how much (P95 over 2d). From 7 days the P95 covers the full week.

Self-hosted, an admin can change the minimum under Administration → System → General → Right-sizing → Minimum usage history: 12 to 168 hours, applied to every cluster and to Kobi with no restart. Lower it for a trial so a new cluster shows recommendations on its first day; below 48 hours they are marked as an early estimate. On Cloud we set the minimum; it is not an organization setting.

Three cases worth knowing:

Requirements

Three ways to connect OpenCost

Pick one under Cost monitoring (OpenCost) in the Add cluster wizard — it renders the final Helm values. The flags below go on the agent’s helm upgrade; on an existing release, add --reset-then-reuse-values to keep its backend URL, auth and cluster settings. See Connect clusters.

1. Install OpenCost for me (bundled) — the agent chart installs the official OpenCost sub-chart and scrapes it at http://<release>-opencost.<namespace>.svc:9003/metrics with no further setting. Simplest path if you don’t run OpenCost today. OpenCost itself reads usage from a Prometheus to attribute cost to workloads, so point the sub-chart at yours — the wizard asks for the URL and fills these in:

--set opencost.enabled=true \
--set opencost.opencost.prometheus.internal.serviceName=<prometheus-service> \
--set opencost.opencost.prometheus.internal.namespaceName=<prometheus-namespace> \
--set opencost.opencost.prometheus.internal.port=9090

For a Prometheus outside the cluster, use opencost.opencost.prometheus.internal.enabled=false, opencost.opencost.prometheus.external.enabled=true and opencost.opencost.prometheus.external.url=<url> instead. If the sub-chart’s Service name differs (a fullnameOverride), set opencost.exporterUrl.

Without a reachable Prometheus the sub-chart defaults to prometheus-server.prometheus-system and OpenCost fails to start where that Service doesn’t exist. If it reaches a Prometheus without the usage history it needs, node pricing still flows but the per-namespace and per-workload breakdown is partial: a total that looks right over a breakdown that does not add up to it.

2. Scrape my OpenCost — you already run OpenCost; the agent scrapes its exporter. List it under the agent’s exporters:

--set-string collectors.exporters.opencost=http://opencost.opencost.svc.cluster.local:9003/metrics

The scrape is leader-elected: one agent pod in the cluster does it, so the DaemonSet does not multiply the same samples by the number of nodes.

3. Read from my Prometheus — your existing Prometheus already scrapes OpenCost; the agent reads the cost metric families through the same promRead connection used for metrics:

--set agent.promRead.enabled=true \
--set agent.promRead.cost.enabled=true \
--set agent.promRead.url=http://prometheus.monitoring:9090

promRead is mutually exclusive with the scrape sidecar, so the wizard switches the cluster’s metrics source to it: one Prometheus feeds metrics and cost. In this mode the agent asks for the cost families listed below, appended to the core KubeBolt metrics and never replacing them. Before agent 1.4.10 it asked for eight of them, so volume claims, spot nodes, load balancers and network egress never arrived this way.

A Prometheus that scrapes OpenCost without honor_labels (a kube-prometheus ServiceMonitor’s default) puts the OpenCost pod’s namespace, pod, container and instance on every cost series and renames OpenCost’s own to exported_*. Read as it came, every container’s cost landed in the namespace OpenCost runs in. Since agent 1.4.10 promRead gives cost families their exported_* labels back; the cluster and tenant labels it stamps are never replaced.

One source of cost. With the exporter (modes 1 and 2) and agent.promRead.cost.enabled both on, every cost family was stored twice and every total read double. Since agent 1.4.10 the chart refuses that combination: helm upgrade fails with a message naming both. Pick one: promRead.cost reads OpenCost from your Prometheus, the exporter reads OpenCost directly.

What each path ships

One list of 13 cost families is what every path ships, asks for and keeps:

What it pricesFamilies
Nodesnode_total_hourly_cost, node_cpu_hourly_cost, node_ram_hourly_cost
Containerscontainer_cpu_allocation, container_memory_allocation_bytes
Volumespv_hourly_cost, pod_pvc_allocation
Spot nodeskubecost_node_is_spot
Networkkubecost_load_balancer_cost, kubecost_network_internet_egress_cost, kubecost_network_nat_gateway_egress_cost, kubecost_network_region_egress_cost, kubecost_network_zone_egress_cost

The GPU families stay out until a KubeBolt view reads them: on a node without a GPU they are a series per container that always reads zero. On KubeBolt Cloud the 13 families count as core series on every plan.

Verify

# 1. OpenCost is exporting
#    bundled: the Service is <release>-opencost in the agent's namespace
kubectl port-forward -n kubebolt svc/kubebolt-agent-opencost 9003:9003
#    your own: adjust namespace and Service, e.g. -n opencost svc/opencost
curl -s localhost:9003/metrics | grep node_total_hourly_cost

# 2. The agent is scraping it (the leader pod only)
kubectl logs -n kubebolt -l app.kubernetes.io/name=kubebolt-agent --tail=50 | grep -i exporter

Then the Cost tab appears on the cluster dashboard within a couple of minutes. The OpenCost card under Administration → Agents & Ingest → Integrations reports the same state, for the cluster selected in the top bar — it separates running from feeding by checking whether this cluster’s cost samples actually reached storage:

SymptomCause
Card “installed”, with “no cost samples reaching KubeBolt yet”OpenCost runs, but nothing ships its metrics — the exporter is not wired on the agent
Card “degraded” — “OpenCost pods not fully ready”Often the bundled sub-chart crash-looping because it has no reachable Prometheus
Card “not installed”No pods labelled app.kubernetes.io/name=opencost, and no pod named opencost* in the opencost, kubecost or monitoring namespaces
Total looks right, breakdown is empty or partialOpenCost’s Prometheus lacks the usage history for allocation
Everything at $0OpenCost has no price list for the provider — on-prem and kind need a custom one
Network trend emptyMode 3 on an agent before 1.4.10, which does not read the network families, or the cluster has no priced load balancers or egress
Every total reads about doubleThe exporter and promRead.cost both on, on an agent before 1.4.10: turn one off
Every container’s cost sits in OpenCost’s namespaceMode 3 on an agent before 1.4.10, reading a Prometheus that scrapes OpenCost without honor_labels: upgrade the agent
The agent’s helm upgrade fails naming agent.promRead.cost.enabled and the OpenCost exporterTwo sources of cost: turn one off. Until then the release stays on its previous version

Cost samples are ordinary metrics: they follow the retention of KubeBolt’s metrics store (30 days on the bundled VictoriaMetrics, metrics.storage.embedded.retention in the kubebolt chart), so history goes back as far as that.