Deployment Guides
Google GKE Guide
Deploy KubeBolt on Google Kubernetes Engine, including GCE Ingress, Autopilot and Google Managed Prometheus.
On this page
On KubeBolt Cloud the only thing that runs in your GKE cluster is the agent, and the Add cluster wizard prints its command. Of this page, Google Managed Prometheus is the part that may apply to you; the rest is for running KubeBolt yourself.
Quick install
The chart works out of the box on GKE — it creates a ServiceAccount and a ClusterRole granting KubeBolt read access to the cluster plus the write verbs its actions use (exec, port-forward, scale, restart, edit, delete).
helm install kubebolt oci://ghcr.io/clm-cloud-solutions/kubebolt/helm/kubebolt \
--namespace kubebolt --create-namespace --wait
kubectl -n kubebolt get secret kubebolt-admin-password \
-o jsonpath='{.data.password}' | base64 -d ; echo
kubectl -n kubebolt port-forward svc/kubebolt 3000:80
Open http://localhost:3000 and log in as admin with the printed password
(other ways to get it: Quick Start).
GKE ships Metrics Server, so live CPU and memory bars work immediately.
Authentication
KubeBolt itself needs no GCP permissions — it talks to the Kubernetes API and nothing else. Workload Identity matters only when the ServiceAccount also has to reach a Google Cloud service, which for KubeBolt means one case: reading Google Managed Prometheus with the agent, below.
Ingress with GCE
GKE’s Ingress controller ignores ingressClassName: it picks up an Ingress
by the kubernetes.io/ingress.class annotation, and an Ingress that sets any
ingressClassName without the annotation gets no load balancer. So leave
ingress.className empty (the API server rejects an Ingress that sets both)
and set the annotation, which is easier in a values file than through --set:
cat > kubebolt-gce.yaml <<'EOF'
ingress:
enabled: true
className: ""
annotations:
kubernetes.io/ingress.class: gce # gce-internal for an internal load balancer
hosts:
- host: kubebolt.example.com
paths:
- path: /
pathType: Prefix
EOF
helm upgrade --install kubebolt \
oci://ghcr.io/clm-cloud-solutions/kubebolt/helm/kubebolt \
--namespace kubebolt --create-namespace -f kubebolt-gce.yaml
kubectl -n kubebolt get ingress kubebolt # ADDRESS is the load balancer's IP
gce-internal also needs a proxy-only subnet in the cluster’s region. The
chart’s Services are ClusterIP, which GKE serves through container-native
load balancing — on by default on VPC-native clusters under most conditions;
where it isn’t, annotate the Service with
cloud.google.com/neg='{"ingress": true}'. On a routes-based cluster, add
--set service.type=NodePort instead.
Raise the backend timeout. The load balancer waits 30 s by default for a
complete response, and Kobi streams each answer as one long response, so
answers that take longer are cut. The chart has no value for Service
annotations; attach a BackendConfig to the kubebolt Service yourself, as
below. Helm leaves an annotation it doesn’t manage in place on upgrade.
kubectl -n kubebolt apply -f - <<'EOF'
apiVersion: cloud.google.com/v1
kind: BackendConfig
metadata:
name: kubebolt
spec:
timeoutSec: 600
EOF
kubectl -n kubebolt annotate service kubebolt \
cloud.google.com/backend-config='{"default": "kubebolt"}'
The Ingress fronts the UI and API (the web Service proxies /api and /ws).
Agents in other clusters dial the gRPC port, kubebolt-agent-ingest on 9090,
which you expose separately — see
The agent cannot reach the backend.
Instead of an Ingress, the chart can create a Gateway API Gateway with an
HTTPRoute for the UI and a GRPCRoute for agents (gatewayAPI.enabled=true,
mutually exclusive with ingress). Its defaults target Envoy Gateway
(GatewayClass eg, plus an Envoy ClientTrafficPolicy); with another
controller set gatewayAPI.gatewayClassName and gatewayAPI.forceHTTP1=false.
Pass web.enabled=true too, or the route skips the web UI and points at the
API alone. The chart README
has the full command.
GKE Autopilot
The kubebolt chart needs no privileged containers, so it installs on
Autopilot clusters as is. Autopilot may raise the chart’s small default
requests to meet its minimums — that is expected.
helm install kubebolt \
oci://ghcr.io/clm-cloud-solutions/kubebolt/helm/kubebolt \
--namespace kubebolt --create-namespace
Where Autopilot doesn’t allow bursting it sets limits equal to requests. The
API caches every resource it watches, Events included, which is why its
default memory limit is 1Gi against a 128Mi request; on such a cluster, raise
api.resources.requests.memory to what the cluster needs.
Google Managed Prometheus
If your metrics already live in GMP, you do not need a second scrape pipeline. Deploy the optional agent in promread mode and a separate single-replica Deployment queries GMP’s Prometheus-compatible API directly, authenticating with GCP IAM through Workload Identity — no service-account key file.
1. Find the endpoint
GMP’s query endpoint is global per project; the location/global segment is
fixed and there is no region to pass:
PROJECT_ID=$(gcloud config get-value project)
GMP_ENDPOINT="https://monitoring.googleapis.com/v1/projects/${PROJECT_ID}/location/global/prometheus"
TOKEN=$(gcloud auth print-access-token)
curl -s -H "Authorization: Bearer $TOKEN" \
"${GMP_ENDPOINT}/api/v1/query?query=count(up)"
A status: success response means the endpoint and the bearer-auth path work.
2. Create the Google service account and bind it
GSA=kubebolt-promread
NAMESPACE=kubebolt-system # where the agent goes; the Add cluster wizard's default
KSA=kubebolt-agent # the chart's ServiceAccount for a release named kubebolt-agent
gcloud iam service-accounts create $GSA \
--display-name="KubeBolt agent — reads from Google Managed Prometheus"
# monitoring.viewer is read-only: no ingest, no admin.
gcloud projects add-iam-policy-binding $PROJECT_ID \
--member="serviceAccount:${GSA}@${PROJECT_ID}.iam.gserviceaccount.com" \
--role="roles/monitoring.viewer" \
--condition=None
# The Kubernetes ServiceAccount does not exist yet — helm creates it in
# step 3. GKE resolves this binding lazily when it appears.
gcloud iam service-accounts add-iam-policy-binding \
${GSA}@${PROJECT_ID}.iam.gserviceaccount.com \
--role="roles/iam.workloadIdentityUser" \
--member="serviceAccount:${PROJECT_ID}.svc.id.goog[${NAMESPACE}/${KSA}]"
The namespace and ServiceAccount in that member must be the ones the agent
runs under. If you install it elsewhere, change NAMESPACE before running
this.
Workload Identity has to be on at both layers on a Standard cluster
(Autopilot always has it). Enabling it at the cluster scope
(--workload-pool) is not enough — every node pool running the agent also
needs --workload-metadata=GKE_METADATA. Without it the pod silently
inherits the node’s Compute Engine service account and you get
403 PERMISSION_DENIED on monitoring.timeSeries.list even though the binding
above is correct. Check both layers with the commands below.
CLUSTER=my-gke-cluster
LOCATION=us-central1 # the cluster's region or zone
gcloud container clusters describe $CLUSTER --location=$LOCATION \
--format='value(workloadIdentityConfig.workloadPool)' # PROJECT_ID.svc.id.goog
gcloud container node-pools list --cluster=$CLUSTER --location=$LOCATION \
--format='table(name,config.workloadMetadataConfig.mode)' # GKE_METADATA on each
3. Install the agent in promread mode
The kubebolt-agent-token Secret carries the ingest token (Administration →
Agents & Ingest → Agent Tokens, or the wizard), under the token key:
BACKEND_URL=agent.kubebolt.io:443 # KubeBolt Cloud; self-hosted: your agent gRPC host:port
CLUSTER_NAME=my-gke-cluster
INGEST_TOKEN='paste-the-token-here'
kubectl create namespace "$NAMESPACE" --dry-run=client -o yaml | kubectl apply -f -
kubectl -n "$NAMESPACE" create secret generic kubebolt-agent-token \
--from-literal=token="$INGEST_TOKEN" --dry-run=client -o yaml | kubectl apply -f -
helm upgrade --install kubebolt-agent \
oci://ghcr.io/clm-cloud-solutions/kubebolt/helm/kubebolt-agent \
--namespace "$NAMESPACE" \
--set backendUrl="$BACKEND_URL" --set tls.enabled=true \
--set cluster.name="$CLUSTER_NAME" \
--set auth.mode=ingest-token \
--set auth.ingestToken.existingSecret=kubebolt-agent-token \
--set scrape.enabled=false \
--set agent.promRead.enabled=true \
--set agent.promRead.url="${GMP_ENDPOINT}" \
--set agent.promRead.auth.mode=gcpIam \
--set serviceAccount.annotations."iam\.gke\.io/gcp-service-account"="${GSA}@${PROJECT_ID}.iam.gserviceaccount.com"
Notes that catch people out:
backendUrlis a gRPC dial target,host:port, no scheme. For an agent in the same cluster as a self-hosted backend, use the in-cluster Service (kubebolt-agent-ingest.kubebolt.svc.cluster.local:9090) without TLS.agent.promRead.urlis the base endpoint — the agent appends/api/v1/query_rangeitself.scrape.enabled=falsekeeps the bundledvmagentsidecar off (it and promread are mutually exclusive). It does not disable the DaemonSet’s kubelet collectors, which still complement GMP with KubeBolt’s canonical metrics.- The default matchers use explicit metric names rather than a regex on
__name__, because GMP rejects{__name__=~"…"}selectors. Keep any override as an explicit name list. - Do not set
agent.deferNodeStress=trueon GMP. GMP managed collection does not scrape node-exporter, so the agent’s own collector is the only source ofnode_load1/5/15and the pressure series. Deferring it empties the Load average panel. (That flag is the right call on Azure, not here.)
The Add cluster wizard builds this command too: in its advanced options, pick
Read from existing Prometheus under Prometheus integration, auth GCP
IAM, and add the iam.gke.io/gcp-service-account ServiceAccount annotation.
See Connecting clusters and Metrics.
4. Verify
kubectl -n "$NAMESPACE" get pods -l kubebolt.dev/role=promread # 1 pod, Running
kubectl -n "$NAMESPACE" get lease kubebolt-promread # HOLDER is that pod
kubectl -n "$NAMESPACE" logs -l kubebolt.dev/role=promread --tail=50 \
| grep -E "promread enabled|channel registered|matcher failed"
You want promread enabled … auth_mode=gcpIam and channel registered.
promread matcher failed with 403 PERMISSION_DENIED means Workload Identity
isn’t reaching the pod (see the two layers above); a 400 mentioning
__name__ means a custom matcher used a regex. In KubeBolt, the Prometheus
(read) card in Administration → Agents & Ingest → Integrations turns
Installed after the first poll.
Troubleshooting
API pod in CrashLoopBackOff. Check kubectl -n kubebolt logs -l app.kubernetes.io/component=api.
The usual cause is a missing ClusterRoleBinding — verify with
kubectl get clusterrolebinding | grep kubebolt.
Binary Authorization blocks the images. Add an exemption for
ghcr.io/clm-cloud-solutions/kubebolt/*, or attest the images under your own
policy.
403 on some resources. Expected, and handled: KubeBolt degrades gracefully. Restricted resources are dimmed in the sidebar behind a “Limited access” banner. See RBAC & Permissions.