KubeBolt docs
ENES GitHub

Deployment Guides

Google GKE Guide

Deploy KubeBolt on Google Kubernetes Engine, including GCE Ingress, Autopilot and Google Managed Prometheus.

On this page
  1. Quick install
  2. Authentication
  3. Ingress with GCE
  4. GKE Autopilot
  5. Google Managed Prometheus
  6. Troubleshooting

On KubeBolt Cloud the only thing that runs in your GKE cluster is the agent, and the Add cluster wizard prints its command. Of this page, Google Managed Prometheus is the part that may apply to you; the rest is for running KubeBolt yourself.

Quick install

The chart works out of the box on GKE — it creates a ServiceAccount and a ClusterRole granting KubeBolt read access to the cluster plus the write verbs its actions use (exec, port-forward, scale, restart, edit, delete).

helm install kubebolt oci://ghcr.io/clm-cloud-solutions/kubebolt/helm/kubebolt \
  --namespace kubebolt --create-namespace --wait

kubectl -n kubebolt get secret kubebolt-admin-password \
  -o jsonpath='{.data.password}' | base64 -d ; echo

kubectl -n kubebolt port-forward svc/kubebolt 3000:80

Open http://localhost:3000 and log in as admin with the printed password (other ways to get it: Quick Start).

GKE ships Metrics Server, so live CPU and memory bars work immediately.

Authentication

KubeBolt itself needs no GCP permissions — it talks to the Kubernetes API and nothing else. Workload Identity matters only when the ServiceAccount also has to reach a Google Cloud service, which for KubeBolt means one case: reading Google Managed Prometheus with the agent, below.

Ingress with GCE

GKE’s Ingress controller ignores ingressClassName: it picks up an Ingress by the kubernetes.io/ingress.class annotation, and an Ingress that sets any ingressClassName without the annotation gets no load balancer. So leave ingress.className empty (the API server rejects an Ingress that sets both) and set the annotation, which is easier in a values file than through --set:

cat > kubebolt-gce.yaml <<'EOF'
ingress:
  enabled: true
  className: ""
  annotations:
    kubernetes.io/ingress.class: gce   # gce-internal for an internal load balancer
  hosts:
    - host: kubebolt.example.com
      paths:
        - path: /
          pathType: Prefix
EOF

helm upgrade --install kubebolt \
  oci://ghcr.io/clm-cloud-solutions/kubebolt/helm/kubebolt \
  --namespace kubebolt --create-namespace -f kubebolt-gce.yaml

kubectl -n kubebolt get ingress kubebolt   # ADDRESS is the load balancer's IP

gce-internal also needs a proxy-only subnet in the cluster’s region. The chart’s Services are ClusterIP, which GKE serves through container-native load balancing — on by default on VPC-native clusters under most conditions; where it isn’t, annotate the Service with cloud.google.com/neg='{"ingress": true}'. On a routes-based cluster, add --set service.type=NodePort instead.

Raise the backend timeout. The load balancer waits 30 s by default for a complete response, and Kobi streams each answer as one long response, so answers that take longer are cut. The chart has no value for Service annotations; attach a BackendConfig to the kubebolt Service yourself, as below. Helm leaves an annotation it doesn’t manage in place on upgrade.

kubectl -n kubebolt apply -f - <<'EOF'
apiVersion: cloud.google.com/v1
kind: BackendConfig
metadata:
  name: kubebolt
spec:
  timeoutSec: 600
EOF
kubectl -n kubebolt annotate service kubebolt \
  cloud.google.com/backend-config='{"default": "kubebolt"}'

The Ingress fronts the UI and API (the web Service proxies /api and /ws). Agents in other clusters dial the gRPC port, kubebolt-agent-ingest on 9090, which you expose separately — see The agent cannot reach the backend.

Instead of an Ingress, the chart can create a Gateway API Gateway with an HTTPRoute for the UI and a GRPCRoute for agents (gatewayAPI.enabled=true, mutually exclusive with ingress). Its defaults target Envoy Gateway (GatewayClass eg, plus an Envoy ClientTrafficPolicy); with another controller set gatewayAPI.gatewayClassName and gatewayAPI.forceHTTP1=false. Pass web.enabled=true too, or the route skips the web UI and points at the API alone. The chart README has the full command.

GKE Autopilot

The kubebolt chart needs no privileged containers, so it installs on Autopilot clusters as is. Autopilot may raise the chart’s small default requests to meet its minimums — that is expected.

helm install kubebolt \
  oci://ghcr.io/clm-cloud-solutions/kubebolt/helm/kubebolt \
  --namespace kubebolt --create-namespace

Where Autopilot doesn’t allow bursting it sets limits equal to requests. The API caches every resource it watches, Events included, which is why its default memory limit is 1Gi against a 128Mi request; on such a cluster, raise api.resources.requests.memory to what the cluster needs.

Google Managed Prometheus

If your metrics already live in GMP, you do not need a second scrape pipeline. Deploy the optional agent in promread mode and a separate single-replica Deployment queries GMP’s Prometheus-compatible API directly, authenticating with GCP IAM through Workload Identity — no service-account key file.

1. Find the endpoint

GMP’s query endpoint is global per project; the location/global segment is fixed and there is no region to pass:

PROJECT_ID=$(gcloud config get-value project)
GMP_ENDPOINT="https://monitoring.googleapis.com/v1/projects/${PROJECT_ID}/location/global/prometheus"

TOKEN=$(gcloud auth print-access-token)
curl -s -H "Authorization: Bearer $TOKEN" \
  "${GMP_ENDPOINT}/api/v1/query?query=count(up)"

A status: success response means the endpoint and the bearer-auth path work.

2. Create the Google service account and bind it

GSA=kubebolt-promread
NAMESPACE=kubebolt-system   # where the agent goes; the Add cluster wizard's default
KSA=kubebolt-agent          # the chart's ServiceAccount for a release named kubebolt-agent

gcloud iam service-accounts create $GSA \
  --display-name="KubeBolt agent — reads from Google Managed Prometheus"

# monitoring.viewer is read-only: no ingest, no admin.
gcloud projects add-iam-policy-binding $PROJECT_ID \
  --member="serviceAccount:${GSA}@${PROJECT_ID}.iam.gserviceaccount.com" \
  --role="roles/monitoring.viewer" \
  --condition=None

# The Kubernetes ServiceAccount does not exist yet — helm creates it in
# step 3. GKE resolves this binding lazily when it appears.
gcloud iam service-accounts add-iam-policy-binding \
  ${GSA}@${PROJECT_ID}.iam.gserviceaccount.com \
  --role="roles/iam.workloadIdentityUser" \
  --member="serviceAccount:${PROJECT_ID}.svc.id.goog[${NAMESPACE}/${KSA}]"

The namespace and ServiceAccount in that member must be the ones the agent runs under. If you install it elsewhere, change NAMESPACE before running this.

Workload Identity has to be on at both layers on a Standard cluster (Autopilot always has it). Enabling it at the cluster scope (--workload-pool) is not enough — every node pool running the agent also needs --workload-metadata=GKE_METADATA. Without it the pod silently inherits the node’s Compute Engine service account and you get 403 PERMISSION_DENIED on monitoring.timeSeries.list even though the binding above is correct. Check both layers with the commands below.

CLUSTER=my-gke-cluster
LOCATION=us-central1   # the cluster's region or zone
gcloud container clusters describe $CLUSTER --location=$LOCATION \
  --format='value(workloadIdentityConfig.workloadPool)'   # PROJECT_ID.svc.id.goog
gcloud container node-pools list --cluster=$CLUSTER --location=$LOCATION \
  --format='table(name,config.workloadMetadataConfig.mode)'   # GKE_METADATA on each

3. Install the agent in promread mode

The kubebolt-agent-token Secret carries the ingest token (Administration → Agents & Ingest → Agent Tokens, or the wizard), under the token key:

BACKEND_URL=agent.kubebolt.io:443   # KubeBolt Cloud; self-hosted: your agent gRPC host:port
CLUSTER_NAME=my-gke-cluster
INGEST_TOKEN='paste-the-token-here'

kubectl create namespace "$NAMESPACE" --dry-run=client -o yaml | kubectl apply -f -
kubectl -n "$NAMESPACE" create secret generic kubebolt-agent-token \
  --from-literal=token="$INGEST_TOKEN" --dry-run=client -o yaml | kubectl apply -f -

helm upgrade --install kubebolt-agent \
  oci://ghcr.io/clm-cloud-solutions/kubebolt/helm/kubebolt-agent \
  --namespace "$NAMESPACE" \
  --set backendUrl="$BACKEND_URL" --set tls.enabled=true \
  --set cluster.name="$CLUSTER_NAME" \
  --set auth.mode=ingest-token \
  --set auth.ingestToken.existingSecret=kubebolt-agent-token \
  --set scrape.enabled=false \
  --set agent.promRead.enabled=true \
  --set agent.promRead.url="${GMP_ENDPOINT}" \
  --set agent.promRead.auth.mode=gcpIam \
  --set serviceAccount.annotations."iam\.gke\.io/gcp-service-account"="${GSA}@${PROJECT_ID}.iam.gserviceaccount.com"

Notes that catch people out:

The Add cluster wizard builds this command too: in its advanced options, pick Read from existing Prometheus under Prometheus integration, auth GCP IAM, and add the iam.gke.io/gcp-service-account ServiceAccount annotation. See Connecting clusters and Metrics.

4. Verify

kubectl -n "$NAMESPACE" get pods -l kubebolt.dev/role=promread   # 1 pod, Running
kubectl -n "$NAMESPACE" get lease kubebolt-promread              # HOLDER is that pod
kubectl -n "$NAMESPACE" logs -l kubebolt.dev/role=promread --tail=50 \
  | grep -E "promread enabled|channel registered|matcher failed"

You want promread enabled … auth_mode=gcpIam and channel registered. promread matcher failed with 403 PERMISSION_DENIED means Workload Identity isn’t reaching the pod (see the two layers above); a 400 mentioning __name__ means a custom matcher used a regex. In KubeBolt, the Prometheus (read) card in Administration → Agents & Ingest → Integrations turns Installed after the first poll.

Troubleshooting

API pod in CrashLoopBackOff. Check kubectl -n kubebolt logs -l app.kubernetes.io/component=api. The usual cause is a missing ClusterRoleBinding — verify with kubectl get clusterrolebinding | grep kubebolt.

Binary Authorization blocks the images. Add an exemption for ghcr.io/clm-cloud-solutions/kubebolt/*, or attest the images under your own policy.

403 on some resources. Expected, and handled: KubeBolt degrades gracefully. Restricted resources are dimmed in the sidebar behind a “Limited access” banner. See RBAC & Permissions.