Deployment Guides
Amazon EKS Guide
Run KubeBolt against Amazon EKS — in-cluster with Helm, from your machine, or with the agent — including ALB exposure, IRSA and reading Amazon Managed Prometheus.
On this page
On KubeBolt Cloud the only thing that runs in your EKS cluster is the agent, and the Add cluster wizard prints its command. Of this page, Metrics Server and Amazon Managed Prometheus apply to you; the rest is for running KubeBolt yourself.
Quick install
The chart creates two PVCs (10 GiB for VictoriaMetrics, 1 GiB for KubeBolt’s
own data) on the default StorageClass. On EKS that needs the Amazon EBS
CSI driver add-on (and its IAM role), and a StorageClass marked
(default): clusters created on Kubernetes 1.30 or later no longer mark
gp2 as default. Check with kubectl get storageclass. Without both, the
PVCs stay Pending and the pods never start. To name a class instead, set
metrics.storage.embedded.persistence.storageClass and
auth.persistence.storageClass.
helm install kubebolt oci://ghcr.io/clm-cloud-solutions/kubebolt/helm/kubebolt \
--namespace kubebolt --create-namespace --wait
kubectl -n kubebolt get secret kubebolt-admin-password \
-o jsonpath='{.data.password}' | base64 -d ; echo
kubectl -n kubebolt port-forward svc/kubebolt 3000:80
Open http://localhost:3000 and log in as admin with the printed password
(other ways to get it: Quick Start).
Authentication
How KubeBolt authenticates to EKS depends on where it runs:
- In the cluster (Helm) — the API uses its own ServiceAccount and the chart’s ClusterRole. No AWS credentials are involved.
- On your machine (Homebrew, krew, single binary) — it uses your kubeconfig
like
kubectldoes, so theaws eks get-tokenexec in an EKS kubeconfig needs the AWS CLI and an active profile or SSO session on the host. - Single-container image — the image ships
aws-cli. Mount your kubeconfig and your AWS config into the non-root user’s home:
docker run -p 3000:3000 \
-v ~/.kube/config:/kubeconfig:ro -e KUBECONFIG=/kubeconfig \
-v ~/.aws:/home/kubebolt/.aws:ro -e AWS_PROFILE=my-profile \
-v kubebolt-data:/data \
ghcr.io/clm-cloud-solutions/kubebolt:latest
Importing a kubeconfig doesn’t help either. A Helm install refuses Import
kubeconfig outright (cannot add clusters in in-cluster mode), and the API
image that Compose runs has no AWS CLI, so an EKS kubeconfig imported there
fails on the same aws eks get-token. Connect those clusters with the agent
instead.
IRSA or EKS Pod Identity is never needed to read the cluster. It comes in at two places:
-
The
kubeboltchart’s ServiceAccount (kubeboltfor a release namedkubebolt): the AWS identity that Kobi’s AWS connector starts from on a self-hosted install. SetmcpConnectors.aws.principalArnto the same role so the trust policy the connector’s setup guide prints names it:helm upgrade --install kubebolt \ oci://ghcr.io/clm-cloud-solutions/kubebolt/helm/kubebolt \ --namespace kubebolt --create-namespace \ --set serviceAccount.annotations."eks\.amazonaws\.com/role-arn"=arn:aws:iam::123456789012:role/kubebolt-api \ --set mcpConnectors.aws.principalArn=arn:aws:iam::123456789012:role/kubebolt-apiWith Pod Identity, drop the annotation and associate the role with ServiceAccount
kubeboltin namespacekubebolt. On Cloud the connector assumes your role from KubeBolt’s side; nothing goes in the cluster. -
The agent’s ServiceAccount, which signs Amazon Managed Prometheus queries — below.
Docker Compose
The compose file mounts ~/.aws for EKS token generation, but the API image it
builds does not include the AWS CLI, so a kubeconfig that runs
aws eks get-token fails inside the container. Run KubeBolt on the host
(Homebrew or the binary), use the single-container image above, or give Compose
a kubeconfig with a static token (for example a ServiceAccount token). The
Compose steps themselves:
git clone https://github.com/clm-cloud-solutions/kubebolt.git
cd kubebolt
kubectl config use-context my-eks-cluster
./deploy/docker-kubeconfig.sh
cd deploy && docker compose up -d
The helper is not an EKS step: compose mounts /tmp/docker-kubeconfig, so it
runs on every install. On a routable endpoint it rewrites nothing. Full Compose
reference in Installation methods.
Helm with ALB Ingress
This needs the AWS Load Balancer Controller
and the storage from Quick install. The chart’s Services are
ClusterIP, so the ALB has to target pod IPs (target-type: ip; the
controller’s default, instance, needs a NodePort). Its default scheme is
internal. Annotations are easier in a values file than through --set:
cat > kubebolt-alb.yaml <<'EOF'
ingress:
enabled: true
className: alb
annotations:
alb.ingress.kubernetes.io/scheme: internet-facing # or internal
alb.ingress.kubernetes.io/target-type: ip
# HTTPS with an ACM certificate:
# alb.ingress.kubernetes.io/listen-ports: '[{"HTTPS":443}]'
# alb.ingress.kubernetes.io/certificate-arn: arn:aws:acm:REGION:ACCOUNT:certificate/ID
hosts:
- host: kubebolt.example.com
paths:
- path: /
pathType: Prefix
EOF
helm upgrade --install kubebolt \
oci://ghcr.io/clm-cloud-solutions/kubebolt/helm/kubebolt \
--namespace kubebolt --create-namespace -f kubebolt-alb.yaml
kubectl -n kubebolt get ingress kubebolt # ADDRESS is the ALB's DNS name
Point kubebolt.example.com at that address. The Ingress fronts the UI and
API only — it routes to the web Service, which proxies /api and /ws. Agents
in other clusters dial the gRPC port (kubebolt-agent-ingest, 9090), which you
expose separately: a LoadBalancer Service (agentIngest.service.type) or a
gRPC-capable proxy, with agentIngest.tls.* for TLS. See
The agent cannot reach the backend.
KubeBolt itself needs no DaemonSet, so the kubeconfig and in-cluster modes
work on EKS Fargate. Fargate pods can’t mount EBS, so point both PVCs at an
EFS-backed StorageClass (metrics.storage.embedded.persistence.storageClass,
auth.persistence.storageClass), or run KubeBolt on a managed node group. The
optional agent’s DaemonSet does not run on Fargate nodes; its promread
Deployment does.
Metrics Server
EKS does not always include Metrics Server. Without it, KubeBolt still shows every resource, event and insight; only the live CPU/memory bars read “no data”. Check, and install it if the API service is missing:
kubectl get apiservice v1beta1.metrics.k8s.io
# As an EKS add-on
aws eks create-addon --cluster-name my-eks-cluster --addon-name metrics-server
# Or from the upstream manifest
kubectl apply -f https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/components.yaml
Amazon Managed Prometheus (AMP)
If your metrics already live in AMP, you don’t need a second pipeline: deploy
the agent with agent.promRead.enabled=true and a separate
single-replica Deployment reads kube-state-metrics, node and target-health
families (and OpenCost’s, with agent.promRead.cost.enabled=true) straight
from your workspace, signing each query with AWS SigV4. AMP only stores what
something pushes to it with remote_write; the agent reads it back.
1. Find the endpoint
AMP endpoints are per workspace and per region:
REGION=us-east-1
# The first workspace in the region; pick yours if there are several
WORKSPACE_ID=$(aws amp list-workspaces --region "$REGION" \
--query 'workspaces[0].workspaceId' --output text)
AMP_ENDPOINT=$(aws amp describe-workspace --workspace-id "$WORKSPACE_ID" \
--region "$REGION" --query 'workspace.prometheusEndpoint' --output text)
echo "$AMP_ENDPOINT" # https://aps-workspaces.<region>.amazonaws.com/workspaces/<id>/
To smoke-test it from your machine, awscurl
signs the request: awscurl --service aps --region "$REGION" "${AMP_ENDPOINT}api/v1/query?query=up"
should return "status":"success". An empty result is normal on a workspace
nothing pushes to yet.
2. Create the IAM role (IRSA)
Create a role the agent’s ServiceAccount can assume, with the AWS-managed
AmazonPrometheusQueryAccess policy — query only, no ingest or admin:
CLUSTER=my-eks-cluster
CLUSTER_REGION=$REGION # change it if the cluster and the workspace are in different regions
NAMESPACE=kubebolt-system # where the agent goes; the Add cluster wizard's default
KSA=kubebolt-agent # the chart's ServiceAccount for a release named kubebolt-agent
# IRSA needs the cluster's OIDC provider in IAM (a no-op if it is already there)
eksctl utils associate-iam-oidc-provider --cluster "$CLUSTER" --region "$CLUSTER_REGION" --approve
eksctl create iamserviceaccount \
--cluster "$CLUSTER" --region "$CLUSTER_REGION" \
--namespace "$NAMESPACE" --name "$KSA" \
--role-name kubebolt-promread --role-only \
--attach-policy-arn arn:aws:iam::aws:policy/AmazonPrometheusQueryAccess \
--approve
ROLE_ARN=$(aws iam get-role --role-name kubebolt-promread --query 'Role.Arn' --output text)
--role-only creates just the role; the chart creates the ServiceAccount and
annotates it in the next step. The agent signs with the AWS SDK’s default
credential chain, so an EKS Pod Identity association for the same
ServiceAccount works too; then leave the annotation out.
3. Install the agent in promread mode
The kubebolt-agent-token Secret carries the ingest token (Administration →
Agents & Ingest → Agent Tokens, or the wizard), under the token key:
BACKEND_URL=agent.kubebolt.io:443 # KubeBolt Cloud; self-hosted: your agent gRPC host:port
CLUSTER_NAME=my-eks-cluster
INGEST_TOKEN='paste-the-token-here'
kubectl create namespace "$NAMESPACE" --dry-run=client -o yaml | kubectl apply -f -
kubectl -n "$NAMESPACE" create secret generic kubebolt-agent-token \
--from-literal=token="$INGEST_TOKEN" --dry-run=client -o yaml | kubectl apply -f -
helm upgrade --install kubebolt-agent \
oci://ghcr.io/clm-cloud-solutions/kubebolt/helm/kubebolt-agent \
--namespace "$NAMESPACE" \
--set backendUrl="$BACKEND_URL" --set tls.enabled=true \
--set cluster.name="$CLUSTER_NAME" \
--set auth.mode=ingest-token \
--set auth.ingestToken.existingSecret=kubebolt-agent-token \
--set scrape.enabled=false \
--set agent.promRead.enabled=true \
--set agent.promRead.url="$AMP_ENDPOINT" \
--set agent.promRead.auth.mode=awsSigV4 \
--set agent.promRead.auth.awsRegion="$REGION" \
--set serviceAccount.annotations."eks\.amazonaws\.com/role-arn"="$ROLE_ARN"
Notes that catch people out:
agent.promRead.auth.awsRegionis required withawsSigV4: the chart refuses to render without it, and a region that differs from the workspace’s fails every query withInvalidSignatureException.backendUrlis a gRPC dial target,host:port, no scheme. For an agent in the same cluster as a self-hosted backend, use the in-cluster Service (kubebolt-agent-ingest.kubebolt.svc.cluster.local:9090) without TLS.agent.promRead.urlis the base endpoint — the agent appends/api/v1/query_rangeitself. The trailing slash is fine.scrape.enabled=falsekeeps the bundledvmagentsidecar off (it and promread are mutually exclusive). The DaemonSet’s kubelet collectors keep running and complement AMP with KubeBolt’s own metrics.- If what pushes to AMP includes node-exporter, add
--set agent.deferNodeStress=true, or the Load average panel draws each series twice. - Egress: the agent needs
sts.<region>.amazonaws.comandaps-workspaces.<region>.amazonaws.comon 443, plus your backend.
The Add cluster wizard builds this command too: in its advanced options, pick Read from existing Prometheus under Prometheus integration, auth AWS SigV4, and add the role ARN as a ServiceAccount annotation. See Connecting clusters and Metrics.
4. Verify
kubectl -n "$NAMESPACE" get pods -l kubebolt.dev/role=promread # 1 pod, Running
kubectl -n "$NAMESPACE" get lease kubebolt-promread # HOLDER is that pod
kubectl -n "$NAMESPACE" logs -l kubebolt.dev/role=promread --tail=50 \
| grep -E "promread enabled|channel registered|matcher failed"
You want promread enabled … auth_mode=awsSigV4 and channel registered.
promread matcher failed with AccessDenied means the role isn’t reaching the
pod (annotation, trust policy or policy); with InvalidSignatureException, the
region is wrong. No channel registered line means the backend rejected the
agent: check the Secret and auth.mode. In KubeBolt, the Prometheus (read)
card in Administration → Agents & Ingest → Integrations turns Installed after
the first poll.