Hands-on Workshop
Collecting logs and generating dashboards is easy. Catching a fault while the dashboard is still green, fast and without a flood of false positives, isn't.
That knowledge lives with the people who run production, so we work it out together, in the open. Start with the free, open-source workshop and see what the agent catches that your dashboards miss.
demo-cluster — bash
Architecture
Everything in the repo on one page, each part tagged with its folder.
Watch every detector and every faultBreak it on purpose
Start with a script, stop with a flag.
OOMKill
payments-api
The crash everyone sees
Slow memory leak
recommend-svc sidecar
Running, until it is not
ConfigMap change
one kubectl patch
No crash, no event
Your desktop · kind cluster · Docker
demo-apps
metrics, pod status, events, ConfigMaps
monitoring
Prometheus
Scrapes every 5s
Grafana - golden signals
Sees the leak climb
Alertmanager
Fires on the OOMKill in seconds
k8sgpt-system
K8sGPT operator ~30s scans
Silent until the pod crash-loops
k8s-watcher - Go, ConfigMap lane
Catches the ConfigMap change
Results
results.core.k8sgpt.ai
Optional. Off by default.
Add an LLM
Explains each Result in plain text
Local models: nothing leaves
Add Slack
A webhook you control
slack-relay posts each new Result
One post per fault, not per retry
Without them, Results stay in the cluster. Read them with kubectl.
DataAgent Lab
Detect and diagnose, on your own desktop.
Free, open source, Apache 2.0. No account, no agent in your environment.
DataAgent in production
Verified remediation, then the RCA.
Kubernetes and AWS, next to the tools you already run.
What it prints
This is what lands in your terminal, your Slack channel and your dashboard when you run the scenarios.
kubectl
spec.details lists the changed keys with digests of their values, never the values.
Slack
slack-relay3:02 AM
DemoContainerOOMKilled
Container payments-api in demo-apps was OOMKilled.
The container went past its memory limit and the kernel killed it; the kubelet restarts it.
Alertmanager posts first, within seconds. K8sGPT's Result follows a minute or two later. Layout simplified.
Grafana
Illustration. The Composite Health Score drops 20 points at 50% of the limit and 30 more at 80%, while K8sGPT is still silent.
Watch how it catches slow leaksQuickstart
quickstart
View on GitHub
Watch the walkthrough
git clone https://github.com/data-agent-co/autonomous-sre-training.git
cd autonomous-sre-training/demo-cluster
./scripts/up.sh
./scripts/trigger-oomkill.sh
kubectl --context kind-demo-cluster -n k8sgpt-system get results
open http://localhost:3000/d/demo-cluster-overviewScripts act only on the cluster named demo-cluster. Creating it switches your kubectl context to it; switch back with kubectl config use-context <name>.
Apache 2.0 · Go and shell · Last updated October 2026
Make it yours. Label your own ConfigMap with k8sgpt-detection-pack.io/critical=true, install charts/k8s-watcher on your dev cluster, or set K8SGPT_LOCAL=true so nothing leaves your desktop.
Clone the repo and run ./scripts/up.sh from demo-cluster (or make up from the root). The cluster, apps and dashboards come up in 5 to 10 minutes.
Trigger a fault: ./scripts/trigger-oomkill.sh. Stop it with --stop. The memory leak has its own script; the ConfigMap change is one kubectl command.
kubectl --context kind-demo-cluster -n k8sgpt-system get results lists what the operator and the watcher wrote. The payments-api pod shows up within a minute or two.
Open Grafana at localhost:3000/d/demo-cluster-overview and compare the panels against the Results. The gap between the two is the point of the exercise.
Community
Open an issue and say which one. The best ones become the next scenario. This is educational code, provided as-is: the maintainers read every issue but make no support commitment. Contributions are welcome; see MAINTAINERS.md in the repo.