DataAgent
one fault, end to end · nothing staged
weave-demo-cluster
Inject›Detect›Analyse›Topology›Plan›Approve›Run
Your clusterai-sre-sandbox
ALB → frontend → checkout → postgres   4/4 pods Running
Fault catalogueinject on purpose
idfault class
F1Network partition
F5Bad deployment / ImagePullBackOff Inject
After injectt+40s
checkout → rolled to v2.1.0-hotfix-broken
⚠ pod 5/5 Pending · ErrImagePull · 0/1 Ready
the old pod keeps serving, so no request has failed yet
Signals arriving at the hubfindings, not telemetry
sourcewhat it sawfault class
k8sgptImagePullBackOff on checkoutbad deployment
event_watcherFailed to pull image, NotFoundbad deployment
prometheuspdb_violation, checkoutbad deployment
Correlatedthe noise filter
Three signals, one cause. Promoted to incident DAT-290 · critical RCA running
DAT-290 · checkout Service has no ready endpoints AI generated · 4 sources
A deployment at 09:33:54Z rolled out image tag v2.1.0-hotfix-broken, which does not exist in the registry. The container cannot start, so the readiness probe never runs and the Service sees zero ready endpoints.
Evidencedeployment revisions
revimage tagstatus
2v2.1.0-hotfix-brokenImagePullBackOff
1f79d3cf3178e…Working
Topologythe impacted resource and its dependencies
ALB frontend checkout postgres IRSA role blast radius: 1 of 3 workloads, no downstream impact
Remediation plan95% confidence · low risk
1  Verify the previous working image exists in ECR.
   aws ecr describe-images --image-ids imageTag=f79d3cf…  read only
2  Revert the Deployment to revision 1.
   kubectl rollout undo deployment/checkout  reversible
3  Wait for Running and Ready 1/1.
   kubectl rollout status deployment/checkout  verify
Nothing has runwaiting for a human
The plan sits in the hub. There is no path from here to your cluster that does not go through a merged pull request.
Pull request · opened by workflowyour repository
clusterweave-demo-cluster
workloadService/checkout
incidentDAT-290
risklow · reversible · 95%
Reviewin your name
✓ approved by nati.shalom@data-agent.co
✓ squash merged → the merge is the trigger
GitHub Actions · your runneryour credentials
resolve› pre-flight› action› verify› report
Loop closedverified, both sides
✓ rollout healthy, 4/4 pods on 1.2.0-weave-cm
✓ incident DAT-290 closed on the hub, not just on yours
fault to fix: minutes. Nobody was paged.
one real fault class · injected on purpose · every screen from the product