---
title: "DataAgent Live Demo"
canonical: https://data-agent.co/guides/live-demo/
type: guide
updated: 2026-10-21
---

# DataAgent Live Demo

Watch it handle a live outage. We broke a deployment on purpose with an image tag that does not exist in the registry. Nobody was paged, and by the time anyone looked the cause was named, the evidence was attached and the fix was written. Every screen here is from that run.

**Who it is for:** The short version first, then the same incident screen by screen, then the route a fix takes to production.

## In short

- A deployment went out with an image tag that does not exist in the registry. About 90 seconds later the cause was named, the evidence attached and a three-step plan written at 95% confidence, with every step marked reversible.
- Nobody was paged, and no change was made to the cluster without approval: no DataAgent component holds write credentials to the cluster.
- The plan becomes a pull request in your repository, a person approves it, and your own runner applies it with the DataAgent CLI under your credentials, then verifies and reports back.
- The interesting runs are the ones that stop: a declared gap, a refused pre-flight, a hand-back to a human, a step refused by policy, or no plan at all.
- Running it yourself takes three pieces, and two of them you already have.

## Nobody was paged. The answer was already written.

A deployment went out with image tag v2.1.0-hotfix-broken, which does not exist in the registry. The pod could not pull it. The rollout stuck.

> **About these screens**
>
> Incident DAT-290 on cluster weave-demo-cluster, 5 October 2026. We injected the fault on purpose in our demo environment. Everything DataAgent did after that ran live and unscripted. The screens are unedited except for grey blocks that mask the namespace owner's name, AWS account numbers and the GitHub username. Sections 6 to 8 use a reference implementation of the execution path, and say so where they do.

*Figure: The estate, while it was happening. Two critical incidents, one cluster failing, one pod on image pull fail. Nothing here needed a human to notice it.*

- **~90 sec**: from the bad rollout to a written root cause with evidence attached
- **95%**: confidence on a three-step plan, every step marked reversible
- **0**: people woken up, and no change made to the cluster without approval

### What it worked out on its own

*Figure: Both critical incidents, same fault. The service has no ready endpoints, and the pod behind it cannot pull its image. Each row already carries its analysis and its next step.*

Two incidents, one cause, raised 16 minutes apart as the second rollout attempt repeated the first.

> The work of triage had already happened. What was left was a decision.

> **What is not happening here**
>
> No DataAgent component holds write credentials to the cluster. No change is applied without a merged pull request. The analysis below is read-only, start to finish.

## Five screens, then the route to production

Each screen answers the question the one before it raises.

1. **It appears: Detection**: Raised, classified and scored without anyone asking.
2. **It explains: Analysis and evidence**: The cause, and every source it was read from.
3. **It scopes: Topology**: What else is downstream of the thing that broke.
4. **It proposes: A plan**: Exact commands, risk and reversibility per step.

*Only the last one is a suggestion. The first three are findings with their sources attached.*

### Two questions worth holding

- **Would my monitoring have caught this?** Only as a symptom. Request metrics show traffic falling, not why. Screen 3 shows what was actually wrong.
- **How did it know?** Nobody told it. Screen 3 lists every source it read, including the rollout history that names the exact revision.

## The incident, screen by screen

Five screens. The line at the bottom of each says what it settles.

### Screen 1: Raised, scored and classified (T+0)

*Figure: Severity, cluster, namespace (masked), signal count, an RCA badge and a next step, before anyone opened it.*

Notice what is **not** here: a severity to triage by hand, a runbook to look up, a link to a dashboard. The row carries the conclusion. 145 signals collapsed into this one incident, which is the fault-class grouping doing its job.

> **What this proves**
>
> Alert volume is not the problem. Alerts without a conclusion are.

### Where to look

- **RCA done**: The analysis ran before anyone looked.
- **Signal count**: 145 signals, one incident.
- **Next step**: In plain language, not a severity level.

### Screen 2: It names the cause (T+90s)

*Figure: The analysis, labelled as generated and counting its sources.*

It names the exact tag that does not exist, the timestamp of the rollout that introduced it, the working image to go back to, and the chain of consequence: cannot pull, so the container cannot start, so the readiness probe never runs, so the Service has zero endpoints. Then the remediation plan at 95% confidence, directly underneath.

> **What this proves**
>
> The warning strip at the top is the part to trust. It says one data source could not be read and that the analysis was written without it, rather than quietly filling the gap.

### Where to look

- **The honest note**: One source unavailable, declared at the top.
- **The chain**: Pull fails, container waits, probe never runs, endpoints empty.
- **The fix**: Named by image digest, not described vaguely.

### Screen 3: It shows its working (evidence)

*Figure: Pod status, readiness conditions, the raw Kubernetes events, and the deployment revision table.*

The revision table at the bottom is the whole case in four rows. Revision 2 carries v2.1.0-hotfix-broken and is in ImagePullBackOff. Revision 1 carries a digest and is marked Working. That is the change trail, and it is why the plan can say exactly what to roll back to instead of guessing.

> **What this proves**
>
> A central telemetry store usually holds metrics, logs and traces, not rollout history. This evidence is read in place, from the cluster, and it costs nothing to collect.

### Where to look

- **Pod status**: Pending, 0/1, with the full image reference (account number masked) and the resolve error.
- **Conditions**: Ready false, ContainersNotReady, but PodScheduled true.
- **Revisions**: Revision 2 broken, revision 1 working. The change trail.

### Screen 4: It scopes the damage (topology)

*Figure: Three workloads and three dependencies around the failing service, with the IRSA roles drawn in.*

Small estate, small map, and that is the point: [the map is derived from this namespace live](https://data-agent.co/guides/ai-sre-kubernetes.md#topology-first), not pulled from a static service catalogue someone forgot to update. Checkout depends on postgres, and both carry IRSA roles. The legend distinguishes edges declared by the operator from edges inferred from a container environment variable.

> **What this proves**
>
> Knowing what sits downstream is what turns one broken pod into a judgement about blast radius, rather than a page at 3am about a pod.

### Where to look

- **The subject**: Checkout, ringed as the incident subject.
- **The dependency**: Postgres, and the IRSA roles behind both.
- **Edge types**: Declared by the operator, or inferred from an env var.

### Screen 5: It writes the fix (plan)

*Figure: Three steps. Verify the target exists, roll back, wait for ready. Each with its risk, its reversibility, its exact command and how to confirm it worked.*

Step one does not change anything. It checks the image it is about to roll back to actually exists in the registry, so the fix cannot replace one broken tag with another. Step two is the single command that fixes it. Step three waits and confirms. And there is a warning on step one flagging a command worth reading before running.

> **What this proves**
>
> You are approving a specific list of commands with a verification attached, not granting an agent permission to try something.

### Where to look

- **Step 1**: Read-only. Confirms the rollback target exists.
- **Step 2**: One command, reversible, named revision.
- **Step 3**: Waits for Ready 1/1 before calling it done.

## The route to production

Everything up to here was read-only. This is the only part that writes, and it runs entirely on your side.

> **A note on the next three screens**
>
> The execution path is [your own GitOps route](https://data-agent.co/guides/remediation-first-in-action.md), so there is no vendor console to screenshot. What follows is the reference implementation of that route, running against a live cluster. The analysis and the plan come from the product exactly as shown above. The approval and the run are yours.

1. **One: The plan exists**: Written by the hub, which never applies it. It sits there and does nothing.
2. **Two: It becomes a pull request**: In your repository, under your branch protection.
3. **Three: A person approves**: A review and a merge, in their name. The merge is the trigger.
4. **Four: Your runner applies it**: The DataAgent CLI runs pre-flight, action, verify, report. Your credentials throughout.

*Autonomy is earned per fault class: recommend only, then approve first, then autonomous, on a verified record. Until a fault class has earned it, a person approves each change. Once it has, the merge no longer waits for a person, but the change still runs in your repository, on your runner, under your credentials.*

### Screen 6: The plan, as a pull request (git)

*Figure: Cluster, namespace (masked), workload, incident, plan id, confidence, risk and the phase flowchart, opened by a workflow into the team's own repository. The GitHub username is masked.*

Everything a change-management process asks for is on one page: what will run, against what, why, and how risky it is. The files changed are the remediation record itself.

> **What this proves**
>
> The approval artefact is a pull request in your repository, not a button in a vendor's console. Any Approve button in the product only submits a review on this same pull request.

### Screen 7: Approved, and running (git)

*Figure: Five jobs chained on a self-hosted runner: resolve, pre-flight, action, verify, report.*

Pre-flight confirms it is talking to the cluster the plan named and that the plan is still the one that was approved. Only then does the action job run, and it is the only one with write access. Verify waits for the rollout before report closes the loop.

> **What this proves**
>
> The runner is yours and sits beside your cluster with your credentials. The vendor never needs access from outside.

### Screen 8: And reported back (git)

*Figure: The run report: pre-flight checks, the action and its exit code, steps skipped and why, the verification, and the call that closed the incident.*

Nothing is declared fixed because a command exited zero. It is fixed because the rollout was checked afterwards and the result was reported back, so the incident closes on the product's side too and not only on yours.

> **What this proves**
>
> A loop that does not close leaves you reconciling two sources of truth by hand.

## The interesting runs are the ones that stop

A demo where everything works proves very little. Every row below is a real outcome, not a hypothetical, and two of them happened while this guide was being written.

| What happens | Why | What the system does |
| --- | --- | --- |
| **The analysis declares a gap** | A data source could not be read. | It says so at the top of the analysis and names the tools it did use, rather than filling the gap silently. This is visible on screen 2. |
| **Pre-flight refuses to run** | The cluster the runner is pointed at does not clearly match the cluster in the incident. | The run stops before the action job and says why. A matching workload name is not treated as proof. |
| **Handed back to a human** | The runner could not reach the cluster. | It stops with the reason. It does not retry blindly and it does not report success. |
| **A step is refused by policy** | A plan step used a verb outside the allowlist. | Refused, with the reason in the report. Observe steps are skipped, never improvised into something executable. |
| **No plan at all** | The analysis did not reach a confident, reversible course of action. | Nothing is proposed. There is no row to approve, which is correct when the honest answer is that it does not know. |

> A system that cannot refuse is not safe, it is lucky.

## What it takes to run this yourself

Three pieces, and two of them you already have.

1. **One: Otto, our agent in the cluster**: About 512MB, read-only, no model weights, nothing stored locally. Reads events, rollout history and the registry.
2. **Two: A repository and a workflow**: Turns a plan into a pull request and a merge into a run. Yours.
3. **Three: A runner beside the cluster**: Runs approved plans with the DataAgent CLI and your credentials. The vendor holds none.

*The pull-request chain is the piece that makes this acceptable to a change manager.*

### Where to start

Start read-only. Put Otto on a real cluster. Setup takes an afternoon and is reversible. Then let it write analyses for a few weeks with the execution side disconnected entirely, and read what it produced against the incidents you actually had. That answers the only question that matters before anything gets an approval button: are the plans any good.

Connect the pipeline after that, for one workload first. The approval step is not a formality. It stays for each fault class until that class has earned autonomy on a verified record. Even after that, write access never goes to DataAgent. Experience changes whether a plan waits for a person, never who holds the key.
