EngineeringSep 27, 202610 min read

AI Is Breaking Production, and Observability Can't Fix It

AI made us faster at producing plausible code, not at keeping production healthy. More observability gives you a better thermometer; a smarter assistant gives you more memos to verify. Neither closes the loop. A remediation-first Digital Immune System does.

Nati ShalomCo-founder & CTO

Diagram comparing a thermometer's open loop (sense, alert, no state change) with a thermostat's closed loop (sense, act, realign).

TL;DR

AI-generated code is putting more failure into production. In New Relic's 2026 State of AI Coding report, 78% of technology leaders report more incidents once AI-generated code ships, and 62% say their teams ship it without line-by-line verification.
More observability won't fix it. Observability is a thermometer: it measures divergence, but by construction it can't act to close it.
A smarter AI assistant won't fix it either. Recommendations without action relocate the work to a human verifier and create a verification tax.
The category that fits is the one Gartner already named: the Digital Immune System, where auto-remediation is a first-class component and observability is one sensor among six.
Remediation-first is how it works in practice: detect → diagnose → act → verify, with humans moved to the edge of the exceptions.

What the 2026 data shows about AI-generated code incidents

In New Relic's 2026 State of AI Coding report, 94% of technology leaders rate AI-generated code as higher quality than human-written code at review time, and then 78% report more incidents once that same code reaches production. 82% of teams had at least one production failure tied to AI-generated code in the past six months. 86% say their senior engineers now spend more time fixing code, not less. And 62% admit their teams ship AI-generated code without line-by-line verification. [VERIFY: 94%, 82% and 86% are not on the report's public landing page — confirm in the full report]

Bar chart titled The Velocity Trap: initial feature shipping velocity 92%, defects discovered after deployment 88%, increase in critical production incidents 83%, unexpected instability events 79%, codebase maintainability degradation 71%, customer dissatisfaction from bugs 63%.

Read those together, and the story is unmistakable. AI made us dramatically faster at producing plausible, well-structured code. It did not make production safer. It made it less safe — more change, moving faster, with less human verification behind it. The thing we got good at (generating) turns out to be a different thing from the thing we actually need (behaving correctly in production, under load, over time).

So the industry has a real, measurable problem: AI is pushing more failure into production. The only question that matters is what you do about it. And the two most popular answers are both wrong.

Wrong answer #1: "We need more observability"

New Relic's own conclusion from that data is that 96% of leaders now consider observability imperative for managing AI-generated code, and 78% are already baking telemetry into the generated code itself — "observable by design." [VERIFY: 78% telemetry figure not on the public landing page] The company's banner for the moment is, in their words, "The Age of AI Observability." Of course it is. New Relic sells observability. When the company whose entire category is the sensor sees production catching fire, it concludes — sincerely, with real data — that everyone needs better sensors.

Observability is a thermometer, not a thermostat

But here's the distinction that the whole framing misses. A thermometer measures temperature. A thermostat has a desired temperature, senses divergence, and acts to close it. Observability is a thermometer. It is a magnificent, increasingly intelligent thermometer, and by construction, it has no capacity to change the temperature. We do not have a seeing problem. The 96% already see plenty. We have a closing-the-loop problem, and you cannot close a loop with a better thermometer.

That's why "AI observability" is the wrong category. Observability is not a destination; it's a sensor, an input to a control loop, not the loop itself. We only ever treated it as the destination because for fifteen years the only actuator at the end of the loop was a human being who needed to see to act. Make the telemetry richer, and you make that human faster. But the human is exactly the part that doesn't scale and, per the survey, exactly the part we're now bypassing.

Wrong answer #2: "We need a smarter assistant"

The second answer is subtler, and I have lived it. A while ago, I wrote a post called "I Ship Code I Don't Read." The coding-assistant world has already run the experiment the SRE world is about to run, and it taught a lesson worth carrying over.

The verification tax

An assistant who only recommends does not remove work. It relocates it to verification. Every suggestion an assistant produces is a suggestion a human now has to read, judge, and either trust or check. When the assistant is fast and prolific, the verification burden grows faster than the typing it saves. That is precisely what the survey is measuring: 86% saying senior engineers spend more time fixing, 74% of AI code needing significant rework [VERIFY: 74% not on the public landing page], and — the release valve — 62% simply shipping without verifying, because verifying everything was untenable. Assistance without action didn't reduce the work. It created a verification tax, and when the tax got too high, people stopped paying it and shipped blind.

Why "AI SRE" assistants repeat the pattern

Now watch the same pattern arrive in operations, dressed as progress. The new "AI SRE" assistants will identify a probable root cause, draft a remediation plan, and walk you through a resolution. That sounds like help. But at 2 a.m., under an incident, it hands the on-call engineer one more artifact to verify before acting — a confident recommendation that may be right, may be subtly wrong, and is now their responsibility to adjudicate. More AI output, more to check, same bottleneck. A smarter thermometer that also writes you a memo about what it thinks you should do is not a thermostat. It's a thermometer with opinions.

So the lesson from Claude Code, stated plainly: an AI that only assists shifts the burden onto the human as verifier — and that is more work, more frustration, and ultimately less safety, not more. The escape isn't a better assistant. It's a different category.

The right category has a name: the Digital Immune System

Gartner already named it. A Digital Immune System (DIS) — one of their top strategic technology trends — is the architecture for software that withstands and recovers from failure on its own, and Gartner projects that organizations that invest in it will cut downtime by up to 80%. [VERIFY: Gartner source page now redirects — confirm the 80% figure and the six-practice list]

Hexagon diagram of Gartner's Digital Immune System: observability, AI-augmented testing, chaos engineering, site reliability engineering, software supply-chain security, and auto-remediation highlighted, with an 80% projected reduction in downtime. — The six practices of a Digital Immune System, with auto-remediation as a first-class component.

The six practices of a Digital Immune System

What makes DIS the right category, and not a rebrand, is its composition. Gartner defines it as six practices working together: observability, AI-augmented testing, chaos engineering, auto-remediation, site reliability engineering, and software supply-chain security.

Look at that list, and two things jump out: First, observability is just one of the six sensors, exactly where it belongs, not the whole system. Second, auto-remediation is a named, first-class component of the act rung, the thermostat, sitting right alongside the sensing. The analysts didn't frame the future as "see more." They framed it as an immune system: something that detects, responds, verifies, and gets more resilient over time.

That biological metaphor earns its keep. Your immune system doesn't page you when it detects a pathogen and waits for you to approve a response. It doesn't hand you a "probable diagnosis" and a recommended course of antibiotics for your review. It detects, it acts, and it confirms the threat is neutralized — and it tells you afterward, if at all. It removes you as the bottleneck and as the verifier. That is the property production reliability actually needs, and it is the property that neither a better dashboard nor a better assistant can provide.

What remediation-first looks like in practice

I call this approach remediation-first, and the name is the whole argument: you lead with the action, not the dashboard and not the recommendation.

Remediation-first loop: detect, diagnose, act, verify, with human escalation only at the verify step. — Remediation-first: move the human from the center of the loop to the edge of the exceptions.

A remediation-first system inverts the order of operations that the last fifteen years trained us. Instead of observing → display → wait for a human to investigate, decide, and act, it runs detect → diagnose → act → verify, and only escalates to a human for the genuine exceptions. Observability still matters — it's the sensor feeding the loop. Investigation still matters — but it runs in service of an action the system will take, not as a report it hands you. And crucially, the system verifies its own fix — it checks that the action actually restored health, the way an immune response confirms the threat is cleared.

Autonomy with verification built in

That last part is what separates remediation-first from the reckless reading of "autonomy." The answer to "62% ship without verification" is not "verify harder by hand" — that's the assistant tax. Nor is it "let the AI act without checking" — that's just automating the failure. It's autonomy with verification built into the loop, so the verifying is done by the system, continuously, instead of by an exhausted human, occasionally. You don't approve of every antibody. You set the policy, and you trust the immune response — because it proves itself.

Where this leaves us

The data is clear that AI is putting more failures into production. The two instinctive responses — buy more observability, or deploy a smarter assistant — both leave the human as the part that has to see, decide, verify, and act. One gives that human a better thermometer. The other gives them more memos to check. Neither closes the loop, and the survey is the receipt for what happens when you don't: more incidents, more rework, more firefighting, and eventually people shipping blind because the verification burden broke them.

The category that actually fits the problem is the one the analysts already named, a Digital Immune System, and the discipline that makes it real is remediation-first: detect, diagnose, act, and verify, with the human moved from the center of every loop to the edge of the exceptions.

New Relic is right that this is a turning point. They've just named it after the sensor. It isn't the age of AI observability. It's the age of the digital immune system, and an immune system's defining trait is not that it sees the infection. It's that it does something about it.

FAQ

Why isn't AI observability enough for production reliability?

Observability tells you that something diverged; it has no mechanism to act on it. Richer telemetry makes the human responder faster, but it keeps the human as the only actuator in the loop. With AI increasing the volume and speed of change, that human is the part that doesn't scale.

Is it safe to let an autonomous system remediate production incidents?

It is safe when verification is part of the loop rather than an afterthought. The system should confirm that each action actually restored health, act only within a policy you set, and escalate anything outside that policy to a human. Autonomy is something you enable deliberately, not a default.

Which incidents are good candidates for autonomous remediation first?

Start with common, well-understood incidents where the corrective action is known and reversible: restarting a failed service, scaling resources under pressure, or rolling back a bad deployment. These restore service quickly, and deeper investigation can happen once the system is stable. For the terminology, see what remediation means in production systems.

Does a Digital Immune System replace my existing observability tools?

No. Observability is one of the six practices in a Digital Immune System — it's the sensor that feeds the loop. The shift is in what happens after the signal: instead of ending at a dashboard or an alert, the loop continues to diagnosis, action and verification.

How is remediation-first different from an AI SRE assistant?

An assistant recommends and leaves the decision and the verification to the on-call engineer. A remediation-first system acts, verifies its own fix, and only hands the human the genuine exceptions. The difference is who carries the verification burden.

Build your own

Convinced the future is remediation-first, but wondering how you'd actually build one? I've started a free, open-source training series on how to build your own autonomous SRE — the detect → diagnose → act → verify loop, wired to your own Kubernetes and cloud planes, built up step by step. This post is the why; the series is the how.

Link: How to build your own autonomous SRE