What You Won’t Hear at ObservabilityCON

Next week, a few thousand people who run production systems will walk into Pier 27 in San Francisco for ObservabilityCON 2026. They will hear a lot about AI agents. That is a safe prediction, because every large vendor in the category spent the last three months announcing one:

• Datadog: “not just observing, it’s fixing,” said its CEO, after shipping autonomous remediation in Bits AI.
• Dynatrace: an Autonomous SRE Agent, and Arize for $915 million.
• Palo Alto Networks: Cortex XCOR, observability with an AI SRE agent built in.
• New Relic: Infrastructure 360, feeding its Autopilot SRE agents.

These are serious products from serious teams. But look at where the agent lives in every one of them. It sits on top of the vendor’s own data store, reading the data you already paid to send there. The agent lives where the bill is.

That is the part you won’t hear on stage. And it is the part the people who run production are already talking about.

Half of them are still getting up at 3 a.m.

Conference agendas tell you what vendors want to say. To hear what practitioners actually worry about, we went where they talk to each other with no vendor in the room: Reddit. Between June and October we read the posts and comments of 355 practitioners in r/sre, r/devops, r/kubernetes and r/PrometheusMonitoring. They post anonymously, unprompted, to peers, so they write about what actually hurts, not what a survey asked them. This is a keyword-matched sample, not a survey, and Reddit leans toward hands-on engineers more than buyers. Treat the numbers as direction, not precision. The direction is clear anyway.

Half of them (179) wrote about paging pain: being woken up, carrying the laptop to dinner, getting paged for the same issue every week. About one in seven wrote about the cost of the tools they already pay for. Only nine people, under 3%, used the words autonomous remediation.

Read that carefully, because it is easy to read backwards. It is not a lack of demand. Half of them describe exactly the problem autonomous remediation exists to solve: being woken up for something a machine could have fixed.

Now look at what an AI assistant does to that complaint. It investigates the incident and explains it, which is genuinely useful. And then it still needs you awake at 3 a.m. to act on the explanation. An assistant shortens the night. It does not give it back. Only a fix you trust lets you stay asleep. What practitioners do not have yet is the vocabulary for that, or a reason to trust a machine with the fix. That second part is the honest section further down.

Figure 1: What 355 SRE, DevOps and Kubernetes practitioners wrote about, June to October 2026. Source: the author’s analysis, a keyword-matched sample from r/sre, r/devops, r/kubernetes and r/PrometheusMonitoring.

The most honest summary came from a thread title in r/devops: “If we already have Grafana, Datadog, Splunk, PagerDuty, etc., why are production incidents still so hard?”

That question deserves a better answer than “add an agent.” Grafana’s own 2026 Observability Survey found that the average company already runs eight observability tools, and that complexity (38%) is now the top concern, ahead of cost. More tools have not made incidents easier. They have made the stack harder to own.

The outage nobody was alerted to

One senior SRE told a story that makes this concrete. They had a 40-minute outage and nothing alerted, because traffic dropped 95%.

Think about why. Every alert in that stack was a rule a person wrote, for a failure a person imagined. Error rate above a line. Latency above a line. Nobody wrote “page me when the errors stop because the users stopped coming.” The data was there the whole time. The dashboards showed it. Nobody was looking, and no rule was waiting for that shape.

Another thread title says the rest: “2am page, the only person who’d know why is gone.” The stack holds the data. The understanding lives in people, and people leave.

Ship the data to the agent, or the agent to the data

The industry has been here before.

Twenty years ago, big data hit the same wall. Copying terabytes to a central server so a program could read them was slow and expensive. Hadoop’s answer was written into its design docs as a principle: “moving computation is cheaper than moving data.” Send the program to where the data sits. That one idea reshaped data infrastructure.

Observability never made that move, and for a good reason. The consumer was a human. A human needs one screen, one query language, one place to look. So we built pipelines that ship every metric, log and trace to a central data cloud, and we built twenty years of layers on top so people could read it: dashboards, alert editors, query languages, pre-aggregated rollups.

An agent does not need one screen. It can go to the cluster, read the state of what is running, check what changed, pull the logs for the one pod that matters, and leave. It reads data where it lives, when it needs it.

Figure 2: Two architectures. On the left, every signal is shipped to a central store so people can read it. On the right, the agent runs next to the systems it watches, reads what it needs in place, and the central store becomes long-term storage.

Practitioners are already describing the right-hand side, without calling it an alternative yet. One r/devops commenter sketched the whole pattern in one line: “OTel collectors -> cheap storage -> agent querying/analyzing that data directly, with UI generated on demand.” Another, in r/sre: “The integration/context layer is more important than the model choice.”

None of them said “replace the observability stack.” They said: put the thing that reasons next to the data, and stop routing everything through a screen.

What happens to twenty years of layers

If the agent goes to the data, two things change for the stack you already own.

First, the central backend becomes long-term storage. It is still the system of record. It is where you go for history, audits, trends and the slow investigation that needs six months of data. That is valuable. But it is the value of a well-run archive, not of a live control room.

Second, the layers built for human readers turn from value into overhead. Dashboards nobody opens. Alert rules tuned by hand. Pre-computed rollups waiting for a person to ask. An agent does not use any of it. It builds the view it needs, for the incident in front of it, and throws it away when it is done.

This is not an argument for less sensing. One r/sre commenter put it well: “Observability is more important than ever in a world where most code is generated without rigorous manual review.” They are right. AI-written code means more change, and more change means more to watch. What shifts is where the data is processed and who reads it. People will still want some views of their own: compliance reports, product analytics, the executive summary. Those can be generated when someone asks, instead of maintained forever in case someone does.

What the central data cloud actually costs

This is not a theory. We priced it.

In an earlier piece, Nobody Can Tell You What Observability Really Costs, we took one identical Kubernetes workload and priced it on every major backend: the same active series, the same log volume, the same 60-second scrape, all at list price. Running the open-source stack yourself is the floor. Every bar below is what the same workload costs on each option, as a multiple of that floor.

Figure 3: The managed premium. One identical Kubernetes workload priced on every backend at list price, as a multiple of running the open-source stack yourself. Source: the author’s published cost model (method in “Nobody Can Tell You What Observability Really Costs”).

Two things stand out. The platforms that copy everything into a central store land at roughly three to nearly nine times the floor. And the cheapest managed option, about 1.2 times the floor, is the one that reads the cloud’s own telemetry where it already lives instead of copying it.

The premium is not for storage. It is for the copy, and for the screen built on top of it. As we showed in Good news: your observability bill is about to drop, much of the gap is an architecture choice: some tools re-ingest telemetry the cloud provider already bills for, so you pay twice for the same signal. These are list prices without committed-use discounts, and discounts at scale can reorder the middle of the chart. They rarely close a gap that size.

The bill was built for a reader who left

Here is the part that matters most to whoever signs the contract.

Almost every observability bill is priced on the way in. You pay per gigabyte ingested, per active series, per host, per day of retention, per tier of hot or cold storage. You pay for data when it arrives, whether anyone ever reads it or not. That made sense when the product was a place for people to look. The more you sent, the more there was to see.

When the main reader is an agent that pulls evidence on demand, that pricing points the wrong way. Most of the data is stored and never touched. The expensive part, a model reasoning over the right slice of evidence, happens at read time. Paying for every byte at the door is paying for the warehouse as if it were the store.

The fair version looks different. Storage priced like storage. Compute paid when the agent actually reads. A small set of signals streamed live, the ones that must wake someone up, and everything else kept cheap and close to where it was produced.

Self-hosting is not automatically the answer, and the forums are honest about that too. One team moved from Grafana to Datadog because running the stack themselves “was costing us essentially 2 SREs full time.” Cheap storage is not free if two engineers have to babysit it. The point is not who hosts the data. The point is what the price is attached to.

The honest part: nobody lets the agent fix things alone yet

If this all sounds like a pitch for letting an AI run production, the forums will cool you down fast.

The strongest theme in the research was trust. A junior on-call engineer summed up everything they had read: “agents can investigate alone, nobody lets them fix alone.” Another engineer: “Most of us have been burned enough times that ‘autonomous’ is a scary word, not a selling point.” And the clearest rule anyone wrote: “Let it draft, never let it decide, until checking its work is fast.”

The skepticism runs deeper than trust. The most upvoted comment in the whole set (158 points) says putting an agent on toil is an antipattern: automate it with a deterministic script instead. In r/kubernetes, someone called the category “just a llm wrapper and isn’t worth the cost.”

My own view: these comments remind me of what people said about AI-written code in 2025. How much would you ever trust code a model wrote? By 2026, with tools like Claude Code, that debate was mostly over. I think agents in production will follow the same path. It comes down to trust and the quality of the agent. That is still a real challenge, but I believe it will be solved sooner than most people expect, and it is the area I am personally focused on.

Both points are fair, and neither one argues against moving the agent to the data. Reading in place is useful long before anyone trusts an agent to act. An agent that investigates next to the cluster, with a person approving every change, already removes the need to ship and index everything just in case. And the trust people are asking for (draft first, check fast, earn the right to act one kind of fix at a time) is easier to build when the agent’s evidence is local, specific and auditable than when it is a guess drawn from a lake.

The question to ask at every booth

So is this shift worth acting on now, before the category settles? Yes. Not by ripping out the stack your people rely on, but by changing what you ask it to be. Keep the data. Keep the archive. Stop paying for the pane on every byte, and start asking vendors to price the reading, not the shipping.

If you are at ObservabilityCON next week, here is one question to take to every booth:

“If my agent could read my data where it already lives, what would I still be paying you for?”

The good answers will be about storage, history, governance and the views your people still need. The weak answers will be about ingestion.

The stage will show you an agent on top of the data cloud. Look at which way the arrow points.

If you would rather see this pattern than read about it, the free, open-source autonomous SRE training series walks you through building an agent that runs on your own cluster, reads what it needs in place, and reports to Slack. Two hours, your laptop, your cluster.

References

What the vendors announced

• Datadog, Q2 2026 earnings call transcript (“the future of observability is not just observing, it’s fixing”) and autonomous remediation in Bits AI.
• Dynatrace, Autonomous Operations and the Autonomous SRE Agent; agreement to acquire Arize ($915M, August 13, 2026).
• Palo Alto Networks, Cortex XCOR launch (October 2026).
• New Relic, Infrastructure 360 (October 6, 2026).
• Grafana Labs, ObservabilityCON 2026, San Francisco, October 19 to 21.

What practitioners said

• The author’s analysis of 355 practitioners in r/sre, r/devops, r/kubernetes and r/PrometheusMonitoring, June to October 2026. Keyword-matched, not a survey. Themes per author: paging pain 179, incumbent cost 55, incident response 46, alert noise 39, autonomous remediation 9.
• Threads and comments quoted above, linked in place. All quotes are from public posts; usernames are left out on purpose.

What the central data cloud costs

• Nati Shalom, Nobody Can Tell You What Observability Really Costs: the method. One identical workload priced on every backend at list price, common 60-second scrape, sizing from the CNCF survey.
• Nati Shalom, Good news: your observability bill is about to drop: the managed premium over the self-hosted floor, and the double-bill when a tool re-ingests telemetry the cloud already charges for.

Industry data

• Grafana Labs, 4th annual Observability Survey (2026): complexity the top concern at 38%, an average of eight observability tools per company.
• Apache Hadoop, HDFS Architecture: “Moving Computation is Cheaper than Moving Data.”