Right-Sizing Your Observability

The argument that follows is an architectural one rather than a billing one. An agent reading state and change on demand needs a fraction of what a stack built for a human reader collects continuously, which is why the saving comes from the shape of the system and not from a procurement exercise.

The trap in bolting an agent on top

A bolt-on agent can only read what was shipped: metrics, logs and traces. That is the quarter of the evidence that costs essentially the whole bill, and carries the least weight in finding a cause.

The evidence that actually resolves incidents, what changed just before the fault, cluster state and events, live topology, was never shipped to the central store. A bolt-on agent cannot see it.

It is not searching a haystack. It is searching the wrong haystack.

The deepest version is structural. Topology was never shipped to the central store. So a bolt-on agent gets no map: hundreds of alerts from one failing database arrive looking like hundreds of separate problems.

Autonomy makes this worse, not better. An agent that acts needs more evidence than a dashboard did, and if it sits on the central store the only way to give it more is to ship more data. Bolt-on autonomy has a built-in incentive to grow your bill.

The change: move the agent to the data

Observability moved all data to the middle. Otto, our agent, goes the other way. It runs inside each cluster, next to the thing it watches, reading the evidence at the source.

Central data cloud: move the data to the question. Distributed data cloud: move the question to the data. Section 6 works it through.

What that unlocks

Once detection happens at the source, your backend stops being the real-time layer and becomes what it is genuinely excellent at: forensics, search and audit. It gets promoted to the job it was always best at.

From there the pipeline becomes adjustable rather than fixed:

• Rate. How much you send, and how often.
• Retention. How long it stays in an expensive tier.
• Tiering. Real-time stays at the source. Long-term goes to your existing platform. Cold goes to object storage.
• Cardinality. Which label breakdowns are worth indexing at all.

Two ways in, one estate

For people. Ask in plain language and get a chart or report built on the fly, across every store.

For machines. The DataAgent CLI, so Claude or any agent framework can query the whole estate directly. An assistant that pulls raw telemetry to reason burns enormous context and waits on the network. One that asks a scoped question and gets findings back is far faster and cheaper.

When you see something

Most autonomous SRE products ask you to trust them with production before they have delivered anything. This one pays for itself first, read-only, so you can judge it on evidence before it is allowed to change anything.

Part one: it costs more than you think

Four reasons the number is bigger than the invoice suggests.

• It was built for people. That design decision is most of the cost.
• It compounds. The bill is a multiplication, not a sum.
• It cannot be compared. Every vendor bills in a different unit.
• Most of it buys nothing. Three quarters of what an investigation needs is free.

Nobody was being wasteful. The mix was set for a reader who might arrive later, and nobody went back to ask what each stream was worth.

When an agent runs the day to day, the main reader of telemetry is the agent, on the live stream. Not a person opening a dashboard afterwards.

The pipeline was shaped around a reader who is leaving the room.

Your bill is not one number. It is several multiplied together: how many services you run, how many copies of each, how many signals each one sends, how many label combinations those carry, and how long you keep it all.

They multiply rather than add. So adding a few services and a couple of labels does not make the bill a little bigger. It puts part of it on a curve.

You cannot put these side by side. "$30 per node" and "$6.50 per thousand series" are not the same kind of number.

A vendor can look cheap by pointing at the unit where its rate is low, and staying quiet about the unit where your data actually piles up.

How to compare them fairly

Pick one workload. Run every vendor's own published prices against it. Take the workload size from independent survey data, not from anyone's price list.

Two corrections do most of the work.

Use the same scrape rate

Some platforms collect metrics every 15 seconds by default. That is four times the data of 60 seconds, so they look four times worse for no real reason. Price everyone at 60 seconds.

Count cloud data the same way

Your cloud sends data about managed services whether you want it or not. Some platforms read it where it sits, which is cheap. Others copy it into their own cloud, so you pay twice. Charge each one for what it really does.

That gap is the premium you pay not to run the stack yourself. For many teams it is a fair trade. Running metrics and logging at scale is real work and somebody has to be on call for it.

It is worth paying on data you actually read. It is much harder to justify on data nobody ever queries.

Right-sizing does not mean leaving your vendor. It means not paying that premium on telemetry you never look at.

Three quarters of the answer costs nothing. It is already in your cluster. You just have to live there to read it.

Metrics and logs stay essential and stay always on. They are how the agent notices the problem at all. They simply carry less weight than their share of the bill.

Read that table honestly

These weights are the agent's starting assumptions, then tuned per fault class against your own estate over time.

Whether metrics or logs lead on your bill depends on your stack. Traces will not be worthless on every fleet. On an estate leaning hard on tracing for latency work, that last row looks different.

What does not change is the shape. The evidence carrying most of the answer is neither metrics nor logs, and it is free.

Most of an observability bill is not buying answers. It is buying the feeling of being covered.

Part two: the architecture is the opportunity

This is not a cost-cutting exercise. The saving is a side effect of a change in where data lives and who reads it.

• Data stops moving to the middle. It stays where it is produced.
• The AI agent becomes the single interface over all of them.
• Your existing platform keeps the job it is best at: forensics and audit.
• Every change is gated, measured and reversible.

Centralising worked because people cannot be everywhere. One screen, one place to look, so the data had to come to the screen.

An agent can be everywhere. It runs inside each cluster, next to the thing it is watching.

This is the whole change. Not a cheaper central store. No central store in the live path at all.

Seven layers existed because a person read the output

Ingestion, normalisation, correlation, aggregation, dashboards, alerting, a query language. Every one existed to turn raw signal into something a person could scan at 2am.

An agent does not scan. It queries exactly what its next decision needs, and reads the raw thing in place.

Everything else layers on top of that model. It is the order that matters, and it is the opposite of how most tooling works.

Telling a symptom from a cause is not a reasoning problem. It is a structure problem, and structure is cheap.

Why the order changes the answer

To an agent holding a topology model, a failing database is one problem with a known blast radius. The shared resource is marked as shared, so the storm downstream collapses into one incident: one cause, a list of symptoms.

What this buys, concretely

• Noise collapses. Correlated events fold into one incident per fault, instead of one per affected service.
• Root cause gets faster and more accurate, because the candidate set is the blast radius rather than the whole estate.
• It costs dramatically fewer tokens. The agent reasons over one cause and its radius, not hundreds of duplicate events. That is the difference between an investigation you can afford on every incident and one you cannot.
• Cuts become safe. You can only know which signals a fault class needs if you know the structure that fault travels through.

You ask a question in plain language. The agent decides where to look and goes and gets it.

Recent and local? It reads the live window inside the cluster. Older than that? It reaches into long-term storage, and only because the question needs it.

The intuitive move is to shorten retention. On most platforms that deletes your data and barely moves the bill.

Stopping ingestion at the source moves the bill and keeps your data.

Several vendors offer tools that prune unused series. Keep running them. But they act after the data has arrived, so they cannot stop a charge you already paid. And they see one vendor, so they cannot see the same telemetry billed two or three times across your estate.

• config-reversible. The setting reverts and the state comes back exactly.
• forward-only. The setting reverts, but the gap in data is not refilled.
• destructive. The data is gone. One lever, off by default, needs a second confirmation.

That precision comes from living in the cluster. An external tool reading an invoice cannot see a namespace, so it can only ever propose a percentage.

The plan cuts deeper and protects the investigation, because it comes from the same model the agent diagnoses with.

At the deepest notch the backend changes job. Because the agent investigates on the live stream, the paid firehose no longer has to be the real-time source. It becomes a backup, flushed back to hot the moment an incident opens.

Data is tiered, not deleted, in both. The paid ingestion and the hot tier shrink. The data survives in cold storage.

Every vendor's own cost tooling is real and worth running. All of it shares the two limits from section 9.

An assistant weighs what you already ship, so it can only ask you to ship more. Autonomy weighs everything, and the scale tips to the free side.

No tool that only reads bills can produce that column. No tool that only reads telemetry can either. It takes the thing doing the fixing to also do the accounting.

Not deletion

Data is tiered, not deleted. What you read stays hot. The rest moves to cheap storage and stays queryable.

Not replacement

We right-size what you send. We do not replace the platform you send it to.

Not a one-time cleanup

Estates drift. New clusters log by default and cardinality creeps. Each cut is re-derived continuously, so the saving is re-earned rather than decaying.

Start where we started. Three quarters of the evidence an investigation needs is free and already in your cluster. The quarter you pay for costs more than it is worth.

That imbalance made sense once. Observability was built for a person who might look later, so everything was kept hot just in case. The only way to answer a question was to move the data to the person.

An agent changes the direction of travel. The data stays where it is made. The question goes to the data. And the agent becomes the one interface over all of it, hot or cold, in plain language.

Spotting a fault and doing nothing about it is not health. It only makes the outage well documented.

Observability was only ever the sensor. The bill stops being a fixed cost and becomes a lever you keep pulling.

And you can judge each rung before taking the next. The bill proves the agent reads your estate correctly. Time to recover proves it can reason about it. Only then does a fault class earn the right to act without waiting for a person.

You should not take this on trust. The next page is how to get your own number, read-only, in about half an hour.

The terms

• Read-only. No write access at all.
• Runs in your environment. Your data does not move.
• Only a summary report leaves. Findings, never raw telemetry.
• Nothing to install to see the number.
• The report is yours to keep, whatever you decide.
• If the plan recommends nothing, it says so. That is a useful answer too, and it is still free.