Documentation menu

How It Works

Architecture overview — from cluster signal to dashboard incident.

DataAgent has two parts: a lightweight agent that runs in your cluster, and a cloud hub that your team interacts with through the dashboard.

Architecture

DataAgent architectureYour Cluster runs an Agent Pod, deployed as a single Helm chart, with five containers: the Orchestrator, a Topology Analyzer that also runs K8sGPT analysis, a Metrics Collector, a Watcher, and a Log Watcher. The Orchestrator streams findings over gRPC with mutual TLS to DataAgent Cloud, which contains the Tunnel Broker, a REST Web API, an AI Worker, and a PostgreSQL database. DataAgent Cloud serves the Dashboard in your browser.Your ClusterDataAgent CloudAgent Pod5 containersOrchestratorTopology Analyzer + K8sGPTMetrics CollectorWatcherLog WatcherHelm chart · single deployTunnel Broker (gRPC / mTLS)Web API (REST)AI WorkerPostgreSQLDashboard (browser)

Step by step

  1. Detection — The agent pod runs 5 containers: the orchestrator, topology analyzer, metrics collector, watcher, and log watcher. Four sources detect problems:
    • K8sGPT analysis, running inside the topology analyzer every 30 seconds. It inspects pods, deployments, nodes, persistent volumes, and jobs for known failure patterns.
    • Event watcher — the watcher container reports Kubernetes Warning events and ConfigMap changes as they happen.
    • Log watcher — the log-watcher container scans pod logs for known error patterns, catching application errors that never show up in pod status.
    • Prometheus alerts — from the Prometheus the chart installs. This bundled Prometheus is the only one the agent integrates with.
  2. Deduplication — Each finding is fingerprinted with a SHA-256 hash of its resource kind, namespace, name, and error. Identical findings from repeated analysis cycles are deduplicated — the same issue does not create multiple incidents.
  3. Streaming to Hub — The orchestrator sends findings to the DataAgent cloud over a persistent gRPC stream, secured with mutual TLS. No inbound ports are opened in your cluster.
  4. AI Enrichment — The AI worker receives each new finding and runs an enrichment pipeline using Claude on AWS Bedrock. It queries pod metrics, node conditions, and related incidents to produce a structured explanation.
  5. Dashboard — The enriched incident appears in the dashboard with severity, affected resource, AI explanation, and a chat interface. Your team can investigate, assign, and resolve from a single view.

Security model

  • No inbound connections — the agent initiates an outbound gRPC connection to tunnel.data-agent.co:443. No ports need to be opened in your cluster or firewall.
  • Mutual TLS — the agent authenticates with a certificate issued by DataAgent. The Hub verifies the certificate before accepting any data.
  • Tenant isolation — all data is scoped to your organisation. No data is shared between tenants.
  • Read-only cluster access — cluster-wide, the agent’s Kubernetes service account can only read. It cannot modify your workloads or any other cluster resource. Its only write permissions are in its own namespace: it can create Secrets and manage its own certificate Secret, and it can request short-lived tokens for its own read-only dataagent-live-read service account. See Agent permissions for the full breakdown.

Cookie Settings