Documentation menu

How Detection Works

From cluster analysis to incident creation.

Analysis cycle

K8sGPT analysis runs inside the topology analyzer every 30 seconds by default (configurable via topologyAnalyzer.analyze.interval). Each cycle inspects all namespaces and resource types simultaneously.

Alongside it, three other sources report problems as they happen:

  • Event watcher — the watcher container reports Kubernetes Warning events and ConfigMap changes.
  • Log watcher — the log-watcher container scans pod logs for known error patterns.
  • Prometheus alerts — firing alerts from the Prometheus the chart installs. This bundled Prometheus is the only one the agent integrates with.

What gets analysed

K8sGPT inspects these Kubernetes resource types:

  • Pods — crash loops, OOM kills, image pull failures, pending state
  • Deployments and ReplicaSets — unavailable replicas, stuck rollouts
  • StatefulSets — unready pods, PVC binding failures
  • DaemonSets — desired vs. ready count mismatches
  • Jobs and CronJobs — failed jobs, missed schedules
  • Nodes — memory pressure, disk pressure, PID pressure, NotReady
  • PersistentVolumeClaims — unbound volumes
  • Services and Ingresses — misconfigured routing

Deduplication

Each finding is fingerprinted with a SHA-256 hash of:

resource_kind | namespace | resource_name | error_text

If the same condition is detected in multiple consecutive analysis cycles, it does not create multiple incidents. The existing incident’s Last seen timestamp is updated instead.

A new incident is only created when a genuinely new condition is detected — one with a hash that has not been seen from this agent before.

Signal suppression window

After a finding is first detected, it enters a brief suppression window before being promoted to an incident. This prevents transient conditions (a pod briefly restarting during a rolling update) from generating noise.

The suppression window is configured per organisation. Contact support to adjust it.

Metrics collection

In parallel with analysis, the metrics collector gathers:

  • Pod-level: phase, readiness, restart count, OOM kill count, crash loop count, image pull failures, last error reason
  • Node-level: memory/disk/PID pressure, CPU and memory capacity vs. allocatable
  • Workload-level: desired, ready, and available replica counts, rollout status

These metrics are attached to incidents during AI enrichment.

Cookie Settings