Definition
CrashLoopBackOff is a Kubernetes pod status that means a container has exited repeatedly and the kubelet is delaying the next restart, doubling the wait each time up to five minutes. It is not an error in itself but a signal that the container is failing to run. The cause is in the container’s exit code, its logs, or the pod’s events.
Explanation
Diagnosis starts with three commands. kubectl describe pod <name> shows the last state, exit code and events. kubectl logs <name> --previous shows the output of the crashed container, not the one currently starting. kubectl get events --field-selector involvedObject.name=<name> shows scheduling, mounting and probe failures the container never got to log.
| Symptom | Likely cause | Fix |
|---|---|---|
| Exit code 1, stack trace in previous logs | Application error, missing env var, bad config value | Fix the config or image, or roll back the last change |
| Exit code 137, Reason: OOMKilled | Memory limit below actual use, or a leak | Raise the limit after checking usage, or fix the leak |
| Exit code 0, restarts anyway | Process finishes immediately | Correct command/args, or use a Job for one-shot work |
| Liveness probe failed events | Probe too aggressive, slow startup, wrong port or path | Add a startup probe, raise initialDelaySeconds, fix the endpoint |
| CreateContainerConfigError before the loop | Referenced ConfigMap or Secret missing | Create the object or fix the reference |
| Crashes only on some nodes | Node-level difference | Check node conditions, then cordon and drain the node |
| Started after a deploy | New image or config | kubectl rollout undo deployment/<name> |
A common wrong move is deleting the pod: the controller recreates it and the loop continues. The back-off resets only when the container runs successfully for ten minutes.
How DataAgent approaches it
CrashLoopBackOff is one of the Kubernetes faults in DataAgent’s fault catalog. The agent detects it from pod status and Warning events, reads the crashed container’s logs and node conditions, and explains the likely cause. The remediation plan lists the ordered steps, such as reverting a config key or rolling back a deployment, and can be dry-run before it executes. A restart alone is almost never the fix.
Example
After a deploy at 11:02, pod api-7c9d shows CrashLoopBackOff with 6 restarts. kubectl logs api-7c9d --previous ends with KeyError: 'DB_POOL_SIZE'. kubectl describe shows the environment comes from ConfigMap api-config, edited at 11:00, which no longer has that key. Fix: restore the key or roll the deployment back, then confirm the container stays running and the restart count stops climbing.
Related terms
- OOMKilled and exit code 137
- Liveness, readiness and startup probes
- CreateContainerConfigError
- Rollback
- Kubernetes events
- Auto-remediation
FAQ
- How do I see the logs of a container in CrashLoopBackOff?
- Use kubectl logs <pod> --previous, adding -c <container> for multi-container pods. Without --previous you get the newly started container, which often has no output yet.
- Will deleting the pod fix CrashLoopBackOff?
- No. A Deployment, StatefulSet or DaemonSet recreates the pod and the container fails the same way. Fix the cause, or roll back the change that introduced it.
- How long does Kubernetes wait between restarts?
- The back-off starts at 10 seconds and doubles with each crash up to 5 minutes. It resets after the container has run for 10 minutes without exiting.