Monitoring helps your team understand how a Kubernetes cluster and its workloads are behaving, where resources are being used, and what needs attention. You’ll use that information to investigate problems and decide what to change.
The harder question is what to collect. Kubernetes exposes metrics across several layers, so you need to decide which ones will help you understand a problem or take action.
Monitoring tracks a system’s health and performance over time, giving teams data they can use to investigate changes and trigger alerts when intervention is needed. It overlaps with observability, which is the ability to understand a system’s internal state through its outputs.
Metrics, logs, and traces help teams build that understanding and diagnose problems, including ones they didn’t anticipate.
Pods and nodes can be created and removed as the cluster changes, so monitoring needs to account for more than a fixed set of machines. Workloads also share infrastructure: containers on the same node can compete for CPU and memory, and multi-tenant environments add teams and workloads sharing cluster resources.
Networking adds another area to investigate. Communication between pods, nodes, and Services depends on the cluster’s network implementation, which matters when workloads can’t communicate as expected.
Kubernetes components, add-ons, and applications expose metrics that serve different purposes. The API server and controller manager expose component metrics, while Metrics Server provides CPU and memory resource metrics for autoscaling and basic inspection. Metrics Server doesn’t replace a full monitoring pipeline.
The Kubernetes control plane also needs monitoring. Slow API responses or etcd disk writes can delay scheduling and autoscaling operations, even when worker nodes have available CPU and memory. Severe or sustained problems can cause requests to time out and disrupt those operations.
Monitoring spans the cluster, its nodes, and the workloads running on them:
Some metrics are useful at more than one layer, so keep the scope clear when interpreting them.
Prometheus and Grafana are commonly used together for Kubernetes monitoring: Prometheus collects and stores metrics, and Grafana provides dashboards for exploring them. Commercial platforms such as Datadog are another option.
Whichever tools you use, you’ll need to configure the monitoring pipeline to discover and collect the metrics you need. An exposed metrics endpoint makes data available, but doesn’t mean your monitoring system is collecting it.
Much of the data you collect is useful without requiring an immediate response. Dashboards and historical data help teams investigate problems and understand trends.
Alerting adds a decision about when someone needs to act. A page should identify a problem that requires timely human intervention; other findings can wait for review without interrupting the on-call engineer.
Collecting and retaining more monitoring data can increase storage costs and make useful signals harder to find. As you choose what to collect, consider what each metric will help your team understand or investigate.
Fairwinds recommends covering these areas:
Workload Costs: Track what it costs to run your clusters and workloads. Use resource usage and rightsizing data to understand how CPU and memory settings affect capacity and spend.
The areas above help you understand infrastructure and workload behavior. To assess service health, Google’s Site Reliability Engineering book describes four golden signals:
To investigate latency further, traces show how a request moves through services and how much time individual operations take. With appropriate instrumentation and context propagation, tracing through OpenTelemetry can help investigate latency across service boundaries.
When a service slows down or requests fail, you need to connect what users are experiencing with what’s happening in the application and the cluster. Service metrics, traces, Kubernetes events, and resource data provide different parts of that picture. Use them together to investigate where the problem is occurring.
For software vendors supporting customer-hosted deployments, that investigation also involves working across application and infrastructure teams. Clear responsibilities between the application team and the team managing Kubernetes help establish who investigates each finding and where they need to collaborate.
Reserve paging for conditions that require timely human intervention. These can include problems affecting users and resource constraints that could soon affect service. Route lower-priority findings to a channel where the team can review them during normal working hours.
Each page should give the responder a clear reason to act. Where a problem can be resolved safely through automation, reduce the need for repeated manual intervention.
Need help managing your Kubernetes platform? Explore Fairwinds Managed Kubernetes or talk with our team.
Originally published July 18, 2023. Updated October 22, 2026.
