Why do Kubernetes clusters run mostly idle, and what is rightsizing?
Kubernetes clusters run mostly idle because Pods reserve far more CPU and memory than they use, and nodes carry capacity that no Pod reserves at all. Rightsizing is the practice of setting each container’s resource requests from its measured usage, so the scheduler can fit the same workloads onto fewer or smaller nodes. It only lowers the bill when the freed capacity actually leads to fewer nodes.
How idle are Kubernetes clusters?
Two vendor reports give a consistent picture:
- Cast AI’s 2026 State of Kubernetes Optimization Report (published April 2026, 2025 data from tens of thousands of clusters on AWS, Google Cloud and Azure) puts average CPU utilization at 8% and memory utilization at 20%, down from 10% and 23% a year earlier.
- Datadog’s State of Cloud Costs 2024 found that 83% of container costs went to idle resources: 54% to cluster idle and 29% to workload idle.
The Datadog split names the two layers of waste.
| Layer | What it is | Fix |
|---|---|---|
| Workload idle | Requests above what containers use | Rightsizing requests |
| Cluster idle | Node capacity that no Pod requests | Bin packing, consolidation and node autoscaling |
This article covers the first layer. The second is covered in bin packing in Kubernetes.
Why teams over-request
- Fear of OOM kills. Running out of memory kills the container, so teams add generous headroom and never revisit it.
- Sizing for the peak or the load test. A request set for a launch-day spike stays in the manifest for years.
- Copied defaults. Helm chart defaults and templates get reused across services with very different needs.
- No feedback loop. The team that writes the manifest rarely sees the node bill, and nothing breaks when requests are too high.
Because the scheduler places Pods by requests, not usage, each of these turns directly into nodes you pay for.
How to measure requested vs used
Compare each container’s request with its usage over a window long enough to include weekly peaks and batch runs, such as two to four weeks. Use high percentiles, not averages: an average hides the bursts that a request must cover.
- CPU: compare the request with the 95th percentile of usage (
container_cpu_usage_seconds_totalfrom cAdvisor) and add headroom. Under-requesting CPU makes the container slower under contention but does not kill it. - Memory: compare the request with the maximum or 99th percentile of the working set (
container_memory_working_set_bytes) and add headroom. Under-requesting memory leads to eviction or OOM kills, so memory gets the more conservative margin.
Requests come from kube_pod_container_resource_requests in kube-state-metrics. A Prometheus monitoring stack collects all three.
The JVM caveat
Java services complicate memory rightsizing. The JVM sizes its default maximum heap as a percentage of the container’s memory limit and rarely returns heap to the operating system, so the observed working set reflects the heap setting rather than what the application needs. Rightsize the heap flags (-Xmx or -XX:MaxRAMPercentage) together with the request, and watch garbage collection time after each change.
Automating it with VPA
The Vertical Pod Autoscaler (VPA) recommends requests from observed usage and can apply them. Its update modes decide how far it goes:
| Mode | What it does |
|---|---|
Off |
Computes recommendations only. Nothing changes. |
Initial |
Sets requests when a Pod is created, never afterwards. |
Recreate |
Evicts Pods whose requests differ significantly from the recommendation, so they restart with new values. |
InPlaceOrRecreate |
Resizes running Pods in place and falls back to eviction if that fails. |
Auto |
Deprecated. Currently behaves like Recreate. |
Many teams run VPA only in Off mode and apply the recommendations through their normal deployment process. Automatic modes restart Pods, and the VPA documentation says it should not be combined with a Horizontal Pod Autoscaler that scales on the same CPU or memory metric. In-place Pod resize, stable since Kubernetes v1.35, removes most of the restart cost for CPU, though memory changes often still need a container restart because many runtimes cannot shrink their memory use on the fly.
One-off vs continuous rightsizing
A one-off rightsizing exercise starts decaying the next week. New releases change memory profiles, traffic grows and shrinks, and new services arrive with copied defaults. Continuous rightsizing re-evaluates requests on a schedule or automatically, and treats requests as something to tune like any other configuration.
Rightsizing has to shrink the node count
Lowering requests frees capacity on paper. The bill only drops when the cluster then runs fewer or smaller nodes, which requires node autoscaling or consolidation that moves Pods together and removes the emptied nodes. See how to reduce Kubernetes costs for how rightsizing fits with the other levers, and how GPUs work in Kubernetes for why GPU requests follow different rules: GPUs are requested in whole devices unless you use sharing.
How Cloudfleet handles the node side
Cloudfleet Kubernetes Engine (CFKE) uses node auto-provisioning to size nodes to the requests of pending Pods and delete nodes that are no longer needed. Its consolidation profiles decide whether it also evicts running Pods to repack them onto fewer or smaller nodes once requests go down.