How do you upgrade a Kubernetes cluster safely?
A safe Kubernetes upgrade moves one minor version at a time, upgrades the control plane before the nodes, and starts only after you have removed every API the target version no longer serves. Then you replace or drain nodes in small batches, with PodDisruptionBudgets protecting each application’s availability. Because a control plane upgrade is hard to undo, a tested backup and a staging cluster on the target version are part of the procedure, not extras.
The order of an upgrade
The Kubernetes upgrade guide and the version skew policy set the order:
- Control plane first. On a self-managed cluster, that means etcd, then every
kube-apiserver, thenkube-controller-manager,kube-schedulerand the cloud controller manager. On a managed service, the provider does this in one operation. - Nodes next. kubelets may be up to three minor versions older than the API server, but never newer, so nodes always follow the control plane.
- Clients and manifests last. Update
kubectl, CI tooling and manifests to match the new API.
The API server must not skip minor versions, and the kubeadm upgrade guide states that skipping minor versions is unsupported. To go from 1.33 to 1.36, you upgrade three times. Install the latest patch release of your current version first, then move to the latest patch of the next one.
Before you start: find removed APIs
Removed APIs cause most failed upgrades. Kubernetes deprecates API versions and later stops serving them, and objects or manifests that still use them fail. Past examples include the batch/v1beta1 CronJob API and PodSecurityPolicy, both gone in 1.25. The deprecated API migration guide lists every removal by version.
To find what you still use:
- Ask the API server. It returns warnings to clients that call deprecated APIs, exposes the
apiserver_requested_deprecated_apismetric, and marks such requests in the audit log. - Scan manifests and Helm releases. Pluto checks code repositories and Helm releases. kube-no-trouble (kubent) checks a live cluster.
- Check your provider. Managed services run their own checks, and some block an upgrade when they detect recent calls to removed APIs.
Fix the sources, not only the live objects: Helm charts, GitOps repositories and operators will recreate old versions if you leave them. kubectl convert can rewrite a manifest to a newer API version.
Check add-on compatibility
Your cluster runs more than Kubernetes. Each of these components supports a specific range of Kubernetes versions, so check its release notes against the target version:
- the CNI plugin, such as Cilium, Calico or a cloud provider’s plugin,
- CSI storage drivers and snapshot controllers,
- the ingress or Gateway API controller,
- cert-manager, external-dns, the metrics server and autoscalers,
- operators and admission webhooks, including policy engines.
Some add-ons need upgrading before the control plane moves, others after. Admission webhooks that fail on the new version can block every Pod creation in the cluster.
Strategies for upgrading nodes
Drain a node before a minor kubelet upgrade: the skew policy does not support upgrading the kubelet’s minor version with Pods still running on it. Three approaches are common:
| Strategy | How it works | Trade-off |
|---|---|---|
| In-place, rolling | Cordon and drain one node, upgrade the kubelet, return it to service, repeat. | No extra capacity needed, but each node is out of service during its upgrade. |
| Surge or replacement | Add new nodes on the new version, drain old nodes onto them, then delete the old nodes. | Fast and easy to roll back by keeping the old nodes, but needs temporary extra capacity. |
| Blue-green node pools | Create a complete new node pool, shift workloads, remove the old pool. | Cleanest rollback path, at the cost of running two pools for a while. |
Drains respect PodDisruptionBudgets. A PDB such as minAvailable: 2 for a three-replica Deployment stops a drain from evicting too many Pods at once. Check that every critical application has one, and that none is so strict that it blocks drains forever, such as maxUnavailable: 0. Applications with a single replica will have downtime during a drain whatever you configure.
Plan for rollback
Kubernetes upgrades have traditionally been one-way. Earlier API server versions cannot always read data written by newer ones, so a control plane could not be moved back to the previous minor version. Upstream now has compatibility versions, which let new binaries emulate the previous version’s behavior, and some providers build rollback on top of it. Amazon EKS allows a rollback to the previous minor version within 7 days of an in-place upgrade. Treat these as a safety net and plan as if the upgrade cannot be reversed:
- Back up etcd and your cluster resources, and test a restore. kubeadm keeps local etcd and manifest backups under
/etc/kubernetes/tmpduring an upgrade, but they are no substitute for an off-cluster backup. - Upgrade a staging cluster first, with the same add-ons and real manifests, and run your tests against it.
- Upgrade production in stages, cluster by cluster, starting with the least critical.
How often to upgrade
Kubernetes ships a new minor version roughly every four months and patches each one for about 14 months. Upgrading once per release keeps each step small: one set of API removals, one set of add-on changes. Teams that wait a year face several consecutive upgrades and, on managed services, extended support fees. How long Kubernetes versions are supported lists the windows and fees per provider. For node-level patching between minor upgrades, see immutable operating systems for Kubernetes nodes, and for keeping many clusters on the same schedule, see Kubernetes fleet management.
Upgrades on Cloudfleet
Cloudfleet upgrades the control plane with a rolling update, so the API stays available, and leaves existing nodes untouched. Nodes pick up the new version when node auto-provisioning replaces them, or when you drain them. See Kubernetes versions and upgrades.