What is edge Kubernetes and how do you run it across many sites?
Edge Kubernetes is Kubernetes running outside the data center, at stores, factories, warehouses, vehicles, cell sites or branch offices, usually on one to a few small machines per location. There are two main architectures: a complete cluster at every site, managed together as a fleet, or worker nodes at each site joined to one central control plane. The choice depends on what a site must still be able to do when its network link is down.
Why run Kubernetes at the edge
- Latency. Machine vision on a production line or a point-of-sale system cannot wait for a round trip to a cloud region.
- Bandwidth. Video and sensor data are filtered and aggregated on site, and only results travel over the WAN.
- Data location. Some data must stay in the building or the country where it is produced.
- One deployment model. The same container images, manifests and pipelines deploy to the cloud and to hundreds of sites.
Two architectures
| Aspect | A cluster per site | Central control plane, nodes at the sites |
|---|---|---|
| Hardware per site | Control plane and workers, often on the same machines | Worker nodes only |
| When the WAN link drops | The site keeps scheduling, restarting and scaling locally | Running Pods keep running; nothing new is scheduled until the link returns |
| Control planes to upgrade and back up | One per site, so hundreds | One |
| Management | Fleet management tooling and GitOps across clusters | One cluster, one API, standard scheduling with node labels |
| Fits | Sites that must change workloads while offline for long periods | Sites with mostly reliable connectivity that need central control |
With a central control plane, behavior during an outage follows Kubernetes defaults. The node controller marks a node unhealthy after it has not reported for 50 seconds, and Pods carry a default toleration of 300 seconds for unreachable nodes before the control plane tries to replace them elsewhere. Containers on the disconnected node keep running throughout, because the kubelet does not need the control plane to keep them alive. Tolerations, affinity and DaemonSets control whether replacements should appear at other sites at all. What happens when the control plane connection drops goes through each case.
Managing many separate clusters is a discipline of its own, covered in Kubernetes fleet management.
Lightweight Kubernetes for small hardware
Edge machines are often small: industrial PCs, ARM boards or a single server in a closet. Several projects target them:
- k3s is a CNCF sandbox project packaged as a single binary. Its documented minimum is 2 CPU cores and 2 GB of RAM for a server node and 1 core and 512 MB for an agent node, on x86_64, armhf and arm64. See what k3s is.
- MicroK8s from Canonical is installed as a snap package and supports single-node and small multi-node clusters.
- KubeEdge splits Kubernetes into a cloud part and a lightweight edge agent, EdgeCore, and is designed so that edge nodes and their applications keep running when the network is unstable or the node restarts offline. It also manages IoT devices. KubeEdge became a CNCF graduated project in October 2024.
Operating hundreds of sites
What works for three clusters breaks at three hundred. The practices that scale:
- Zero-touch provisioning. A machine shipped to a site boots a prepared image, registers itself with the control plane or fleet manager using a token, and receives its workloads without anyone on site typing commands.
- Pull-based deployment. Each site pulls its desired state from Git or a central API. This works over outbound-only connections behind NAT and catches up after an outage. GitOps describes the pattern.
- Staged rollouts. Release to a few canary sites, watch, then widen. A bad release at every site at once is the edge version of a full outage.
- Recoverable operating system updates. Image-based operating systems with A/B partitions roll back automatically if an update fails to boot, which avoids a technician visit.
- Local buffering. Applications queue data locally while the WAN is down and forward it when the link returns.
Security of physically exposed hardware
Edge machines sit in places where people can touch them. Encrypt disks with keys tied to a TPM, enable secure boot, avoid long-lived credentials on the device, and keep sites from accepting inbound connections. Network policies limit what a compromised workload can reach, and a lost device should be revocable centrally.
How Cloudfleet fits
Cloudfleet Kubernetes Engine (CFKE) follows the central control plane model: an edge machine running Linux joins the cluster as a node with only an outbound internet connection, works behind NAT, and talks to the control plane and other nodes over an encrypted WireGuard® overlay. If a site loses its uplink, its Pods keep running and the cluster applies changes again when the link returns. See self-managed nodes.