Node auto-provisioner

CFKE manages the infrastructure provisioning on behalf of its users. This means that users are not required to create and right-size new compute nodes, deleting unused ones, or managing upgrades. Cloudfleet has a built-in node auto-provisioner that calculates the optimal number and size of nodes based on the current workload. This allows users to focus on their applications rather than on the infrastructure. Customers can influence the node provisioner’s decisions through Kubernetes Pod labels, Pod resource requests, and affinity rules. The node provisioner determines the required node topology and the optimal node sizes, and provisions them accordingly. When the nodes are no longer needed, the node provisioner deletes them. If a node becomes unhealthy or is destroyed by the cloud provider due to spot instance preemption, the node auto-provisioner replaces it with a new one.

If your pods don’t have specific requirements for how or where to run, CFKE can select nodes from any configured cloud provider in your Fleets, always choosing the most cost-effective instance type. However, you can use the node auto-provisioner’s layered constraints model to ensure that your pods have the exact resources and type they need. Reasons for constraining where your pods run could include needing to run in zones with specific dependencies, requiring certain hardware, or wanting to use techniques like topology spread for high availability.

Node auto-provisioner kicks-in when a Pod is created but there is no available node to run it in the cluster. In this case the Pod stays in Pending state. Node auto-provisioner detects this and provisions a new node that meets the Pod’s requirements. Once the node is ready, the Pod is scheduled on it.

The Cloud Providers in your Fleets define the initial constraints for the nodes to provisioned, including all available instance types, CPU architectures, zones, and purchase types. Additional constraints come from the workload specifications of your Kubernetes pods. Pod scheduling constraints must fall within a Fleet’s constraints; otherwise, CFKE won’t find the right nodes and pods will not deploy. For example, if a Fleet has a limit of certain number of vCPUs, scheduling one more pod that would exceed this limit will not be possible.

Constraints you can request in a Pod request include:

• Resource requests: Request that a certain amount of memory or CPU be available.
• Node selection: Choose to run on a node that has a particular label (nodeSelector).
• Node affinity: Draws a pod to run on nodes with particular attributes (affinity).
• Topology spread: Use topology spread to help ensure availability of the application.
• Pod affinity/anti-affinity: Draws pods towards or away from topology domains based on the scheduling of other pods.

CFKE adheres to standard Kubernetes scheduling constraints, which enables you to establish a uniform set of rules that govern both existing and provisioned capacity.

Additionally, CFKE supports specific Well-Known Labels, Annotations, and Taints that are beneficial for scheduling.

Resource requests

Within a Pod spec, you can both make requests and set limits on resources a pod needs, such as CPU and memory. For example:

apiVersion: v1
kind: Pod
metadata:
  name: example-app
spec:
  containers:
  - name: example-app
    image: nginx
    resources:
      requests:
        memory: "256Mi"
        cpu: "1000m"
      limits:
        memory: "512Mi"
        cpu: "2000m"

In this example, the container is requesting 256MiB of memory and 1 CPU. Its limits are set to 512MiB of memory and 2 CPUs. In this case, CFKE’s node auto-provisioner algorithm only uses requests to find out the instance type to accommodate this pod, but limits may also be configured to enable resource oversubscription.

See Managing Resources for Containers for details on resource types supported by Kubernetes, Specify a memory request and a memory limit for examples of memory requests..

Accelerators/GPU Resources

CFKE supports the following accelerator (e.g., GPU) values:

• nvidia.com/gpu

Additionally, include a resource requirement in the workload manifest. This will cause the GPU dependent pod to be scheduled onto the appropriate node.

Here is an example of an accelerator resource in a workload manifest (e.g., pod):

spec:
  template:
    spec:
      containers:
      - resources:
          limits:
            nvidia.com/gpu: "1"

If an instance with NVIDIA GPU is scheduled, CFKE installs the NVIDIA drivers on the node. Please check the GPU-based workloads documentation for more information.

Selecting nodes

With nodeSelector you can ask for a node that matches selected key-value pairs. This can include well-known labels or custom labels you create yourself.

You can use affinity to define more complicated constraints, see Node Affinity for the complete specification.

Labels

Well-known labels may be specified as pod scheduling constraints. You can also select them using nodeAffinity or nodeSelectors on your Pods.

Well-Known Labels

CFKE translates the following deprecated labels to their stable equivalents: failure-domain.beta.kubernetes.io/zone, failure-domain.beta.kubernetes.io/region, beta.kubernetes.io/arch, beta.kubernetes.io/os, and beta.kubernetes.io/instance-type.

You can find all the possible values for these labels in the Reference table in the end of this documentation.

Node selectors example

Here is an example of a nodeSelector for selecting nodes:

nodeSelector:
  cfke.io/provider: aws
  topology.kubernetes.io/zone: us-west-2

This example features a well-known label (topology.kubernetes.io/zone) to schedule a instance in AWS’s us-west-2 region.

Preferences

CFKE usually treats pod affinity, pod anti-affinity, pod topology, and node affinity as requirements. It considers these preferences when deciding if a pod can be scheduled or moved to a new node.

CFKE initially considers preferred affinities as required affinities when building requirements for a pod. If these requirements aren’t met, it relaxes the pod’s preferences one by one in ascending weight order, retrying the remaining requirements each time.

However, CFKE doesn’t treat preferred affinities as required when building topology requirements for scheduling to a node. In such cases, required affinities should be used as documented in Node Affinity.

Node affinity

Examples below show how to use Node affinity rules to include (In) and exclude (NotIn) objects. See Node affinity for more details. When setting rules, the following Node affinity types define how hard or soft each rule is:

• requiredDuringSchedulingIgnoredDuringExecution: This is a hard rule that must be met.
• preferredDuringSchedulingIgnoredDuringExecution: This is a preference, but the pod can run on a node where it is not guaranteed.

CFKE may create more nodes than you anticipate if you set preferred affinities on pods. This is because CFKE prioritizes creating new nodes to meet your preferences.

The IgnoredDuringExecution part of each tells the pod to keep running, even if conditions change on the node so the rules no longer match. You can think of these concepts as required and preferred, since Kubernetes never implemented other variants of these rules.

All examples below assume that the NodePool doesn’t have constraints to prevent those zones from being used. The first constraint says you could use us-west-2a or us-west-2b, the second constraint makes it so only us-west-2b can be used.

 affinity:
   nodeAffinity:
     requiredDuringSchedulingIgnoredDuringExecution:
       nodeSelectorTerms:
         - matchExpressions:
           - key: "topology.kubernetes.io/zone"
             operator: "In"
             values: ["us-west-2a", "us-west-2b"]
           - key: "topology.kubernetes.io/zone"
             operator: "In"
             values: ["us-west-2b"]

Changing the second operator to NotIn would allow the pod to run in us-west-2a only:

           - key: "topology.kubernetes.io/zone"
             operator: "In"
             values: ["us-west-2a", "us-west-2b"]
           - key: "topology.kubernetes.io/zone"
             operator: "NotIn"
             values: ["us-west-2b"]

Continuing to add to the example, nodeAffinity lets you define terms so if one term doesn’t work it goes to the next one. Here, if us-west-2a is not available, the second term will cause the pod to run on a spot instance in us-west-2d.

 affinity:
   nodeAffinity:
     requiredDuringSchedulingIgnoredDuringExecution:
       nodeSelectorTerms:
         - matchExpressions: # OR
           - key: "topology.kubernetes.io/zone" # AND
             operator: "In"
             values: ["us-west-2a", "us-west-2b"]
           - key: "topology.kubernetes.io/zone" # AND
             operator: "NotIn"
             values: ["us-west-2b"]
         - matchExpressions: # OR
           - key: "karpenter.sh/capacity-type" # AND
             operator: "In"
             values: ["spot"]
           - key: "topology.kubernetes.io/zone" # AND
             operator: "In"
             values: ["us-west-2d"]

CFKE will evaluate each of the nodeSelectorTerms in sequence and select the first one that fulfills the requirements. If CFKE is unable to provision using the first nodeSelectorTerms, it will attempt to use the subsequent ones. If all nodeSelectorTerms fail, CFKE will not be able to provision the pod. CFKE will then back off and retry over a period of time. If capacity becomes available at a later time, it will schedule the pod without requiring any action from the user.

Topology spread

TopologySpreadConstraints in Kubernetes can be used to spread pods away from each other within a cluster to minimize the impact of an outage. This feature can be considered as an enhancement of pod affinity, as it enables the association of pods with respect to nodes while still allowing for distribution.

Note that using Preferred topology spread (ScheduleAnyway) might lead to the creation of more nodes than anticipated. CFKE prioritizes the creation of new nodes to meet spread constraints. Refer to the preferences documentation for more information.

For example:

spec:
  topologySpreadConstraints:
    - maxSkew: 1
      topologyKey: "topology.kubernetes.io/zone"
      whenUnsatisfiable: ScheduleAnyway
      labelSelector:
        matchLabels:
          environment: dev
    - maxSkew: 1
      topologyKey: "kubernetes.io/hostname"
      whenUnsatisfiable: ScheduleAnyway
      labelSelector:
        matchLabels:
          environment: dev

Adding this to your podspec would result in:

• Pods being spread across zones, hosts, and capacity-type (topologyKey).
• The dev labelSelector will include all pods with the label of environment=dev in topology calculations. It is recommended to use a selector to match all pods in a deployment.
• No more than one pod difference in the number of pods on each host (maxSkew). For example, if there were three nodes and five pods the pods could be spread 1, 2, 2 or 2, 1, 2 and so on. If instead the spread were 5, pods could be 5, 0, 0 or 3, 2, 0, or 2, 1, 2 and so on.

The two supported topologyKey values that CFKE supports are:

• topology.kubernetes.io/zone
• kubernetes.io/hostname

See Pod Topology Spread Constraints for details.

NodePools will adhere to zonal Topology Spread Constraints defined for Pods that require multi-zone durability, optimizing for compute costs. CFKE itself does not balance or rebalance availability zones for nodes.

Pod affinity/anti-affinity

The CFKE node auto-provisioner can be informed of your scheduling preferences by configuring podAffinity and podAntiAffinity on a pod spec. This allows you to indicate whether pods should be scheduled together or apart in relation to different topology domains.

Keep in mind that using preferred affinities on pods can lead to the creation of more nodes than anticipated. This is due to CFKE’s preference for creating new nodes to satisfy these preferences. For further information, refer to the preferences documentation.

For example:

spec:
  affinity:
    podAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
      - labelSelector:
          matchExpressions:
          - key: environment
            operator: In
            values:
            - production
        topologyKey: topology.kubernetes.io/zone
    podAntiAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
      - labelSelector:
          matchLabels:
            app: frontend
        topologyKey: kubernetes.io/hostname

The above pod affinity rule would cause the pod to only schedule in zones where a pod with the label environment=production is already running.

The anti-affinity rule would cause it to avoid running on any node with a pod labeled app=frontent. If this anti-affinity term was on a deployment pod spec along with a matching app=frontend label, it would prevent more than one pod from the deployment from running on any single node.

Check out Inter-pod affinity and anti-affinity in the Kubernetes documentation for details.

Scheduling examples

Below you can find some examples of how to use the node auto-provisioner to schedule your workloads based on Kubernetes Pod labels and affinity rules.

A deployment that is locked to a specific cloud provider and region

apiVersion: apps/v1
kind: Deployment
metadata:
    name: homepage
spec:
    selector:
        matchLabels:
            app: nginx
    replicas: 3
    template:
        metadata:
            labels:
                app: nginx
        spec:
            nodeSelector:
                cfke.io/provider: gcp
                topology.kubernetes.io/region: europe-west4
            containers:
                -   name: nginx
                    image: nginx:1.24
                    imagePullPolicy: Always
                    ports:
                        -   containerPort: 8080
                            name: http
                    resources:
                        requests:
                            cpu: 50m
                            memory: 56Mi
            terminationGracePeriodSeconds: 0

A deployment spread across clouds

apiVersion: apps/v1
kind: Deployment
metadata:
    name: homepage
spec:
    selector:
        matchLabels:
            app: nginx
    replicas: 3
    template:
        metadata:
            labels:
                app: nginx
        spec:
            affinity:
                podAntiAffinity:
                    requiredDuringSchedulingIgnoredDuringExecution:
                        -   labelSelector:
                                matchExpressions:
                                    -   key: app
                                        operator: In
                                        values:
                                            - nginx
                            topologyKey: cfke.io/provider
            containers:
                -   name: nginx
                    image: nginx:1.24
                    imagePullPolicy: Always
                    ports:
                        -   containerPort: 8080
                            name: http
                    resources:
                        requests:
                            cpu: 50m
                            memory: 56Mi
            terminationGracePeriodSeconds: 0

A deployment spread across regions within a cloud

apiVersion: apps/v1
kind: Deployment
metadata:
    name: homepage
spec:
    selector:
        matchLabels:
            app: nginx
    replicas: 3
    template:
        metadata:
            labels:
                app: nginx
        spec:
            nodeSelector:
                cfke.io/provider: gcp
            affinity:
                podAntiAffinity:
                    requiredDuringSchedulingIgnoredDuringExecution:
                        -   labelSelector:
                                matchExpressions:
                                    -   key: app
                                        operator: In
                                        values:
                                            - nginx
                            topologyKey: topology.kubernetes.io/region
            containers:
                -   name: nginx
                    image: nginx:1.24
                    imagePullPolicy: Always
                    ports:
                        -   containerPort: 8080
                            name: http
                    resources:
                        requests:
                            cpu: 50m
                            memory: 56Mi
            terminationGracePeriodSeconds: 0

A deployment spread across zones within a region

apiVersion: apps/v1
kind: Deployment
metadata:
    name: homepage
spec:
    selector:
        matchLabels:
            app: nginx
    replicas: 3
    template:
        metadata:
            labels:
                app: nginx
        spec:
            nodeSelector:
                cfke.io/provider: gcp
                topology.kubernetes.io/region: europe-west4
            affinity:
                podAntiAffinity:
                    requiredDuringSchedulingIgnoredDuringExecution:
                        -   labelSelector:
                                matchExpressions:
                                    -   key: app
                                        operator: In
                                        values:
                                            - nginx
                            topologyKey: topology.kubernetes.io/zone
            containers:
                -   name: nginx
                    image: nginx:1.24
                    imagePullPolicy: Always
                    ports:
                        -   containerPort: 8080
                            name: http
                    resources:
                        requests:
                            cpu: 50m
                            memory: 56Mi
            terminationGracePeriodSeconds: 0

Scheduling options Reference

Below you can find a list of well-known labels that CFKE auto node-provisinoner is aware of and can be used in your workloads.