Public cloud load balancing

This page describes Kubernetes Services and their use in Cloudfleet Kubernetes Engine (CFKE). Kubernetes have different types of Services, which can be used to group a set of Pod endpoints into a single resource. Before reading this page, it is recommended to read the Kubernetes documentation on Services.

A Service is designed to aggregate a collection of Pod endpoints into one resource. You can set up multiple methods to access this aggregation. Typically, a stable cluster IP address is provided, which clients within the cluster can use to reach the Pods in the Service. When a client sends a request to this stable IP address, it is directed to one of the Pods in the Service. There are five types of Services:

• ClusterIP (default)
• NodePort
• LoadBalancer
• ExternalName
• Headless

ClusterIP, ExternalName and Headless type of services work same in CFKE as in any other Kubernetes cluster. Kubernetes documentation provides a good overview of these types of services.

Node Port

When you create a Service with the NodePort type, CFKE opens a static port on the cluster’s nodes. You can access the Service using the node’s public IP address and the port number. See the Kubernetes documentation for more information.

TCP Load Balancing

Cloudfleet supports provisioning TCP load balancers in some of the cloud providers supported by the Node Auto-provisioning. This feature provisions and manages TCP Load Balancers across different cloud providers and regions using the standard implementation of Kubernetes Services.

Please find the current status of the TCP Load Balancing features in the table below:

When creating a Service with the LoadBalancer type, many cloud providers will automatically provision a load balancer implementation of theirs to route traffic to the Service in the region where the cluster is deployed. This is a valid architecture for classical cloud providers because their clusters contain compute nodes that are only placed in one specific region. However, CFKE’s architecture is different in a way that the compute nodes can be placed in different cloud providers and regions.

In CFKE, when you create a Service with the LoadBalancer type, by default, CFKE will provision a load balancer in every cloud provider and region where at least one node exists. The IP addresses of these Load Balancers are shared in the Service LoadBalancer Ingress field and you can expect to see multiple IP addresses in this field. This allows you to serve traffic from all the different cloud providers and regions where the cluster is present.

If you want to avoid the extra costs of multiple Load Balancers and the extra network hops that may be caused by routing traffic through the in-cluster network, you can set the External Traffic Policy to Local. To learn more about this, see the Load balancing in multi-cloud environment section.

Supported annotations

It is possible to configure the load balancing configuration using annotations. The following annotations on the Service object are supported:

Example: Exposing a Service to using a TCP Load Balancer

To start with Load Balancing, make sure that you have a deployment that has Pods deployed on auto-provisioned nodes from supported Cloud Providers.

To create an external load balancer, add the following line to your Service manifest:

   type: LoadBalancer

Your manifest might then look like:

apiVersion: v1
kind: Service
metadata:
 name: example-service
spec:
 selector:
   app: example
 ports:
   - port: 80
     targetPort: 80
 type: LoadBalancer
 externalTrafficPolicy: Local

Setting up DNS for your Load Balancer Service

You can find the IP address created for your service by getting the service information through kubectl:

kubectl describe services example-service

which should produce output similar to:

Name:                     example-service
Namespace:                default
Labels:                   app=example
Annotations:              <none>
Selector:                 app=example
Type:                     LoadBalancer
IP Families:              <none>
IP:                       10.3.22.96
IPs:                      10.3.22.96
LoadBalancer Ingress:     142.132.244.132 (VIP), 2a01:4f8:c011:2c4::1 (VIP)
Port:                     <unset>  80/TCP
TargetPort:               80/TCP
NodePort:                 <unset>  30593/TCP
Endpoints:                172.17.0.3:9376
Session Affinity:         None
External Traffic Policy:  Local
Events:                   <none>

You can see the IP addresses of the load balancer in the LoadBalancer Ingress field. In most cases, you will see multiple IP addresses in this field, one for each cloud provider and region where at least one node exists. Moreover, the IP addresses shown in LoadBalancer Ingress can change when:

• The Pods backing the Load Balancer service are rescheduled to different regions or cloud providers
• The externalTrafficPolicy is set to Local and pod placement changes

This occurs because cloud providers typically support one load balancer per region, and if Cloudfleet decides to move the pods to a different region or cloud provider to optimize resources, it needs to create a new load balancer in the new region or cloud provider and remove the old one. This means that using IP addresses directly to point to your final DNS record is not the best practice. Cloudfleet provides a stable DNS name for each Load Balancer service that you can use in your DNS configuration. The format of the DNS name is as follows:

<service-name>.<namespace>.<cluster-id>.<control-plane-region>.cfke.cloudfleet.dev

For example, if your cluster is hosted in the europe-central-1a region, has 79f9275a-eec6-4dd9-af0a-78604eacb773 as its ID, and the example-service service is in the default namespace, the DNS name for this service will be:

example-service.default.79f9275a-eec6-4dd9-af0a-78604eacb773.europe-central-1a.cfke.cloudfleet.dev

To minimize the recreation of load balancers, you can consider locking the Pods serving a load balancer to a specific cloud provider and region by specifying a node selector. For example, to lock the Pods to the Hetzner Nürnberg region, you can use the following node selector:

spec:
  template:
    spec:
      nodeSelector:
        cfke.io/provider: hetzner
        topology.kubernetes.io/region: nbg1
      # Your pod specification

This configuration will ensure that the load balancer is only created in this specific region. However, this Pod will also be able to reach other Pods in the cluster using the in-cluster network.

Load balancing in multi-cloud environment

“LoadBalancer Ingress” contains the IPv4 and IPv6 addresses of the Load Balancers created for your service.

Unlike other Kubernetes solutions that typically only handle load balancing for a single cloud provider’s region, Cloudfleet supports multiple cloud providers and regions under one cluster. This affects how Load Balancers are provisioned.

Kubernetes Load Balancer Services have a setting called “External Traffic Policy” that can be set to either “Local” or “Cluster.” When set to “Cluster,” the service is exposed on all the nodes of the cluster. Even if there is no Pod on a specific node that serves the service, traffic is routed using the in-cluster network to a serving Pod. When set to “Local,” the service is only exposed on the nodes that have a Pod belonging to the service.

In a Cloudfleet cluster, you may have nodes from different clouds and different regions. When the service’s External Traffic Policy is set to Cluster, Cloudfleet tries to fulfill the requirement of exposing the service on all nodes in the cluster; therefore, it provisions Load Balancers in every cloud provider and every region where at least one node exists. The IP addresses of these Load Balancers are shared in the Service LoadBalancer Ingress field.

Although this mode provides maximum flexibility and enables you to serve traffic from all the different cloud providers and regions where the cluster is present, it also presents a few trade-offs:

• Multiple Load Balancers may cause extra costs
• If a node does not directly serve a Pod, the traffic is routed through the in-cluster network. This adds extra network hops and may also strain the in-cluster network, which is encrypted.

To avoid these risks, you can set the External Traffic Policy to Local. In this mode, Cloudfleet provisions Load Balancers only in cloud providers and regions where Pods belonging to this service are actually running. However, the disadvantage is that if the Pods move to another cloud or another region within the same cloud, new Load Balancers are created for the new situation, and the IP addresses will change. Users may want to use solutions like External DNS to keep an updated list of IP addresses in this case.

Ingress and Gateway API support

LoadBalancer type of services are running on L3 level. This means that they are not aware of the HTTP/HTTPS protocol. To have HTTP/HTTPS aware load balancers, you need to use Ingress or Gateway API resources which are native Kubernetes APIs.

CFKE does not support Ingress and Gateway API resources out of the box. The reason is, different customers have different preferences to support these resources. Users are free to use any Ingress Controller or Gateway API implementation they want.

NGINX Ingress Controller or Traefik are two of many solutions that are tested by both Cloudfleet team and the community. For a complete step-by-step guide on installing NGINX Ingress Controller with automatic HTTPS certificate management using cert-manager, see the Install NGINX Ingress Controller and cert-manager tutorial.