Public cloud load balancing

This page describes Kubernetes Services and their use in Cloudfleet Kubernetes Engine (CFKE). Kubernetes have different types of Services, which can be used to group a set of Pod endpoints into a single resource. Before reading this page, it is recommended to read the Kubernetes documentation on Services.

A Service is designed to aggregate a collection of Pod endpoints into one resource. You can set up multiple methods to access this aggregation. Typically, a stable cluster IP address is provided, which clients within the cluster can use to reach the Pods in the Service. When a client sends a request to this stable IP address, it is directed to one of the Pods in the Service. There are five types of Services:

• ClusterIP (default)
• NodePort
• LoadBalancer
• ExternalName
• Headless

ClusterIP, ExternalName and Headless type of services work same in CFKE as in any other Kubernetes cluster. Kubernetes documentation provides a good overview of these types of services.

Node Port

When you create a Service with the NodePort type, CFKE opens a static port on the cluster’s nodes. You can access the Service using the node’s public IP address and the port number. See the Kubernetes documentation for more information.

TCP Load Balancing

Cloudfleet supports provisioning TCP load balancers in some of the cloud providers supported by the Node Auto-provisioning. This feature provisions and manages TCP Load Balancers across different cloud providers and regions using the standard implementation of Kubernetes Services.

Please find the current status of the TCP Load Balancing features in the table below:

When creating a Service with the LoadBalancer type, many cloud providers will automatically provision a load balancer implementation of theirs to route traffic to the Service in the region where the cluster is deployed. This is a valid architecture for classical cloud providers because their clusters contain compute nodes that are only placed in one specific region. However, CFKE’s architecture is different in a way that the compute nodes can be placed in different cloud providers and regions.

In CFKE, when you create a Service with the LoadBalancer type, by default, CFKE will provision a load balancer in every cloud provider and region where at least one node exists. The IP addresses of these Load Balancers are shared in the Service LoadBalancer Ingress field and you can expect to see multiple IP addresses in this field. This allows you to serve traffic from all the different cloud providers and regions where the cluster is present.

If you want to avoid the extra costs of multiple Load Balancers and the extra network hops that may be caused by routing traffic through the in-cluster network, you can set the External Traffic Policy to Local. To learn more about this, see the Load balancing in multi-cloud environment section.

Supported annotations

It is possible to configure the load balancing configuration using annotations. The following annotations on the Service object are supported:

Example: Exposing a Service to using a TCP Load Balancer

To start with Load Balancing, make sure that you have a deployment that has Pods deployed on auto-provisioned nodes from supported Cloud Providers.

To create an external load balancer, add the following line to your Service manifest:

   type: LoadBalancer

Your manifest might then look like:

apiVersion: v1
kind: Service
metadata:
 name: example-service
spec:
 selector:
   app: example
 ports:
   - port: 80
     targetPort: 80
 type: LoadBalancer
 externalTrafficPolicy: Local

Finding your IP address

You can find the IP address created for your service by getting the service information through kubectl:

kubectl describe services example-service

which should produce output similar to:

Name:                     example-service
Namespace:                default
Labels:                   app=example
Annotations:              <none>
Selector:                 app=example
Type:                     LoadBalancer
IP Families:              <none>
IP:                       10.3.22.96
IPs:                      10.3.22.96
LoadBalancer Ingress:     142.132.244.132 (VIP), 2a01:4f8:c011:2c4::1 (VIP)
Port:                     <unset>  80/TCP
TargetPort:               80/TCP
NodePort:                 <unset>  30593/TCP
Endpoints:                172.17.0.3:9376
Session Affinity:         None
External Traffic Policy:  Local
Events:                   <none>

Load balancing in multi-cloud environment

“LoadBalancer Ingress” contains the IPv4 and IPv6 addresses of the Load Balancers created for your service.

Unlike other Kubernetes solutions that typically only handle load balancing for a single cloud provider’s region, Cloudfleet supports multiple cloud providers and regions under one cluster. This affects how Load Balancers are provisioned.

Kubernetes Load Balancer Services have a setting called “External Traffic Policy” that can be set to either “Local” or “Cluster.” When set to “Cluster,” the service is exposed on all the nodes of the cluster. Even if there is no Pod on a specific node that serves the service, traffic is routed using the in-cluster network to a serving Pod. When set to “Local,” the service is only exposed on the nodes that have a Pod belonging to the service.

In a Cloudfleet cluster, you may have nodes from different clouds and different regions. When the service’s External Traffic Policy is set to Cluster, Cloudfleet tries to fulfill the requirement of exposing the service on all nodes in the cluster; therefore, it provisions Load Balancers in every cloud provider and every region where at least one node exists. The IP addresses of these Load Balancers are shared in the Service LoadBalancer Ingress field.

Although this mode provides maximum flexibility and enables you to serve traffic from all the different cloud providers and regions where the cluster is present, it also presents a few trade-offs:

• Multiple Load Balancers may cause extra costs
• If a node does not directly serve a Pod, the traffic is routed through the in-cluster network. This adds extra network hops and may also strain the in-cluster network, which is encrypted.

To avoid these risks, you can set the External Traffic Policy to Local. In this mode, Cloudfleet provisions Load Balancers only in cloud providers and regions where Pods belonging to this service are actually running. However, the disadvantage is that if the Pods move to another cloud or another region within the same cloud, new Load Balancers are created for the new situation, and the IP addresses will change. Users may want to use solutions like External DNS to keep an updated list of IP addresses in this case.

Ingress and Gateway API support

LoadBalancer type of services are running on L3 level. This means that they are not aware of the HTTP/HTTPS protocol. To have HTTP/HTTPS aware load balancers, you need to use Ingress or Gateway API resources which are native Kubernetes APIs.

CFKE does not support Ingress and Gateway API resources out of the box. The reason is, different customers have different preferences to support these resources. Users are free to use any Ingress Controller or Gateway API implementation they want.

NGINX Ingress Controller or Traefik are two of many solutions that are tested by both Cloudfleet team and the community. You can find more information about these solutions in their respective documentation.