Edit This Page

Federated Cluster

This guide explains how to use Clusters API resource in a Federation control plane.

Different than other Kubernetes resources, such as Deployments, Services and ConfigMaps, clusters only exist in the federation context, i.e. those requests must be submitted to the federation api-server.

• Before you begin
• Listing Clusters
• Creating a Federated Cluster
• Deleting a Federated Cluster
• Labeling Clusters
• ClusterSelector Annotation
• Clusters API reference

Before you begin

• This guide assumes that you have a running Kubernetes Cluster Federation installation. If not, then head over to the federation admin guide to learn how to bring up a cluster federation (or have your cluster administrator do this for you). Other tutorials, such as Kelsey Hightower’s Federated Kubernetes Tutorial, might also help you create a Federated Kubernetes cluster.
• You should also have a basic working knowledge of Kubernetes in general.

Listing Clusters

To list the clusters available in your federation, you can use kubectl by running:

kubectl --context=federation get clusters

The --context=federation flag tells kubectl to submit the request to the Federation apiserver instead of sending it to a Kubernetes cluster. If you submit it to a k8s cluster, you will receive an error saying

the server doesn't have a resource type "clusters"

If you passed the correct Federation context but received a message error saying

No resources found.

it means that you haven’t added any cluster to the Federation yet.

Creating a Federated Cluster

Creating a cluster resource in federation means joining it to the federation. To do so, you can use kubefed join. Basically, you need to give the new cluster a name and say what is the name of the context that corresponds to a cluster that hosts the federation. The following example command adds the cluster gondor to the federation running on host cluster rivendell:

kubefed join gondor --host-cluster-context=rivendell

You can find more details on how to do that in the respective section in the kubefed guide.

Deleting a Federated Cluster

Converse to creating a cluster, deleting a cluster means unjoining this cluster from the federation. This can be done with kubefed unjoin command. To remove the gondor cluster, just do:

kubefed unjoin gondor --host-cluster-context=rivendell

You can find more details on unjoin in the kubefed guide.

Labeling Clusters

You can label clusters the same way as any other Kubernetes object, which can help with grouping clusters and can also be leveraged by the ClusterSelector.

kubectl --context=rivendell label cluster gondor key1=value1 key2=value2

ClusterSelector Annotation

Starting in Kubernetes 1.7, there is alpha support for directing objects across the federated clusters with the annotation federation.alpha.kubernetes.io/cluster-selector. The ClusterSelector is conceptually similar to nodeSelector, but instead of selecting against labels on nodes, it selects against labels on federated clusters.

The annotation value must be JSON formatted and must be parsable into the ClusterSelector API type. For example: [{"key": "load", "operator": "Lt", "values": ["10"]}]. Content that doesn’t parse correctly will throw an error and prevent distribution of the object to any federated clusters. Objects of type ConfigMap, Secret, Daemonset, Service and Ingress are included in the alpha implementation.

Here is an example ClusterSelector annotation, which will only select clusters WITH the label pci=true and WITHOUT the label environment=test:

  metadata:
    annotations:
      federation.alpha.kubernetes.io/cluster-selector: '[{"key": "pci", "operator":
        "In", "values": ["true"]}, {"key": "environment", "operator": "NotIn", "values":
        ["test"]}]'

The key is matched against label names on the federated clusters.

The values are matched against the label values on the federated clusters.

The possible operators are: In, NotIn, Exists, DoesNotExist, Gt, Lt.

The values field is expected to be empty when Exists or DoesNotExist is specified and may include more than one string when In or NotIn are used.

Currently, only integers are supported with Gt or Lt.

Clusters API reference

The full clusters API reference is currently in federation/v1beta1 and more details can be found in the Federation API reference page.