fleet Documentation

Affinity and anti-affinity

For a variety of reasons a service or container should only run on a specific type of hardware. Maybe one machine has faster disk speeds, another is running a conflicting application, and yet another is part of your bare-metal cluster. Each of these would be a good reason to run an application here instead of there, and being able to control where an application gets run can make all of the difference.

nodeSelector Example

Let's say we're running a Kubernetes cluster on AWS and we want to run an Nginx pod on m3.medium instance. We also want a different httpd pod to only run in the us-west-2 region and only on nodes with the example.com/load-balancer key set to true. Here's how that mypods.yaml would look:

If we add the example.com/load-balancer=true key to one of our nodes, the nginx pod will get scheduled to that node.
Once we have the example.com/load-balancer key set to true, the nginx pod will be scheduled.

When a node label changes, pods are not moved. We can demonstrate this by changing the example.com/load-balancer key on ip-10-0-0-187.us-west-2.compute.internal to false, then adding the example.com/load-balancer=true label to our other worker node to see what happens:

As you can see, both pods keep running on the same host, because the initial httpd pod isn't moved or unscheduled. Later httpd pods will be assigned to the second node, according to the new example.com/load-balancer=true label.

Affinity

Kubernetes also has a more nuanced way of setting affinity called nodeAffinity and podAffinity. These are fields in under Pod metadata and take automatic or user-defined metadata to dictate where to schedule pods. affinity differs from nodeSelector in the following ways:

Schedule a pod based on which other pods are or are not running on a node.

Request without requiring that a pod be run on a node.

Specify a set of allowable values instead of a single value requirement.

Affinity selector

Requirements met

Requirements not met

Requirements lost

requiredDuringSchedulingIgnoredDuringExecution

Runs

Fails

Keeps Running

preferredDuringSchedulingIgnoredDuringExecution

Runs

Runs

Keeps Running

(un-implemented) requiredDuringSchedulingRequiredDuringExecution

Runs

Fails

Fails

In addition to affinity/anti-affinity for specific nodes nodeAffinity

nodeAffinity example

Lets take the above example of deploying a nginx and a httpd pod, except we have a more complicated set of requirements:

nginx cannot run on the same node as httpd

httpdshould run on a node with the x-web:yes label, but can run anywhere.