Aws-Cluster-Autoscaler (deprecated)

Share

No longer maintained. Use https://kubedex.com/resource/cluster-autoscaler/

The cluster autoscaler on AWS scales worker nodes within any specified autoscaling group. It will run as a Deployment in your cluster. This README will go over some of the necessary steps required to get the cluster autoscaler up and running. This chart bootstraps an aws-cluster-autoscaler deployment on a Kubernetes cluster using the Helm package manager.

Prerequisites

Kubernetes 1.3+ with Beta APIs enabled

Installing the Chart

In order for the chart to configure the aws-cluster-autoscaler properly during the installation process, you must provide some minimal configuration which can’t rely on defaults. This includes at least one element in the autoscalingGroups array and its three values: name, minSize and maxSize. These parameters cannot be passed to helm using the –set parameter at this time, so you must supply these using a values.yaml file such as:

autoscalingGroups:

– name: your-asg-name maxSize: 10 minSize: 1

To install the chart with the release name my-release:

$ helm install stable/aws-cluster-autoscaler –name my-release -f values.yaml The command deploys aws-cluster-autoscaler on the Kubernetes cluster using the supplied configuration. The configuration section lists the parameters that can be configured during installation.

Auto-Discovery Setup

To run a cluster-autoscaler which auto-discovers ASGs with nodes use the –node-group-auto-discovery flag. For example, –node-group-auto-discovery=asg:tag=k8s.io/cluster-autoscaler/enabled,k8s.io/cluster-autoscaler/<YOUR CLUSTER NAME> will find the ASGs where those tag keys exist. It does not matter what value the tags have.

Note that:

It is recommended to use the second tag like k8s.io/cluster-autoscaler/<YOUR CLUSTER NAME> when k8s.io/cluster- autoscaler/enabled is used across many clusters to prevent ASGs from different clusters recognized as the node groups There are no –nodes flags passed to cluster-autoscaler because the node groups are automatically discovered by tags No min/max values are provided when using Auto-Discovery, cluster-autoscaler will respect the current min and max values of the ASG being targeted, and it will adjust only the “desired” value.

Common Notes and Gotchas:

The /etc/ssl/certs/ca-certificates.crt should exist by default on your ec2 instance. If you use Amazon Linux 2, use /etc/ssl/certs/ca-bundle.crt instead.

Cluster autoscaler is not zone aware (for now), so if you wish to span multiple availability zones in your autoscaling groups beware that cluster autoscaler will not evenly distribute them.

By default, cluster autoscaler will not terminate nodes running pods in the kube-system namespace. You can override this default behaviour by passing in the –skip-nodes-with-system-pods=false flag.

By default, cluster autoscaler will wait 10 minutes between scale down operations, you can adjust this using the –scale- down-delay-after-add, –scale-down-delay-after-delete, and –scale-down-delay-after-failure flag. E.g. –scale-down- delay-after-add=5m to decrease the scale down delay to 5 minutes after a node has been added.

If you’re running multiple ASGs, the –expander flag supports three options: random, most-pods and least-waste. random will expand a random ASG on scale up. most-pods will scale up the ASG that will scheduable the most amount of pods. least-waste will expand the ASG that will waste the least amount of CPU/MEM resources. In the event of a tie, cluster autoscaler will fall back to random.