ECS Autoscaling

The topic of ECS autoscaling is a vast area of heated discussions and broken dreams. It is quite hard to
come up with efficient scaling policies for your ECS services. And the more distributed your architecture,
the more issues with cascading load and increasing latency you are going to face. But fear not, the
promised salvation in form of autoscaling for your services is here to save the day and distribute your
computing load evenly across your micro services. So lets examine what we have to work with to achieve that.

Scaling services

Autoscaling of ECS services is implemented as an automated action executed upon an event: scale in or scale out. The source of such event can be either an alarm with StepScaling type of policy or TargetTrackingScaling type. Usage of the target tracking is very similar to the implementation for DynamoDB, with selection of ECSServiceAverageCPUUtilization and ECSServiceAverageMemoryUtilization metrics available for tracking. Notice that ECS can track only average metrics of the service, so it means you need to make sure that tasks have load distributed evenly on load balancer. Significant gaps between maximum and average consumption can lead to a termination of a task with out of memory or CPU and lead to 502 errors.

New and Old

ECS scaling policies can be combined to produce even greater efficiency in load distribution. Usage of the StepScaling policies to handle scale out events on ALB or SQS metrics by estimating the load in the input source. ALB’s Target Group metrics such as AWS/ApplicationELB/RequestCountPerTarget are good baseline to start policies. Size of the SQS queue is another example of deterministic metric to estimate incoming load for service.
Combination of StepScaling and TargetTrackingScaling looking at ECSServiceAverageCPUUtilization or ECSServiceAverageMemoryUtilization can allow greater flexibility in the how your service can react on load. If it is possible to determine of the service in question is mostly CPU or memory bound, then selection of a threshold for one of these average metrics should be pretty easy by observing the service under generated test load.

CloudFormation support for ECS scaling

To define an ECS service with scaling policies in CloudFormation you need to have a cluster, instance role for EC2 hosts and other essentials omitted from this example.

First we need a service role to perform scaling actions on our behalf.

Now we’re going to have a look at a service definition, its target group for ALB, scaling targets and policies and a CloudWatch alarm. For this example we are going to define ExampleCPUAutoScalingPolicy for a new capacity to grow to a value so that current usage ECSServiceAverageCPUUtilization accounts for 50% and ExampleRequestsAutoScalingPolicy when we have more than 1000 requests per target in a minute.

Notice that parameters section of the ExampleCPUAutoScalingPolicy resource contains DisableScaleIn: true for a specific reason. In order to guarantee that requests per target scaling events have priority over target tracking, the scale in logic of tracking can be disabled completely.

Stability, stability is the key

Ok, so now we have the service scaling up and down based on the number of requests per target in ELB. However, you would notice that the threshold in StepAdjustments for scale up starts right after scale down. It means that your service’s desired count would oscillate around some value, going up and down with new tasks spun up. To allow for a window of stability, you need to have a range with ScalingAdjustment: 0, where you would have a boundary to increase & decrease desired count. That way it is possible to make alarm to alert on the scale in boundary, and StepAdjustments to interpret the range. Lets see the example, where we want to scale out on more than RequestsScaleOutThreshold requests per target, and scale in on less than RequestsScaleInThreshold:

Here we have a range between MetricIntervalLowerBound=RequestsScaleInThreshold and MetricIntervalUpperBound=RequestsScaleOutThreshold where ScalingAdjustment=0 and no changes are done to desired count. This will ensure that oscillation of desired count does not happen for you.

Only one? Take two!

Another approach would be to define to alarms, one to scale out and one to scale in. Each would have a specific range and specific policy associated. Such approach in fact is used quite a lot, but the problem is that CloudWatch alarms are not free, in fact they are pretty expensive.

Recent Posts

About

Hi, I am Stan and I do software. I have blog where I (sometimes) post about what I do or my interests. I enjoy making various tools and libraries and sharing them with community. You can find a list of things I did here.