How to configure Auto-Scaling for Private Cloud

Purpose:

Have you checked the auto-scaling feature provided in public cloud solutions like AWS and Azure and wished to get the same feature in your private cloud environment? Do you have an existing private cloud environment or building a new one and want to make it auto-scale enabled? This post covers this exact topic. It details what auto-scaling is and provides step by step guide on how you can build one using various VMware products.

Introduction:

In recent months, during my interactions with customers, one requirement came up pretty often than others. That is of auto-scaling. Seems majority of customers who deploy Private Cloud require auto-scaling in some or other formats. Since out of the box vRealize Automation provides “Scale-Out” and “Scale-In” functions (albeit manual), these can be used in conjunction with other products to provide auto-scaling functionality. I had to configure this feature for multiple customers, so thought of writing a blog post detailing the steps. Readers can follow the blog to do it themselves. Also, auto-scaling is very dynamic in nature. Typically auto-scaling parameter requirement changes from environment to environment. Keeping that mind I have explained the steps involved so that you can customize it as per your need.

Required prior knowledge:

Though you can simply import the package in vRealize Orchestrator and follow the guide to configure other products, having a knowledge of the following will help you further configure it.

Working knowledge of vRealize Orchestrator

If you want to customize the workflows, then you need to know a bit of JavaScript

For configuration of Webhook Shims, basic knowledge of Linux will help (though not strictly required, John Dias did an amazing job providing step by step guide).

Familiarity with vRealize Operations Manager will help

Working knowledge of vRealize Automation is required

If you want to replicate my example of multi-tier application with application installation at runtime. Then you need to know NSX usage and advanced configuration of blueprints in vRealize Automation.

Acknowledgement:

Before I start writing this blog I need to say thanks to few people. Though I demonstrated this feature (with PowerCLI and vCenter) 2 years back to a customer, it was never a true auto-scaling solution. So, here it goes:

First and loudest thanks to Carsten Schaefer for com.vmware.pso.cemea.autoscaling package. It had the main “Scale Up Blueprint Layer based on VM MOID” and “Scale Down Blueprint Layer based on VM MOID”. All my other works are based on these two core workflows. These two workflows do the actual task. So thanks a lot mate for your hard work and help.

Thanks to Vishal Jain, Diwan Chandrabose, Ajay Kalla and team for the Load balancer handling script. Normally when an alert is fired, it is based on a VM. But when network load comes from Load-Balancer and it fires an alert, we get the load balancer name. The script co-relating the load balancer to the corresponding virtual machine is written by the team. They showed how we can use NSX and vROps integration to handle load balancer parameters. Thanks a lot guys for this.

Last but not least Vinith Menon, I was wondering how I would put a load on the test website. I was thinking of using JMeter. But it was too much to just put HTTP requests on a web page. Your one liner is absolutely fantastic and time saver for me. Thanks a lot brother for that.

My friend Vinith Menon also have written a blog post on auto-scaling. You can check it here.

Where to get the package for auto-scaling?

I have created a single vRealize Orchestrator package containing all the workflows, SNMP Policy and Action items. Download the package from the GitHub repository (https://github.com/sajaldebnath/auto-scaling-vra) and import it into vRealize Orchestrator server. Rest of the details are provided in the rest of the blog post.

Note, the first two workflows are core workflows (written by Carsten) all other workflows depends on these two to get the work done. If you are not using Webhook Shims, then you do not need to configure workflows which endds with “vROps REST Notification”. Also, for SNMP, you do not need to configure “Get VM Name from vROps REST Notification” and “JSON Invoke a REST operation” workflows. Alternately, if you are not going to use SNMP traps, then you do not need to configure the SNMP policy.

Pre-Requisites:

Before you can run everything you need to have the environment ready. I used the following versions:

vRealize Automation 7.3

vCenter & vSphere 6.5

vRealize Operations 6.5

vRealize Orchestrator 7.3 (internal to vRA)

NSX 6.3

Webhook Shims

The workflows should work with other versions as well. You need to have these products installed, configured and integrated to follow the example end to end.

How to configure auto-scale?

There are too many steps and explanation to cover in a single post. So instead I created a video to cover all of that. Please watch the video for details on how to configure the different components. It also covers where and how you can modify the steps. This video covers everything along with few ppt slides to cover the environment and slight background.

Conclusion:

You can use the steps detailed in the video to configure auto-scaling in your environment. This is an amazing feature. It will be a real help if you can test it out and let me know the outcome. Also, any further suggestions are welcome. I hope this helps you as it helped me. Do provide me feedbacks on this. Also, let me know if I missed something or you need further clarification.

Comments

4 Comments been added so far

Vinit

June 19th, 2017

Hi Sajal Debnath,
Thanks for your efforts. On top of that, I would like to know if I can fetch utilization of VMs via vROperation Manager?
If yes, it would be great help if you through light on that part too?

Sajal Debnath

June 19th, 2017

Hi Vinit,
Yes, by this weekend two more blogs will be posted, one on gathering data from vROps and another on how to push data into vROps. Those will answer those exact questions. So watch this place for those posts.