Automating Azure: How to Deploy a Temporary Virtual Machine Resource

Because Azure is designed from the ground-up to allow automated provisioning, It provides a number of interesting opportunities for use. With a .NET language such as C#, it is possible to create a very flexible 'on-demand' infrastructure in Azure, using Azure Resource Manager templates, that allows users to economically deploy the resources they need just for the interval of time that they are required, and then destroy them.

Automating Azure for Resources On-Demand

In this article I will show how to deploy a temporary Virtual Machine resource in Azure by using a C# console application and an ARM (Azure Resource Manager) template. After the resource is created, we will perform a simple task and then remove the resource. This is the first step on the way of creating a very flexible infrastructure in Azure, on demand.

An overview of the steps performed:

Set up a VM via the Azure portal and save the Parameters.JSON and Template.JSON files to a BLOB storage

Create a batch service and pool

Create an AAD app to be used for Service-to-Service authentication

Create a C# console app and add the code which will create the VM

Setup a VM in the portal and save the Template JSON files

A useful aspect of Azure is that it allows the user to create scripts to automate any object in the Azure environment. If ,for example, we have an existing Virtual Machine in Azure, we would be able to go to the portal and download the automation scripts to create the exact same resource at a later time. This is very valuable for such tasks as backups, reviews or transferring objects between subscriptions.

This is possible simply by selecting the resource in the portal and clicking on the ‘Automation script’ link:

The ‘Automation Script’ blade presents us with a set of choices for deployment: there is a PowerShell Script, CLI, .NET, Ruby:

The first two tabs, ‘Template’ and ‘Parameters’ are the actual JSON script files we will be working with.

In Azure there is also the possibility to get the ‘Template’ and ‘Parameters’ JSON scripts for an object which is configured in the portal, but is not deployed yet. And this is a great shortcut for customizing the properties of an Azure resource via the portal UI, and then using the ‘Template’ and ‘Parameters’ files for an automated deployment at a later time, instead of composing the JSON code by hand.

For this article, I will customize a small Virtual Machine and take the ‘Template’ and ‘Parameters’ files before deploying it.

Let’s navigate to the Azure portal, and start creating a new Virtual Machine resource. For my example, I will be using Windows Server 2008 R2 SP1 and a Resource Manager deployment model:

When we click on ‘Create’, we are presented with the opportunity to setup our VM:

For the sake of brevity I will not follow the entire process of configuring the VM, since this is a very specific task to each user and the type of workload they will be automating. It is important, however, that the VM in this case is Windows based, and it is also important not to click on ‘Purchase’ in the last step, but to click on the ‘Download template and parameters’ link and download the templates for it.

When we click on the ‘Download template and parameters’ link, we are presented with the opportunity to download the files. Click on the ‘Download’ button and a zipped file will be saved on your machine. The file contains all deployment options, but in this case we are interested in the JSON files.

When we have the JSON files on our machine, we will upload those two files to a BLOB storage container.

Again, for the brevity of this article, I will not go through each step it takes to create a BLOB storage account and a container, since most readers already have done this step. It is important to note the URL of the BLOB storage container for later use in our code, however.

Creating the C# console app

In this step we will use Visual Studio to create a C# console app which will allow us to debug our deployment:

Now that we have the application, we have to install some NuGet packages by running the following commands in the Package Management console:

Debugging the app

It is very unlikely that the console application will work perfectly from the first run. And when the app fails, it will give us a cryptic message like

As you can see, “Long running operation failed with status ‘Failed’” does not really tell us much. For us to get more information about the failure, we have to go to the Azure portal, find and click on the resource group we just created, and there we can see a link to the failed deployment attempt:

Clicking on the Failed link will take us to the Deployments tab, and there we will see the specific error:

Of course, before deploying the VM again, we need to delete the resource group.

And now, when I re-run my C# application again, it creates the VM successfully and I get to print out some of the VM’s properties to the console window, i.e. the machine type and the machine name:

This means that the machine is up and running, and it also means that we can get a handle of the properties and the resources of the VM from our code.

The options from here are unlimited, and depend on each use case. For example, if this was a Linux-based machine, we would use SSH.NET library to run ssh commands on the newly created machine. (We will do that in my next article in which we will be creating an on-demand HDInsight cluster and using SSH.NET to run R scripts)

Conclusion:

In this article we explored the automation possibilities in Azure for using ARM templates and C#. Of course, this article only demonstrates the concepts of resource automation, since for the trivial activity of creating a Virtual Machine on-demand there is a Batch Service automation in Azure. In the Batch Automation service we can bring up a pool of Virtual Machines behind the scenes and carry out a workload and then discard the resources. The concepts in this article are important, however, because not all types of resources and workloads are supported by Batch Service. This means that if we are able to control resources in Azure from C#, then we can use Custom .NET Activities together with ADF automation to deploy virtually any resource and execute any workload in Azure on demand. In my next article we will do exactly this: we will create a HDInsight cluster with RServer installed on it and we will perform some R script workload, after which we will delete the HDInsight cluster.

Subscribe for more articles

Subscribe to our fortnightly newsletter

Feodor has a background of many years working with SQL Server and is now mainly focusing on data analytics, data science and R.

Over more than 15 years Feodor has worked on assignments involving database architecture, Microsoft SQL Server data platform, data model design, database design, integration solutions, business intelligence, reporting, as well as performance optimization and systems scalability.

In the past 3 years he has expanded his focus to coding in R for assignments relating to data analytics and data science.

On the side of his day to day schedule he blogs, shares tips on forums and writes articles.

Hi
I am trying to replicate same demo in my environment.I getting error {“The client
‘ca247e04-efe2-4cb5-b6d4-a7ee285037c0’
with object id ‘ca247e04-efe2-4cb5-b6d4-a7ee285037c0’
does not have authorization to perform action ‘Microsoft.Compute/availabilitySets/write’
over scope ‘/subscriptions/ee72cb97-e3ac-47b3-b068-d9467f338fbb/resourceGroups/test/providers/Microsoft.Compute/availabilitySets/mypkAVSet’.”}

Related articles

I’ve been speaking a lot on the topic of DevOps lately, and there’s a question I like to ask. What is the purpose of your organization? I get all kinds of answers. “We make software that delights our customers and makes their jobs easier.” “Our company makes low-cost widgets. “ “We make the best gizmos … Read more

The AWS Elastic Beanstalk feature is a web application provisioning service that allows you to automatically scale as needed. In this article, Gerald Bauer explains how to set up and configure Elastic Beanstalks through the AWS dashboard and Visual Studio. … Read more

This article pulls together the concepts from two previous articles to demonstrate a way to automate the “on-demand” creation and deletion of an HDInsight cluster. This not only serves as a demonstration of the power of automating Azure provisioning, but is a practical solution for the user who only occasionally requires the power of HDInsight for scalable computations.… Read more