ARM Templates tagged posts

Today as a part of the Azure Governance and management announcements at Microsoft Ignite 2018Azure Blueprints Public Preview was announced. Azure Blueprints are a core part of the cloud governance and management story. They go hand and hand with Management Groups and will take the enterprise management story of Azure up a level. In this blog post I will take a deep dive into Azure Blueprints explaining what they are and give an example of how they can be used.

NOTE:This is a long blog post so I have also published this content as a whitepaper. The whitepaper PDF can be downloaded here.

BLUEPRINTS OVERVIEW

At a high-level Azure Blueprints help you meet organizational cloud standards, patterns, and requirements through governed subscriptions enabled and enforced by the grouping of artifacts like ARM templates, Azure policies, RBAC role assignments, and resource groups within a Blueprint.

Blueprints can be used to lay a cloud foundation, as cloud patterns, and group cloud governance frameworks. Blueprints are a one-click solution for deploying a cloud foundation, pattern, or governance framework to an Azure subscription. Think of an Azure Blueprint as re-usable design parameters for cloud that can be shared and used across an enterprise.

Azure architects typically map out and plan the many aspects of a cloud foundation for an organization such as access management, networking, storage, policy, security/compliance, naming conventions, tagging, monitoring, backup, locations, and more. Now Azure architects can step this designing a step further build these designs as Azure Blueprints and then apply them to subscriptions. The Blueprints give architects a way to orchestrate the deployment of grouped components to speed up the development and provisioning of new Azure environments ensuring they are meeting organizational compliance.

BLUEPRINTS ARE NOT AZURE POLICY

Azure policy is a service targeted to resource properties that exists or when being deployed with allow or explicit deny policies. It is used to ensure resources in an Azure subscription adhere to requirements and standards of an organization.

Azure policies can exist on their own or be a part of an Azure Blueprint. Blueprints do not replace Policy they are one of the Artifact types that make up a Blueprint.

THE MAKEUP OF A BLUEPRINT

Definition

A Blueprint consists of a Definition. The Definition is the design of what should be deployed it consists of the name of the Blueprint, the description and the Definition location. The Definition Location is the place in the Management Group hierarchy where this Blueprint Definition will be stored and determines the level assignment is allowed at. Currently you must have Contributor access to a Management Group to be able to save a Blueprint Definition to it. A Blueprint can be assigned at or below the Management Group it has set in its Definition Location. Here is a diagram to visualize Blueprint Assignment in relation to Management Group hierarchy:

Artifacts

The Definition is where Blueprint Artifacts are added. As of right now the following is a list of the Artifact types:

Policy Assignments – Lets you add an Azure Policy. This can be a built-in or custom policy.

Role Assignments – Lets you add a user, app, or group and set the role. Only built-in roles are currently supported.

Azure Resource Manager templates – Lets you add an ARM Template. This does not let you import a parameters file. It does let you pre-set the parameters or set the parameters during assignment of the Blueprint.

Resource Groups– Lets you add a Resource Group to be created as a part of this Blueprint.

In my opinion the ARM Template artifact is the most impactful of the Blueprint artifact types because you can define such a variety of resources here. It opens the Blueprint to the power of ARM in general. Hopefully in the future we will see more scripting capability or the ability to load PowerShell scripts, runbooks, and or Functions.

There are two levels in the Artifacts. The first level is Subscription. The second level is Resource Group. Resource Group artifacts cannot be added to a Resource Group artifact. A Resource Group artifact can be created in a Subscription. An ARM Template artifact can only be created in a Resource Group artifact. A Policy Assignments or Role Assignments can be created at either the Subscription or Resource Group level.

Assignment

After a Blueprint has been built it needs to be applied. Applying a Blueprint is known as Blueprint assignment. The assignment is essentially the “what was deployed” for a Blueprint. This is how the artifacts are pushed out to Azure and used to track and audit deployments in Azure.

Sequencing

When the assignment of a Blueprint is processed the default order of resource creation is:

Role assignment artifacts at the Subscription level

Policy assignment artifacts at the Subscription level

Azure Resource Manager template artifacts at the Subscription level

Resource group artifacts and its child artifacts (role assignment, policy assignment, ARM Templates) at the Resource Group level

When a blueprint includes multiple Azure Resource Manager templates there may be a need to customize the sequencing order in which the Blueprint will deploy artifacts during assignment. You customize the artifact deployment sequence by deploying a Blueprint from an ARM Template declaring a dependency within it or declaring a dependency within an ARM Template artifact in the Blueprint. You declare a dependency using the dependsOn property in JSON. This essentially is a string array of artifact names.

Resource Locking

In cloud environments consistency is key. Naturally Azure Blueprints can also leverage resource locking in Azure. Blueprints have a Locking Mode. This Locking Mode can be applied to None or All Resources and is determined during the assignment of the Blueprint. The decision on cannot be changed later. If a locking state needs to be removed, then you must first remove the Blueprint assignment.

Some Blueprint artifacts create resources during assignment. These resources can have the following state:

Artifacts that become Resource groups get the state of Cannot Edit / Delete automatically but you can create, update, and delete resources within them.

The high-level stages of an Azure Blueprint are Create it, assign it to a scope, and track it.

Anatomy of a Blueprint:

Blueprint does have a REST API. I am not covering the REST API in this blog post as I have not had the opportunity to spend much time working with it yet.

Now let’s look at building and assigning an Azure Blueprint.

BUILD A BLUEPRINT

Now I am going to give an example of building and using an Azure Blueprint in a cloud foundation mock scenario. In my mock scenario I have 3 Azure subscriptions. Each subscription should have a Core services Resource Group consisting of a core VNet with 3 subnets, an NSG for each subnet, and the web subnet should be ready for DMZ traffic. For the core VNet and any additional VNet added to the Core Services Resource Group I need network watcher deployed to it.

Each subscription also should have a core storage account and a blob storage that is ready for general storage needs. I want a tag applied to any Blueprint assignment labeling it with the assignment name, so it is easy to track. The last requirement I have is that I need the CloudOps team to automatically be owner of all core services resources. To accomplish all of this I created the following Blueprint:

Now let’s walk through the parts of creating and assigning the Blueprint. The first step is to create the Blueprint Definition.

In the basics step I give it a meaningful name and meaningful description. I set the Definition Location to the root of my Management groups. Doing this will allow me to assign this Blueprint to all 3 subscriptions in turn creating the core services RG in each subscription.

Next the Artifacts need to be added. Note that when adding an Artifact at the Subscription level you have these options as types:

The Resource Group Artifact type is only available at the subscription level and the ARM template Artifact type is only available at the Resource Group level. I added the Resource Group that the core networking and core storage will be deployed into.

I looked for an existing ARM template that would create multiple Linux VM’s. I found only one that creates some in a scale set. The use case I was working with did not call for a scale set so I needed a different template.

I found a simple ARM template for creating multiple Windows VM’s on Azure here. It had exactly what I needed for my use case but did not cover Linux.

I modified the template and uploaded to Github in case this is helpful to anyone else. The repo has two templates. There is one for Ubuntu and one for SUSE. When you deploy the template it will need the following parameters:

The ARM template will create an availability set (AS) with N number of VM’s put in that AS, network interfaces, and public IP’s for each VM along with a VNet and Subnet as shown in the following screenshot:

We all know that DevOps brings together people, processes, and technology. In the Microsoft DevOps world A large part of the technology piece is utilizing Visual Studio Team Services (VSTS) for continuous deployment of workloads to Azure.

Microsoft launched their Hybrid Cloud on July 10th 2017. Azure Stack is the secret sauce of Microsoft’s the Hybrid Cloud. Microsoft’s offering is the only one true Hybrid Cloud in the market bringing Azure to on-premises data centers.

As Microsoft continues to move their Hybrid Cloud forward the DevOps integration and capabilities we have for Azure extend to Azure Stack. Again I was fortunate to participate in a preview of the VSTS integration with Azure Stack. I was happy to see Microsoft putting a priority on this functionality because DevOps on Azure Stack is a HUGE need. Cloud is often the catalyst to helping organizations adopt a DevOps culture fostering digital transformation. Some organizations not being able to put all workloads in public cloud Azure Stack is a good way for them to get the same cloud capabilities on-premises DevOps integration being one of them. The setup and integration between VSTS and Azure Stack is working nicely. The team at Microsoft has given me permission to share about this topic via my blog.

In this blog post I am going to cover setting up VSTS to work with Azure and setting up a continuous-integration and-continuous deployment (CI/CD) pipeline to Azure Stack. With Microsoft DevOps you can utilize the pieces of VSTS that make sense for you to use leaving the control up to you. Through VSTS you can use many other DevOps tools such as Jenkins, Octopus deploy, GitHub, Bitbucket etc into your pipeline making Azure Stack just as flexible as Azure is. Let’s Jump in!

Steps to prep Azure Stack for Visual Studio Team Services (VSTS)

#1 Ensure you have installed the Azure Stack PowerShell and Azure PowerShell modules.

You will use information from the Service Connection output in the next step.

Steps to configure Azure Stack as a Service Endpoint in VSTS

Log into your VSTS account at visalstudio.com

Navigate to one of your projects.

Go into Settings.

Click on Services.

Click on New Service Endpoint

A window will pop up. Click on “use full version of the endpoint dialog.”

Next input the needed data. This data comes from the Service Connection info that you copied.

You can put whatever you want in the Connection name and the Subscription Name. Note do not verify the connection. It will not succeed as VSTS cannot access your private Azure Stack yet. Click OK when done.

Setup build agent on Azure Stack host

Next you need to setup the build agent on the Azure Stack host. (Note: In this post I am using the ASDK.) From within VSTS download the Windows agent. Extract the download to a local folder.

Go to Security under your profile in VSTS.

Next add a Personal access token(PAT) for Azure Stack.

Copy the token. Note it will not be shown again ever after you leave this screen.

In the folder with the extracted build agent you will see the following. We need to run the run.cmd file from an elevated command prompt.

Here is a screenshot of running the run.cmd. I recommend deploying the build agent as a service. You will use your personal access token (PAT) here and the azure stack admin account.

After the run.cmd finished the folder with the extracted contents should look like the following:

You can now see the agent in VSTS.

That’s it for the setup for connecting VSTS to Azure Stack. Next let’s look at setting up a continuous-integration and-continuous deployment (CI/CD) pipeline for VM-deployment to Azure Stack.

THE BUILD

What I cover here is focused on infrastructure as code (IaC) using ARM templates. If you need to set up CI/CD to Azure Stack for Web Apps, Mobile Apps, Containers, etc the process is the same as it is on Azure with the only difference being that you point to Azure Stack. Also note that in this post I am using the ASDK not multi-node.

Within VSTS create a new repository and place your ARM template in it.

Next click on Build and Release. Create a new Build Definition.

In the build definition. Point the Get sources to the repository you just created. Add 2 tasks under Phase 1. The first task will copy the ARM template to the build staging directory. The second task will publish the ARM template so that a release definition can pick it up. Both tasks are shown in the following screenshots.

Copy Files to task

Publish Artifact task

OPTIONAL:To setup continuous integration click on Triggers. Here you can set a schedule to run the builds or you can click on the repository as shown in the screenshot and then check Enable continuous integration. By checking the box next to Enable continuous integration it tells VSTS that anytime content in the repo is changed to run a build.

Click on Save & queue. This will start the build.

The build will start. As long as everything is setup properly within your build it will succeed as shown in the following Screenshot.

That’s all for our build. Next up we need to create a release definition (RD) pipeline. The RD will take the build artifacts and deploy to an environment/s you specify.

My demo is this session was titled “Write once, deploy anywhere“. The purpose of this demo was to show using a single ARM template (JSON file) and a single PowerShell script to deploy a VM regardless of deploying to Azure or Azure Stack. The demo was a success so yes this is really possible. In this post I will break down the JSON file, the PowerShell script, how it works and the download link for the files.

Getting the JSON file and the PowerShell script just right was a challenge as there are still some slight differences between the settings of Azure and Azure Stack. Note that this is the case with Azure Stack TP1 and I fully expect that this will change when it GA’s. In any case it is good to look at this stuff now to start to learn the ins and outs. In the end it was the combined Power of the ARM template and PowerShell to overcome any challenges. Let’s start off by taking a look at the differences in ARM between Azure and Azure Stack in the following table:

Property

Azure

Azure Stack

Location

Azure region (example: CentralUS)

local

blobStorageEndpoint

blob.core.windows.net

blob.azurestack.local

vmSize

Standard_D1

Standard_A1

vmName apiVersion

2015-06-15

2015-06-15

StorageAccountName apiVersion

2015-06-15

2015-06-15

nicName apiVersion

2015-06-15

2015-05-01-preview

vrtualNetworkName apiVersion

2015-06-15

2015-05-01-preview

networkSecurityGroupName apiVersion

2015-06-15

2015-05-01-preview

dnsNameForPublicIP apiVersion

2015-06-15

2015-05-01-preview

torageAccountName apiVersion

2015-06-15

2015-05-01-preview

NOTE: For the apiVersion on the resources Azure Stack requires 2015-05-01-preview. Resources in Azure ARM templates default to apiVersion 2015-06-15. So if we left the resources in the ARM template at apiVersion 2015-06-15 the deployment would fail on Azure Stack. However we are in luck as Azure will accept apiVersion 2015-05-01-preview. So I set vmName and StorageAccountName to apiVersion 2015-06-15 and the rest of the resources apiVersion to 2015-05-01-preview.

vmName and StorageAccountName use the same apiVersion for both Azure and Azure Stack. So Azure Stack accepts 2015-06-15 for both. Even those these are not different across Azure and Azure Stack I still wanted to list it anyway in the table.

If you have multiple subscriptions you will need to input the subscription ID. In my case my Azure has multiple subscriptions but my Azure Stack does not in this lab. In my script for Azure you need the subscription ID. In Azure Stack you do not. You may need to modify this behavior in the script if your scenario is different.

For the deployment it consists of two files. These files are:

AzureandAzureStack.json

CreateVMAzureorAzureStack.ps1

Here is what we have if we crack open the JSON file.

A few things to note about the PowerShell script is that

We prompt to identify if it is an Azure or Azure Stack deployment. We then run the appropriate block of code.

In each of the deployment types (Azure or Azure Stack) we have some things hard coded in (for example blobStorageEndpoint and vmSize) and somethings pulled in dynamically by prompting for them during the script execution (for example subscriptionId and adminPassword).

We are pulling in the parameter and variable values when using New-AzureRmResourceGroup and New-AzureRmResourceGroupDeployment.

NOTE:I am not a PowerShell expert. I am sure there are better more efficient ways to accomplish what I am doing here in the PowerShell script. Nothing was available to accomplish the write once, deploy anywhere goal so I put something together. Feel free to enhance the script and release back to the community.

Here is an example of the location parameter and variable in the JSON file.

The parameter:

The variable:

Referenced in the vmName resource:

Here is an example of how we are leveraging this in the PowerShell script.

For Azure:

For Azure Stack:

Note that you can deploy VM’s to Azure or Azure Stack in many ways (Visual Studio, the portal etc..). I decided to leverage PowerShell to do the deployment’s as it gives me a great amount of flexibility. For the official article on using PowerShell to deploy VM’s to Azure Stack visit: