Deploying to Azure Cloud using Spinnaker

Purpose

Spinnaker as a renowned Continuous Delivery tool supports app deployment into all major cloud environments including Azure cloud. Here, We guide you through the configuration and pipeline setup of an app into Azure Cloud using Spinnaker.

Assumptions

We should be having the following items working/configured before configuring Spinnaker…

Valid Azure Subscription/Account Spinnaker 1.16.2

Working Jenkins integrated with Spinnaker. Jenkins does your CI build

Our app is a Java springboot application, to be deployed into Azure cloud. These inputs are quired for configuring Load Balancer and Firewall

saga@sagamc:~$ ssh-keygen -f ddsaga -N ''
#Username is ddsaga (Change it as per your choice)
#cat ddsaga.pub and copy the content without the last part <user>@<host> [e.g. opsmxuser@spinnaker-saga]
saga@sagamc:~$ VMUSER_KEY="ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDMGBDPxjYydvel/ZyYoaLM3zDB9tGY+YbkcpbnI/esqR2qGgs/rG9C1MWP3ews3d2F58Ss2F7C03tj+5phbhB+10tmnPn2ezAq3FXcjEenQluVb2QInxnr+bdfDeZLyYcApFemgs0mgU5LcP2ixGQ6Tli9xyziITRuaIr904pIOcbdo8+OLDXFS7W0+KhSu3RjCkD4EWNpbCB48j+Oz8YD6t6X5aixEjFg7v4Swo0+NxAeFteCyzU8hWZAdVW0eWdjo5zeQosMW7tVe+UFBDVgQfjrZd0ue0xOyYRf1HLplIEcyJpjy2DhWbG99FJW9gdbJqxeGLop7lKvbatrpBMp"

Integrating Azure provider in Spinnaker

Use Halyard command-line program to enable the integration of Azure in Spinnaker.

The Procedure below should be done on hal machine (VM/Pod/Container)

1. During the above procedure “Create Azure resources for Spinnaker Integration“, you would have created resources like Service Principle, Resource Group and Azure Vault. Make aNote those values and set them as env variables here

Note: Now if you create a new application in Spinnaker, the application should have the provider‘azure’ in the list.

Azure Virtual Network

Spinnaker will deploy applications on a Virtual network in Azure cloud. Hence, ensure to have a Virtual network and minimum one Subnet created in eastus region (or other region, it should match hal conﬁguration). You may create the virtual network on portal.azure.com

My Virtual network details are…

Network name: Spin_Vnet
CIDR: 172.16.0.0/22
Subnet: default

The Virtual network and Subnet will be used by Firewall, Load Balancer and Server group conﬁgurations.

Azure and Spinnaker Resource Mapping

Spinnaker generalizes the Cluster terms across multiple Cloud. The resource in Spinnaker could mean different thing in Azure cloud.

Spinnaker Firewall means Network Security Goup in Azure cloud.

Spinnaker Load Balancer is referred as Load Balancer only in Azure.

Spinnaker Server Group is referred as Virtual Machine ScaleSet in Azure.

Conﬁguring Azure application in Spinnaker UI

Note: Perform these steps in Spinnaker UI.

1. Create an Application

Under Applications, Create an application ‘appxinazure’. Ensure to select ‘azure‘ in the provider list.

2. Conﬁgure Application’s Infrastructure

Prior to conﬁguring a CD pipeline, we need to conﬁgure the Infrastructure of the application. A separate resource group by name <appname>-<region> will be created in Azure cloud automatically. In our case ‘appxinazure-eastus’ is the new resource group. The resources created in the Spinnaker UI like Firewall, Load Balancer, VM ScaleSet are stored on this resource group; however the baked images are stored on the resource group conﬁgured in Halyard.

** To successfully conﬁgure the Firewall, Load Balancer and Server group – Virtual network and Subnet are one of the pre-requirement. You should have them created. If not, refer the section Azure Virtual Network to setup one.

a. Create Firewall

Create a firewall first. This firewall settings are inheritted by the VMs. So whiteliest the port 8080 (App is listening on it) for LoadBalancer to reach this port; Similarly whitelist the port 22 for any SSH session that may be required for debugging.

b. Create Load balancer

You should know the application listening port, health check page, app url, so that the load balancer can be conﬁgured correctly.

c. Create Server Group

Do not create server group using the Infrastructure tab, because you may be unsuccessfull – while it will look for a image that will not available until we perform a Bake stage. Hence, conﬁguring server group is part of the pipeline setup process. You will have to create server group in Deploy stage.

Conﬁguring Pipeline

Under Pipeline section, click ‘Create’ to create a New Pipeline with type being selected as Pipeline. Input a valid name for your pipeline.

1. Conﬁgure Stage

Here, select your Jenkins CI account to take Build information. Prior to this step, you should have your Jenkins conﬁgured in Halyard under CI systems.

2. Bake Stage

We are targetting the package to Ubuntu server (16+), while the base image is baked with our app installed on it. The deb package ﬁle name should have been created with the naming convention: name_version-release_arch.

Here, select your package (.deb) name to create a baked image. The package is essentially the output of Jenkins Build. This stage will looke for a valid package within the Jenkins project directory.

Select your region ‘eastus‘ to deploy the application into. Remember the available regions here are
activated by hal command. Then, type your package name without any version information

3. Deploy Stage

The deployment server group in Spinnaker will create a VMScaleSet on Azure, the loadbalancer will route the traﬃc to the VM ScaleSet.

Add a Server group and conﬁgure every section in ‘Conﬁgure Deployment Cluster‘. Choose the account, select the region as ‘eastus‘, valid stack that may denote environment like dev/qa/stg. Detail may contain any text, but I chose to have ‘svrgroup’. Then choose the load balancer you had created in one of the above steps.

Choose the Virtual network and Subnet, where the VM ScaleSets will be created during CD pipeline execution.

You can choose the firewall that you had created earlier to control the traffic to the VM ScaleSet machines.

Choose the Instance type that best ﬁts for your application. Here, we have gone with General purpose computer.

You can associate some tags that may be used as custom meta-data for your VMs, which can be used for ﬁltering and other operations.

Save the conﬁguration. And, you are done with your Pipeline conﬁguration. You are now ready to execute your Pipeline and see how it goess.

Troubleshooting

01. Problem:

In Bake stage, package name was was mentioned as restapp_ (since our build output package was restapp_1.0_all.deb). The Baking process throw the error “Unable to ﬁnd deployable artifact starting with [RestApp ] and ending with .deb in [] and [restapp_1.0_all.deb]”

Solution:

Make sure your deb package ﬁle name complies with the naming convention: name_version- release_arch. In the Bake stage, just mention the name part alone (donot include the underscore).

04. Problem:

Solution:

The problem occurs due to a defect in Packer version 1.4.2 as stated here
https://github.com/has hicorp/packer/issues/7816. The ﬁx is to update the Packer version to 1.4.4 which is packaged with Spinnaker 1.17. This is conﬁrmed here https://github.com/spinnaker/rosco/pull/452. Either update the Spinnaker to 1.17 or you can update the rosco.yml ﬁle with overridden docker image version which has the Packer 1.4.4 updated. See below how to update the docker version for rosco.

Solution:

Spinnaker did not provide enough log information as to which resouce is NotFound. In tracing the resources everything we created, found that the items – VMUsername, and VMSshPublicKey were missing in Azure vault SpinHal-vault

After adding the SSH key and public-key, the deployment stages went successfully.

Refer the Azure CLI procedure again to create the Username and Public-key. In particular the commands,

06. Problem:

Spinnaker Deployment stage is completed successfully. My loadbalancer is public one, but unable to access the app URL from Internet.

Solution:

Solution to this problem involves couple of Network related tracing.

a. Verify the public IP of the load-balancer is reachable by _ping_ command – ‘ping <public-ip-of-lb>‘.

b. Check the status of load-balancer is good as per the Azure portal: Monitor > Service Health > and then make sure that Subscription ID and Resource Type = Load Balancer are selected. If it fails, then your load balancer configuration is incorrect – fixt it.

c. Make sure the application is working on the VM (e.g. 172.16.0.5) and is lisening on the designated port.

‘curl http://localhost:8080/health’

c. Make sure the application URL using the VM’s IP is accessible from another VM (e.g. 172.16.0.6) on the same network – ‘curl http://172.16.0.5:8080/health‘

d. If the step above is working good, it could be NSG that is causing the inbound requests blocking requests to VMs. Even if you have a rule from ‘AzureLoadBalancer’ to ‘VirtualNetwork’ on port 8080, it may fail. Try adding a rule explicitly that would have the Source ‘Any’ and Destination ‘VirtualNetwork’ for port 8080 and have it ‘Allowed’

c. You can also verify from Internet about what all the ports are open to the public-ip of the load balancer using the command ‘nmap -v <public-ip> -Pn’.