AKS tagged posts

CloudSkills.fm is a podcast by fellow Microsoft MVP Mike Pfeiffer and veteran in the tech space with 5 books under his belt and numerous courses on Pluralsight. The podcast can be found here: cloudskills.fm. Mike is an all around good guy and I was honored to be a featured guest on one of his podcast episodes. The podcast is weekly with technical tips and career advice for people working in the cloud computing industry. The podcast is geared for developers, IT pros, those making move into cloud.

On this episode Mike
and I talked about managing both the technical and non-technical aspects of
your career in the cloud computing industry. We also discuss DevOps stuff
around Docker, Azure Kubernetes Service, Terraform and cloud stuff around Azure
management including my 5 points to success with cloud. You can listen to the
podcast here:

I’m very excited
Opsgility recently published a new Azure course by me titled: “Deploy and
Configure Infrastructure”. This course is part of the AZ 300 certification
learning path for Microsoft Azure Architect Technologies. More about the AZ 300
certification can be found here: https://www.microsoft.com/en-us/learning/exam-az-300.aspx.
The course is over 4 hours of Azure content!

Description of the course:

In the course learn
how to analyze resource utilization and consumption, create and configure
storage accounts, create and configure a VM for Windows and Linux, create
connectivity between virtual networks, implement and manage virtual networking,
manage Azure Active Directory, and implement and manage hybrid identities.

Lately I have been hearing a lot about a solution named Rancher in the Kubernetes space. Rancher is an open source Kubernetes Multi-Cluster Operations and Workload Management solution. You can learn more about Rancher here: https://www.rancher.com.

In short you can use
Rancher to deploy and manage Kubernetes clusters deployed to Azure, AWS, GCP
their managed Kubernetes offerings like GCE, EKS, AKS or even if you rolled
your own. Rancher also integrates with a bunch of 3rd party solutions for
things like authentication such as Active Directory, Azure Active Directory,
Github, and Ping and logging solutions such as Splunk, Elasticsearch, or a
Syslog endpoint.

Recently training
opened up for some Rancher/Kubernetes/Docker training so I decided to go. The
primary focus was on Rancher while also covering some good info on Docker and
Kubernetes. This was really good training with a lot of hands on time, however
there was one problem with the labs. The labs had instructions and setup
scripts ready to go to run Rancher local on your laptop or on AWS via
Terraform. There was nothing for Azure.

I ended up getting
my Rancher environment running on Azure but it would have been nice to have
some scripts or templates ready to go to spin up Rancher on Azure. I did find
some ARM templates to spin up Rancher but they deployed an old version and it
was not clear in the templates on where they could be updated to deploy the new
version of Rancher. I decided to spend some time building out a couple of ARM
templates that can be used to quickly deploy Rancher on Azure and add a
Kubernetes host to Rancher. In the ARM template I pulled together it pulls the
Rancher container from Docker Hub so it will always deploy the latest version.
In this blog post I will spell out the steps to get your Rancher up and running
in under 15 minutes.

The repository consists of ARM templates for deploying Rancher and a host VM for Kubernetes. NOTE: These templates are intended for labs to learn Rancher. They are not intended for use in production.

In the repo ARM Template #1 named RancherNode.JSON will deploy an Ubuntu VM with Docker and the latest version of Rancher (https://hub.docker.com/r/rancher/rancher) from Docker Hub. ARM Template #2 named RancherHost.JSON will deploy an Ubuntu VM with Docker to be used as a Kubernetes host in Rancher.

Node Deployment

Deploy the
RancherNode.JSON ARM template to your Azure subscription through “Template
Deployment” or other deployment method. You will be prompted for the
following info shown in the screenshot:

Host Deployment

Deploy the
RancherHost.JSON ARM template to your Azure subscription through “Template
Deployment” or other deployment method. Note that that should deploy this
into the same Resource Group that you deployed the Rancher Node ARM template
into. You will be prompted for the following info shown in the screenshot:

After the Rancher
Node and Rancher Host ARM templates are deployed you should see the following
resources in the new Resource Group:

Name

Type

RancherVNet

Virtual
network

RancherHost

Virtual
machine

RancherNode

Virtual
machine

RancherHostPublicIP

Public
IP address

RancherNodePublicIP

Public
IP address

RancherHostNic

Network
interface

RancherNodeNic

Network
interface

RancherHost_OSDisk

Disk

RancherNode_OSDisk

Disk

Next navigate the
Rancher portal in the web browser. The URL is the DNS name of the Rancher Node
VM. You can find the DNS name by clicking on the Rancher Node VM in the Azure
portal on the overview page. Here is an example of the URL:

The Rancher portal
will prompt you to set a password. This is shown in the following screenshot.

After setting the
password the Rancher portal will prompt you for the correct Rancher Server URL.
This will automatically be the Rancher Node VM DNS name. Click Save URL.

You will then be
logged into the Rancher portal. You will see the cluster page. From here you
will want to add a cluster. Doing this is how you add a new Kubernetes cluster
to Rancher. In this post I will show you how to add a cluster to the Rancher
Host VM. When it’s all said and done Rancher will have successfully deployed
Kubernetes to the Rancher Host VM. Note that you could add a managed Kubernetes
such as AKS but we won’t do that in this blog. I will save that for a future
blog post!

Click on Add Cluster

Under “From my
own existing nodes” Click on custom, give the cluster a name and click
Next.

Next check all the
boxes for the Node Options since all the roles will be on a single Kubernetes
cluster. Copy the code shown at the bottom of the page, click done and run the
code on the Rancher Host.

In order to run the
code on the Rancher Host you need to SSH in and run it from there. To do this
follow these steps:

In the Azure Portal, from within the resource group click on the Rancher Host VM.

On the Overview page click on Connect.

Copy “ssh ranchuser@rancherhost.centralus.cloudapp.azure.com” from the Connect to virtual machine pop up screen.

Open a terminal in either Azure cloud shell or with something like a terminal via VS Code and past the “ssh ranchuser@rancherhost.centralus.cloudapp.azure.com” in.

Running the code
will look like this:

When done you can
run Docker PS to see that the Rancher agent containers are running.

In the Rancher
portal under clusters you will see the Rancher host being provisioned

The status will
change as Kubernetes is deployed.

Once it’s done
provisioning you will see your Kubernetes cluster as Active.

From here you can
see a bunch of info about your new Kubernetes cluster. Also notice that you
could even launch Kubectl right from hereand start running commands! Take some
time to click around to see all the familiar stuff you are used to working with
in Kubernetes. This is pretty cool and simplifies the management experience for
Kubernetes.

If you want to add
more nodes or need the configuration code again just click the ellipsis button
and edit.

In Edit Cluster you
can change the cluster name, get and change settings and copy the code to add
more VMs to the cluster.

That’s the end of
this post. Thanks for reading. Check back for more Azure, Kubernetes, and
Rancher blog posts.

Part of running Kubernetes is being able to
monitoring the cluster, the nodes, and the workloads running in it. Running
production workloads regardless of PaaS, VM’s, or containers requires a solid
level of reliability. Azure Kubernetes Service comes with monitoring provided
from Azure bundled with the semi-managed service. Kubernetes also has built in
monitoring that can also be utilized.

It is important to note that AKS is a free
service and Microsoft aims to achieve at least 99.5% availability for the
Kubernetes API server on the master node side.

But due to AKS being a free service Microsoft
does not carry an SLA on the Kubernetes cluster service itself. Microsoft does
provide an SLA for the availability of the underlying nodes in the cluster via
the Azure Virtual Machines SLA. Without an official SLA for the Kubernetes
cluster service it becomes even more critical to understand your deployment and
have the right monitoring tooling and plan in place so when an issue arises the
DevOps or CloudOps team can address, investigate, and resolve any issues with the
cluster.

The monitoring service included with AKS
gives you monitoring from two perspectives including the first one being
directly from an AKS cluster and the second one being all AKS clusters in a
subscription. The monitoring looks at two key areas “Health status”
and “Performance charts” and consists of:

Insights – Monitoring for the
Kubernetes cluster and containers.

Metrics – Metric based
cluster and pod charts.

Log Analytics – K8s and Container
logs viewing and search.

Azure Monitor

Azure Monitor has a containers section. Here
is where you will find a health summary across all clusters in a subscription
including ACS. You also will see how many nodes and system/user pods a cluster
has and if there are any health issues with the a node or pod. If you click on
a cluster from here it will bring you to the Insights section on the AKS
cluster itself.

If you click on an AKS cluster you will be
brought to the Insights section of AKS monitoring on the actual AKS cluster.
From here you can access the Metrics section and the Logs section as well as
shown in the following screenshot.

Insights

Insights is where you will find the bulk of
useful data when it comes to monitoring AKS. Within Insights you have these 4
areas Cluster, Nodes, Controllers, and Containers. Let’s take a deeper look
into each of the 4 areas.

Cluster

The cluster page contains charts with key
performance metrics for your AKS clusters health. It has performance charts for
your node count with status, pod count with status, along with aggregated node
memory and CPU utilization across the cluster. In here you can change the date
range and add filters to scope down to specific information you want to see.

Nodes

After clicking on the nodes tab you will see
the nodes running in your AKS cluster along with uptime, amount of pods on the
node, CPU usage, memory working set, and memory RSS. You can click on the arrow
next to a node to expand it displaying the pods that are running on it.

What you will notice is that when you click
on a node, or pod a property pane will be shown on the right hand side with the
properties of the selected object. An example of a node is shown in the
following screenshot.

Controllers

Click on the Controllers tab to see the
health of the clusters controllers. Again here you will see CPU usage, memory
working set, and memory RSS of each controller and what is running a
controller. As an example shown in the following screenshot you can see the
kubernetes dashboard pod running on the kubernetes-dashboard controller.

The properties of the kubernetes dashboard pod
as shown in the following screenshot gives you information like the pod name,
pod status, Uid, label and more.

You can drill in to see the container the pod
was deployed using.

Containers

On the Containers tab is where all the
containers in the AKS cluster are displayed. An as with the other tabs you can
see CPU usage, memory working set, and memory RSS. You also will see status,
the pod it is part of, the node its running on, its uptime and if it has had
any restarts. In the following screenshot the CPU usage metric filter is used
and I am showing a containers that has restarted 71 times indicating an issue
with that container.

In the
following screenshot the memory working set metric filter is shown.

You can also filter the
containers that will be shown through using the searching by name filter.

You also can see a containers logs in the containers tab. To do this select a container to show its properties. Within the properties you can click on View container live logs (preview) as shown in the following screenshot or View container logs. Container log data is collected every three minutes. STDOUT and STDERR is the log output from each Docker container that is sent to Log Analytics.

Kube-system is not currently collected and sent to Log Analytics. If you are not familiar with Docker logs more information on STDOUT and STDERR can be found on this Docker logging article here: https://docs.docker.com/config/containers/logging.

In this blog post I am going to walk through the steps for deploying WordPress to Azure Kubernetes Service (AKS) using MySQL and WordPress Docker images. Note that using the way I will show you is one way. Another way to deploy WordPress to AKS would be using a Helm Chart. Here is a link to the WordPress Helm Chart by Bitnami https://bitnami.com/stack/wordpress/helm. Here are the images we will use in this blog post:

The first thing we need to do is save these files as mysql-deployment.yaml and wordpress-deployment.yaml respectively.

Next, we need to setup a password for our MySQL DB. We will do this by creating a secret on our K8s cluster. To do this launch the bash or PowerShell in Azure cloud shell like in the following screenshot and run the following syntax:

The secret is now created. To ensure it was created you can run the following syntax to list the secrets:

kubectl get secrets

You also can see the secret in the Kubernetes dashboard as shown in the following screenshot.

Next the mysql-deployment.yaml and wordpress-deployment.yaml files from the beginning of this post need to be uploaded to Azure cloudrive storage.

You can also do this in the Cloudshell as shown in the following screenshot.

Run ls in the shell to make sure the files are on your clouddrive.

You will need your home drive. Mine was. /home/steve. To see this, click on Download. It will show you what yours is.

Next create the MySQL Pod and service by running the following syntax.

kubectl apply -f /home/steve/mysql-deployment.yaml

NOTE: You could use kubectl create /home/steve/mysql-deployment.yaml instead of apply to create the MySQL pod and service. I use apply because I typically use the declarative object configuration approach. kubectl apply essentially equals kubectl create + kubectl replace. In order to update an object after it has been created using kubectl create you would need to run kubectl replace.

Note that in the mysql yaml file it has syntax to create a persistent volume. This is needed so that the database stays in tact even if the pod fails, is moved etc. You can check to ensure the persistent volume was created by running the following syntax:

kubectl get pvc

Also, you can run the following syntax to verify the mysql pod is running:

kubectl get pods

Deploying the WordPress Pod and service is the same process. Use the following syntax to create the WordPress pod and service:

kubectl apply -f /home/steve/wordpress-deployment.yaml

Again, check to ensure the persistent volume was created. Use the following syntax:

kubectl get pvc

NOTE: When checking right after you created the persistent volume it may be in a pending status for a while like shown in the following screenshot:

You can also check the persistent volume using the K8s dashboard as shown in the following screenshot:

With the deployment of MySQL and WordPress we created 2 services. The MySQL service has a clusterip that can only be accessed internally. The WordPress service has an external IP that is also attached to an Azure Load Balancer for external access. I am not going to expand on what Kubernetes services are in this blog post but know that they are typically used as an abstracted layer in K8s used for access to Pods on the backend and follow the Pods regardless of the node they are running on. For more information about Kubernetes services visit this link: https://kubernetes.io/docs/concepts/services-networking/service.

In order to see that the services are running properly and find out the external IP you can run the following syntax:

kubectl get services (to see all services)

or

kubectl get services wordpress (to see just the WordPress service)

You also can view the services in the K8s dashboard as shown in the following screenshot:

Well now that we have verified the pods and the services are running let’s check out our new WordPress instance by going to the external IP in a web browser.

Thanks for checking out this blog post. I hope this was an easy to use guide to get WordPress up and running on your Azure Kubernetes Service cluster. Check back soon for more Azure and Kubernetes/Container content.

Azure Kubernetes
Service (AKS) is a fully managed Kubernetes (K8s) offering from Microsoft on
the Azure platform. AKS reduces the management overhead of running your own K8s
instance while still being able to take full advantage of Container
Orchestration. Microsoft takes care of the K8s health monitoring and
maintenance. With AKS you only manage the agent nodes while Microsoft manages
the master nodes. Also with AKS you get integration to many of the Azure
services such as load balancers, RBAC, Azure storage etc.

In this blog post I am going to walk through the setup of an AKS cluster step by step. This is to serve as a intro to AKS to show how easy it is to get started with Kubernetes in Azure. In a follow up blog post I will dive into AKS more showing how to deploy an instance MySQL and WordPress containers on AKS. Before we get into the setup of AKS there are a few things to note:

With the AKS managed service you only pay for the agent nodes within your AKS cluster. There is no cost for the master nodes and the managed service itself is free.

At the time of this blog post AKS only supports Linux containers. There is a work around for this until Windows nodes and containers come to AKS.

The Kubernetes API server is exposed as a public fully qualified domain name (FQDN). Access should be restricted on this. It can be restricted using K8s RBAC and AAD.

Deploy AKS

Housekeeping is
done, now let’s get into the deployment of AKS. The first thing you need to do
within the Azure portal is go to Create a resource and search on Kubernetes.
Select the Kubernetes Service.

Click on create.

You will now see the
setup. The setup consists of the following sections shown in the following
screenshot:

Let’s walk through
each section.

Basics

Here you need to
give your AKS instance a name, select the region, K8s version, DNS prefix, and
number of nodes and count.

Authentication

Kubernetes has its
own RBAC within its authentication and authorization system. Azure Active
Directory (AAD) can be integrated with this for authentication. Once the AAD
and K8s integration is setup AAD users can be used for Kubernetes role-based
access control (RBAC) to cluster resources. Select yes to enable RBAC and
integration with AAD.

It is recommended to
setup your own service principle in AAD. For this blog post I let the
deployment create one. The service principle is used by K8s for managing Azure
cloud resources attached to the cluster. The service principle interacts with
Azure APIs. For example when you setup a load balancer service in K8s AKS
creates and Azure load balancer. The service principle is what is used for
authentication to create the load balancer.

Networking

In this section you
chose what you want for networking with AKS. If you select basic AKS will
create all needed VNets, Subnets, NSG’s etc. AKS clusters cannot use the
following ranges 169.254.0.0/16, 172.30.0.0/16, and 172.31.0.0/16. If you
select advanced you can chose an existing VNet or create a new one specifying
the subnet, IP range and DNS settings etc. You would select Advanced if you
need more control over the virtual networking.

HTTP application routing is used to make application endpoints publicly accessible in the AKS cluster. Enabling this essentially configures an Ingress controller in the AKS cluster. When getting started with AKS I recommend leaving this disabled and doing more research on K8s Ingress Controllers here https://kubernetes.io/docs/concepts/services-networking/ingress as there are other options making applications publicly accessible. In the meantime while getting started with AKS you can use the load balancer service type for external access to your applications running on AKS.

Monitoring

With AKS you have
the option to utilize Container monitoring from Azure Monitor. This will give
you performance and health monitoring. The monitoring data comes directly from
an AKS cluster or from all the AKS clusters via Azure Monitor more specifically
Log Analytics. In the future I plan to post a deeper blog about monitoring AKS.

If you chose to
enable this you will need to setup a new
Log Analytics workspace or use an existing one.

Tags

You can set tags for
the AKS cluster.

Create

After
all the sections are completed the new AKS will need to validate. After it is
validated click on Create.

Exploring AKS

After the AKS
cluster is created you will see it in Azure under Kubernetes services.

Also you may notice
two new resource groups in your Azure subscription. The first resource group
will be the one you created during the AKS creation. This is the resource group
that will contain the Azure K8s cluster service. If you selected an advanced network
configuration during deployment to create a new VNet you will see that as well.

You will also see a second resource group with a name format similar to MC_ResourceGroupNAME_AKSClusterNAME_REGION. As shown in the following screenshot I have a resource group named MC_AKS12118RG_AKS12118_centralus. This resource group contains the individual AKS cluster resources such as the nodes.

This resource group also contains supporting Azure services like DNS, public IP’s, storage, load balancers, network security groups and more. Note do not make changes to the resources in this resource group directly. You should only make changes through the AKS service and K8s itself. For example when you deploy a new load balancer service in K8s the corresponding Azure load balancer will automatically be created.

Access Kubernetes Dashboard

Next you can access
the K8s cluster via a shell or access the dashboard. Before you can access the
dashboard the service principle account that was created during the AKS
deployment will need a ClusterRoleBinding that assigns the K8s role
dashboard-admin it. Run the following syntax from the Azure cloud shell to do
this: