Local Development Environment for Kubernetes using Minikube

Kubernetes can be an ultimate local development environment particularly if you are wrangling with a large number of microservices. In this post, we will cover how you can create a local development workflow using Minikube and tools such as Make to iterate fast without the wait imposed by your continuous integration pipeline. With this workflow, you can code and test changes immediately.

Traditionally, you can test your code changes by rebuilding the Docker images (either locally or via your continuous integration pipeline), then pushing the image to a Docker registry after the successful build, and then redeploy to your Kubernetes cluster.

Overall development workflow

Here is our simplified development workflow,

Make changes in your code on your local laptop

Build local Docker images and deploy them to Minikube

Test your code changes after deploying into Minikube

If changes are good, commit them to version control repository

Your version control system triggers continuous integration pipeline

Continuous integration builds Docker images and push it your registry

We are mainly interested in the fast iteration of step 1-3. In later sections, we will discuss how we can streamline step 1-3 using GNU Make and other tools.

Prerequisite

Before we get started, let's have a look at various dependencies you need to install in order to make this workflow work for you.

Docker

As we are dealing with Docker images, I assume you have downloaded and installed Docker for your operating system. I use Docker for Mac.

VirtualBox

You will require VirtualBox (or a similar VM driver such as xhyve driver or VMware Fusion depending on your operating system) which enables Minikube to run a single-node Kubernetes cluster inside a VM. So make sure you have downloaded VirtualBox and installed it.

Kubectl

Kubectl is a command line utility (CLI) for running commands against Kubernetes clusters. You need kubectl to interact with your local Kubernetese cluster (i.e. Minikube) as well as remote clusters.

Kubectl Basics

There are some basic kubectl commands and concepts you should get familiar with.

Using kubectl you can interact with various Kubernetes clusters by setting an appropriate context. Typically a kubeconfig file is used to configure access to one or more clusters and include details required for a context. On Mac, this can be found at ~/.kube/config. For more details please review this page on how to configure access to multiple clusters by using configuration files.

Minikube Basics

Starting the Minikube will create a single-node Kubernetes cluster inside a VM. Minikube start command also creates minikube context and set it as default.

minikube start
minikube status

Kubernetes dashboard is a web-based user interface. You can create and modify individual Kubernetes resources from this UI. It also provides information on the state of Kubernetes resources in your cluster.

By default, Minikube comes with dashboard add-on enabled, and we can access the Kubernetes dashboard in the browser by typing following command.

minikube dashboard

Another way to access dashboard is to use kubectl. Run the following command in your terminal:

In addition, Minikube provides a range of Kubernetes versions to run. You can print a list of available versions to use,

minikube get-k8s-versions

Then start Minikube by passing a specific version,

minikube start --kubernetes-version v1.7.0

In addition, you can run Minikube with more customised configurations such as you can specify a different network plugin than default or use a different virtual machine driver rather than VirtualBox. You can also change default allocation of memory and CPU.

Under the hood, it is using InfluxDB as the storage backend for metric data and Grafana as visualization UI.

minikube addons enable heapster
minikube addons open heapster

Once enabled we can open Grafana UI (to interact with Heapster) in the browser.

I also highly recommend enabling Ingress functionality for your Minikube,

minikube addons enable ingress

Kubernetes Config Generator

When working with Kubernetes, creating and managing config files can be a really tedious job. I highly recommend kubegen - a tool which can simplify your day-to-day Kubernetes workflow, by automating the generation of boilerplate config codes for you and letting you focus on the important things.

Assuming you have Node JS and NPM already installed on your development machine, to install kubegen you need to execute the following command:

npm install -g generator-kubegen

Basically, kubegen is a Yeoman generator. When we run yo kubegen on terminal it ask few questions and based on your inputs it creates config files such as deployment.yml (deployment config), svc.yml (service config), ing.yml (ingress config).

Rapid Development Cycle

Now that we have setup our local Kubernetes development environment lets take it for a ride. We are going to build a Python application message-api. This application will be deployed inside our local Minikube cluster.

Make changes in your code on your local laptop

To start with, we are going to keep this application quite simple. It has

A Dockerfile

A requirements.txt with only one dependency flask

An app.py with a list of flask routes and view functions

Now let's review the content of Dockerfile. Dockerfile has instructions to install requirements for the application and run an application server which enables this application to handle HTTP request/response on container port 5000.

Now all we need is to run make update command after every code change and we will see update results in Minikube.

Test your code changes after deploying into Minikube

Open or get URL for service message-api so we can validate the changes,

minikube service hello-python
minikube service --url hello-python

My workflow

I generally use a more evolved version of workflow described in the previous section. Using kubegen I create Kubernetes YAML files for deployment, service and ingress resources.

I use config files generated by kubegen in my configurable Makefile. On header section of the file, you can change the name of the repo (used for naming image, deployment, service, and ingress). You can edit namespace and path of config files along with version strings. You can an view example Makefile snippet below. As you can see, there are a few new make commands to streamline my Kubernetes development workflow,

make create builds the local image, then uses that image to create a deployment which is exposed as service.

make ingress creates an ingress using the config files specified in the header section (IFILE)

make update creates a time stamped version of the image and update the deployment to use this new image

The big picture

In this post, we mainly focused on the local development environment using Minikube. As illustrated below this is a part of the bigger picture i.e. overall end-to-end workflow. In upcoming posts, we will cover other tools with the potential to fast-track development activities utilizing Kubernetes cluster.