Kubernetes for Python Developers: Part 1

A Kubernetes tutorial for Python developers

Kubernetes is an open-source container-orchestration system for automating deployment, scaling and management of containerised apps.

Kubernetes helps you to run, track and monitor containers at scale. It has become the de facto tool for container management.

Kubernetes is the largest and fastest growing open-source container orchestration software.

This blog post is the first part of a series: Kubernetes for Python developers.

Our goal is to migrate a Celery app app we developed in a previous blog post from Docker Compose to Kubernetes.

You do not need any Kubernetes knowlegde to follow this blog post. You should have some experience with Docker.

In this first part of the series, you will learn how to set up RabbitMQ as your Celery message broker on Kubernetes.

You will learn about kubectl, the Kubernetes command line interface. And by the end of this article you will know how to deploy a self-healing RabbitMQ application with a stable IP address and DNS name into the cluster.

In order to run Kubernetes on your machine, make sure to enable it. You can find instructions here.

kubectl

First you need to know is kubectl. kubectl is the kubernetes command line tool. It is the docker-compose equivalent and lets you interact with your kubernetes cluster.

For example, run kubectl cluster-info to get basic information about your kubernetes cluster. Or kubectl logs worker to get stdout/stderr logs. Very similar to docker-compose logs worker.

Pods

You cannot run a container directly on Kubernetes. A container must always run inside a Pod. A Pod is the smallest and most basic building block in the Kubernetes world.

A Pod is an environment for a single container. Or a small number of tightly coupled containers (think log forwarding container).

A Pod shares some of the properties of a Docker Compose service. A Pod specifies the docker image and command to run. It allows you to define environment variables, memory and CPU resources.

Unlike a Docker Compose service, a Pod does not provide self-healing functionality. It is ephemeral. When a Pod dies, it’s gone.

Nor does a Pod come with DNS capabilities. This is handled by a Service object which we will cover further down. Pods are much lower level compared to Docker Compose services.

What happened here? We deleted the ephemeral Pod rabbitmq-rs-fxdqp. The ReplicaSet then noticed that the actual number of RabbitMQ Pods running was 0. And it created a new RabbitMQ Pod instance named rabbitmq-rs-5sldl. We have a self-healing RabbitMQ instance.

Delete the ReplicaSet and confirm the ReplicaSet and any RabbitMQ Pods are gone:

RabbitMQ 3.7.8-management is successfully deployed, replacing RabbitMQ 3.7.8 and giving you access to the RabbitMQ management plugin. You now know how to create and deploy a self-healing RabbitMQ Kubernetes instance!

Services

We still lack a stable Pod IP address or DNS name.

Remember that Pods are not durable. When a Pod dies, the ReplicaSet creates a new Pod instance. The new Pod’s IP address differs from the old Pod’s IP address.

In order to run a Celery worker Pod, we need a stable connection to the RabbitMQ Pod.

Enter Services. A Kubernetes Service is another Kubernetes object. A service gets its own stable IP address, a stable DNS name and a stable port.

Services provide service discovery, load-balancing, and features to support zero-downtime deployments.

Kubernetes provides two types of Services.

A ClusterIP service gives you a service inside your cluster. Your apps inside your cluster can access that service via a stable IP address, DNS name and port. A ClusterIP service does not provide access from outside the cluster.

A NodePort service provides access to a Pod from outside the cluster. And everything a ClusterIP service provides.

Make the RabbitMQ Pod available inside the cluster under the service name rabbitmq and expose 5672.