Kubernetes Lab using Openstack

REASON:Previously, I mentioned that there were some issues with using Fedora 23 Alpha/Beta and Kubernetes release 0.16.git388061f.fc23 vs. release 0.5.gite44c8e6.fc23. I have since found the issue (these are typically user errors anyway), and have corrected them in the document. I will display both versions in case someone runs into an older version of release 0.5.gite44c8e6.fc23.

UPDATE:Thurs, 15-10-21 at 11:23PM

REASON:Oh, the things we do to stay updated! As described in my last update (Tues, 15-10-06 at 9:14PM), you will need to carefully understand the differences between the two releases of Kubernetes on Fedora 23, the one in the fedora.repo and the one in the fedora-updates-testing.repo. Well, the folks at Fedora decided that when you perform a dnf update -y to disable the fedora-updates-testing.repo automatically. My assumption is that they're attempting to stabilize the release before GA, or that this repository was causing issues somewhere. This will cause issues with etcd, since the stable repository contains version 2.2.0r1.fc23 and the other contains version 2.0.13r3.fc23. I am going to suggest that we use the fedora-updates-testing.repo since we've determined the syntax differences with the Kubernetes configuration files located in /etc/kubernetes/. Don't let this confuse you, just pay close attention to the releases of packages you're using with my instructions below. Please ask me any questions on the Kubernetes-Users Slack Group. My handle is v1k0d3n.

So now we get to have some fun. If you're new to Openstack or Microservices, I would suggest that you read Part 1 before taking on this task. There have been a lot of people who have tried to learn Kubernetes with great intentions and have ultimately become frustrated with the overabundance of information, bringing their journey to an unhappy end. The purpose of this article is to help folks learn Kubernetes in a repeatable fashion, so they can install the environment (hopefully using my Openstack deployment recommendations), destroy it, and start all over again. It's the whole growth mindset attitude.

Getting Started

The first thing you will need to do is obtain the images used for your Openstack environment. We are going to use two different instances, both of which are Red Hat variants.

I must mention that these images are aggressive development builds; they're beta and may deviate from this walkthrough at any given time! Please look at the date of this article for reference, but I'll tell you the key things we'll need from these images:

At the time of writing, there are two versions of Kubernetes available. We're going to pick only v1.1.0 Release 0.5. There are reasons we'll get to later.

The second really important part is the Kubernetes plug-in for Cockpit. It's not really needed, but it's an impressive piece of work that Red Hat has been putting together, based on the Kubernetes API:

Make sure to use your ~/keystonerc_admin file to set the environment variables you'll need to work with and configure Openstack (additionally, make sure that keystonerc_admin is configured correctly for your tenant/project name):

[bjozsa@galvatron projects]# . /home/bjozsa/keystonerc_admin

Next, make sure you understand your environment, but using the following commands:

openstack image list = <IMAGE>

openstack keypair list = <KEYPAIR-NAME>

openstack flavor list = <FLAVOR-ID-NUMBER>

openstack security group list = <SECGRP> (can be more than one separated by an inline comma; example <SECGRP1>,<SECGRP2>)

Now, you're going to bring up one instance of a master, which you will use the Fedora 23 image for. The minions (actually, Kubernetes calls them nodes now) will be created from the CentOS Atomic image.

Master = Fedora 23 Cloud Base (1)

Docker Cache = CentOS Atomic (1)

Nodes/Minions = CentOS Atomic (5)

Create a Docker Repository Mirror

Now, I created my own Docker Registry caching server (which is coincidently run from a container in CentOS Atomic). What this does is force my minions to obtain docker images from the same controlled source, and it allows images to be reused quicker then they would be if I had to fetch them for each pod turn up. Just think about it for a second; you build this ultra-flexible containerized network, but you're delayed every single time you need to fetch images directly from Docker Hub. Na...no thanks! I prefer to load them once, cache them, and then when my pods are killed and brought back up (think about rolling updates, or when a pod is brought down by Kubernetes), the start-up time is drastically reduced.

In order to do this, you will need to bring up another CentOS Atomic instance. After the instance is up, run the following command to start the container:

And that's it! Point your Master/Minions to this caching server in order to speed up your Kubernetes pod startup/load times.

Create the Kubernetes Master

Hopefully all of your instances built without issues. Everything should have gone pretty smoothly, but if they didn't...don't sweat it! Delete them and start over! That's the great thing about having a home lab, nobody cares if you blow it up and it's a great place to learn.

So if you've gotten this far, you're ready to continue setting up your Kubernetes Master.

Log into your Kubernetes Master (we're going to call this fedmaster from now on):

ssh -i ~/<key> fedora@192.168.1.xx

Note: fedora is the default username for the Fedora 23 Alpha image.

In your instance, you'll notice that firewalld is not enabled on Fedora 23 Alpha by default. In case this gets turned on in the future, perform the following commands to disable it:

Turn SELinux to "permissive" if you are using this in a testing environment. (I'm sure the Atomic/RH guys are rolling their eyes right now).

sudo vi /etc/sysconfig/selinux
# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
# enforcing - SELinux security policy is enforced.
# permissive - SELinux prints warnings instead of enforcing.
# disabled - No SELinux policy is loaded.
SELINUX=permissive
# SELINUXTYPE= can take one of three two values:
# targeted - Targeted processes are protected,
# minimum - Modification of targeted policy. Only selected processes are protected.
# mls - Multi Level Security protection.
SELINUXTYPE=targeted

Perform an update (This is where things could get risky, but it needs to be done.)

sudo dnf -y update

Reboot.

sudo reboot

Install Prerequisites, but note(!) this is where you can potentially run into issues.

Option A: In case Kubernetes is broken in Fedora Base 23 (as it is in v1.1.0 Release 0.16). This is the state as of Sept 18th 2015. I told the Fedora team about it over IRC, but they weren't interested at all!

Option B: If you want to take your chances! With this option, you will get the latest updates available, but you may have to later remove broken package, and rerun the installation using the command above (referring to disabling the repo: --disablerepo=updates-testing).

Note: Throughout this lab walkthrough I've tried to use DNS where it can be used. For the Kubelet configuration, make sure that you use IP addresses where I have them identified. I've tested using named values, and they work just as I have them.

Start using Atomic and Kubernetes

If you've made it this far, you should really give yourself a huge pat on the back. You've made it farther than a lot of other people who've given up just because they've struggled through the gauntlet of terms. Many people still struggle with what a true Microservices architecture is, but here you are. You're ready to start using your first Microservices deployment. Congratulations to you!

Now we're ready to start using everything. I'm going to get you started, but I will document most of the user functionality in other articles.

The Cockpit Interface

The first place you may want to start is with the Cockpit interface. Cockpit originally was a way to visually manage your server; however, Cockpit-Kubernetes also allows you to visually manage your Kubernetes clusters. This is powerful, and in reality, this shows some of the awesome tools the community is developing around the Kubernetes API. Let's get started and check out some of the things we can do with Cockpit.

To get started, point your browser to your Kubernetes Master fed master. In our example, this is IP address 192.168.70.62, and Cockpit runs on Port 9090 (so you're going to need to change the security policy for Openstack):

User: fedora Password: atomic

When you're in, you'll notice some management options for the host. But it doesn't end there! Go to the details page, where you'll be able to add the other hosts for management in a single Cockpit instance:

Note: If you click on the + icon, you can add the additional hosts in your cluster. Use their IP address (as given out by Openstack), and de-select "Login with my current credentials" as the CentOS Atomic host uses the default username of CentOS (which you've already figured out by now).

After you've added your hosts, you'll notice that each server in your cluster can be centrally managed. That's pretty awesome!

But this walkthrough isn't really about Cockpit. Cockpit is still just another building block that can be used, but the folks at Red Hat have done a great job with it. I'm impressed, and I'm very happy it's included in Fedora 23.

What we're really after is the Kubernetes management piece. So click on the cluster tab within Cockpit.

When you have running Replication Controllers, Services, and Pods the interface will allow you to view the details of those elements, as well as see them brought up by Kubernetes.

To be perfectly clear, in the picture above I was able to turn up nearly 400 containers in approximately 30 seconds with a single command. Try doing that with a virtual machine!

You can open a terminal session directly in a container within the Kubernetes pod with Cockpit.

You can view all the details for each of the Replications Controllers, Services, and Pods.

Finally, if you prefer to launch an RC, SVR, or POD from the Cockpit interface, you can do that also.

So now you see how powerful tools like Cockpit and the tools built around the Kubernetes API and Atomic/Fedora really are. Pretty awesome stuff! Now you're ready to test on your own. Come back next week for details on Kubernetes and I'll instruct users how to bring up their own Vagrant demo (or use the Openstack lab we've already built) to show Kubernetes in action during a rolling update! For now, enjoy, test, destroy, and learn!