Faq

Access K8s Sparkfabrik Cluster

Introduction

Since late 2016 Sparkfabrik's internal services (Gitlab, CI/CD pipelines, SparkBoard, etc) are running into a Kubernetes cluster hosted on GKE/GCP.

This means that all intermediate environments other than local and production (so integrations, branch builds, epic builds, etc) run in pods into a Google Cloud Engine elastic cluster. The following guide will help you configure your local environment so that you will be able to access services inside pods, open shells into them, read relevant logs and - ultimately - devops all the things! :)

Step 1: Authentication to Google Cloud

As said, the K8s cluster is running over Google Cloud infrastructure. To access it we first need to authenticate on GCP.Rejoy! Your sparkfabrik.com account is enough to perform authentication, but you'll need to open a terminal and install gcloud CLI tool. Follow the link to get gcloud running on your OS.

Once done, you can authenticate running

$ gcloud auth login

Provide your sparkfabrik.com credentials.

Now configure the gcloud docker integration running:

$ gcloud auth configure-docker

Step 2: Accessing the K8s cluster

Access to the cluster and pods therein will happen using K8s CLI tool kubectl.

On MacOSX gcloud command has all that we need to make it work:

$ gcloud components install kubectl

While Ubuntu users can enjoy apt:

$ sudo apt install kubectl

Once kubectl is installed gcloud command will allow us to access the GKE cluster.gcloud CLI manages so many GCP services and areas that there are commands specific to each one. To tame the complexity, all commands are grouped and subgrouped.

Right now, the container group is what we need: it contains groups of commands by which we can manage GKE aspects, like clusters, node-pools, Container Registry images, and so on.

We are going to use a command in the clusters subgroup of the container group to gain access to the cluster. That command is get-credentials which fetches credentials for already running clusters.

Now, the get-credentials command takes a single parameter which is the cluster name. In our case it is spark-op-services. In additions there is a mandatory flag that specifies the region and the datacenter zone inside the region (namely, where is the cluster phisically running?): --zone.

Last but not least, there is a global flag (not specific to the get-credentials command), which is --project. Projects in GCP are similare to realms. Not to be confused with K8s namespaces, (quoting GCP docs)

[...] projects form the basis for creating, enabling, and using all GCP services including managing APIs, enabling billing, adding and removing collaborators, and managing permissions for GCP resources.

So let's specify the correct spark-int-cloud-services, that is the project that holds all the production services in Sparkfabrik.

Beware: environment for customers' projects CI are not customers' assets, they are Sparkfabrik assets, payed and managed by us. That's why accessing these environments involves our production project!

A laconic message should inform you that kubeconfig generated an entry for spark-op-services. No frills but you can pat yourself a shoulder. You're done.

Step 3: Fetching info from clusters

OK, we gained access to the cluster. Mind that's the access is read only, but you have execution permissions (namely you can run kubectl exec) so you can enter running pods.

Let's test if our access is working after all. Run

$ kubectl cluster-info

and you should get a response in the lines of

Kubernetes master is running at https://<IP address>
GLBCDefaultBackend is running at https://<IP address>/api/v1/namespaces/kube-system/services/default-http-backend:http/proxy
Heapster is running at https://<IP address>/api/v1/namespaces/kube-system/services/heapster/proxy
KubeDNS is running at https://<IP address>/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy
Metrics-server is running at https://<IP address>/api/v1/namespaces/kube-system/services/https:metrics-server:/proxy

If not check you followed all previous steps correctly.Mind though that depeding on permissions on your account the output of this command may differ and you can see only a subset of the information and/or a specific error message. Keep this in mind before banging your head to the wall.

Step 4: Namespaces

We mentioned projects, which is GCP realms to address accountability, ACLs and other "administrative" aspects related to the GCP services.

Projects as never to be confused with namespaces. The concept of namespace here is intended as typical of Kubernetes: K8s namespaces allow to segment the same "physical" cluster in reserved spaces, like they are separate clusters.

This makes us sure critical ops won't concur for resources or won't hinder each other in case of malfunctioning, at a cluster level.

We use this feature to make sure each Gitlab project (again, not to be confused with GCP projects: we mean each customer or internal product) that needs build environments in Gitlab, lives in its own namespace.

Some of the preceding namespaces are real. As you can see names are pretty self-explaining (at least the ones related to projects). But if you are in doubt you can check Gitlab to see which namespace is in use by a specific Gitlab projects.Follow Settings -> Integrations -> Kubernetes -> Namespace in the project page to make sure (proper permissions may be necessary, ask your team leader if you can't access that section).

Gotcha: Please remind that not all containers have bash. Some (many actually) of them are based on Alpine Linux or other distros so the available shell may vary.Alpine for example sports ash so you may have to issue

$ kubectl -n acme exec -it acme-ash-test -- /bin/ash

Conclusions

This is a small recipe to get you started with our production K8s environments. From here up it's a matter of experience, docs reading and a bit of work by you to increase your devops skills.