​Demand for Kubernetes is through the roof...

...But does the skills gap make DevOps a security risk?

06-11-2018

The container orchestration platform Kubernetes (aka K8s*) has been massive in 2018. Here at Framework Training, we've seen demand for K8s training reflect this rapid climb in requests for Kubernetes training courses. Another measure of this profound increase in Kubernetes usage is found in the job-ads data.

According to survey findings from the well-used IT Jobs Watch website – now into its 14th year – demand for Kubernetes skills has surged by 752 per cent since 2016.

It’s quite a leap – though of course it also mirrors what is going on out there, as K8s sweeps through companies and microservices architectures built on container technologies move into the IT mainstream.Scrutinise the data further and we also find that security skills for DevOps are predictably also in rising demand, while companies continue their search for experienced Kubernetes developers and engineers in ever-bigger numbers to deliver on their DevOps plans.

Where does K8s sit in the IT skills charts today? Well, K8s developer demand is up a cool 729 places in the rankings, says IT Jobs Watch’s owner CyberArk, meaning K8s developers are now among the top 250 most needed roles in IT today.

“Kubernetes has become a massive money word, and these figures show that DevOps teams are seeking more skills to help them manage and deploy applications at scale,” says Josh Kirkwood, who is the DevOps Security Lead at CyberArk.

Kirkwood also argues there is a danger that, in the rush to gain IT and business advantage in DevOps, security is being pushed down the agenda. OK, so he works in security and would say that, but it’s a very valid point.

“If privileged accounts in Kubernetes are left unmanaged, and attackers get inside the control panel, they could gain control of an organisation’s entire IT infrastructure,” warns Kirkwood, ramping things up some more.

At Framework Training, I suppose we see things in a different light. Many companies approach us looking to boost their expertise in K8s, but it’s fair to say that they aren’t necessarily rushing with undue haste towards an insecure DevOps environment. Quite the opposite, very often – most are being methodical about getting any transition right, and having staff trained up to plug knowledge gaps in good time.

All the same, DevOps security certainly matters a lot. Here are four K8s basics to help you get off on the right foot.

Use strong authentication All Kubernetes components must be authenticated or else an entire Kubernetes environment is susceptible to attack. It’s how Tesla was breached recently, in an incident that had intruders running crypto-mining containers in Tesla's Kubernetes cluster.

Enforce least privileges Authentication gets someone in, but authorisation sets what they can do with that access. Least privileges is the principle that should guide you. Make sure RBAC is enabled to enforce a least-privileges model. From K8s 1.9 onwards, there’s also a pod security policy that means you can restrict the behavior of a pod. It allows you to define a set of conditions that must be met before a pod is authorised to run.

Segment every cluster Segmentation limits the scope of any attack. If there’s a breach, you can contain it to a subset of your deployment. One approach is to use K8s namespaces to create virtual clusters that are separated from one another even when sharing the same infrastructure.

Monitor monitor monitor Monitoring the environment matters. Even if you have worked hard to lock down your Kubernetes cluster, there could still be breaches. So detect and mitigate – for example by using K8s advanced auditing with a webhook to an external analytics and monitoring tool. For those running production environments for customers, dedicated security tools that can detect policy violations and automatically prevent attacks from spreading are also worth investigation.

Remember, the CIS Kubernetes Benchmark includes all the agreed best-practice checks to perform on your nodes. It’s a great point of reference for any project.