Hybrid Cloud As-A-Service

Bay Area Mesos User Group

Benjamin Hindman and Jörg Schad, Mesosphere

Jun 14th, 2018

San Francisco, CA, USA

Abstract

For this meetup we are excited to be joined by two great speakers! Ben Hindman, Co-Founder of Mesosphere and Creator of Apache Mesos, and Joerg Schad, Technical Lead for Community Projects will be talking about running Kubernetes-As-A-Service alongside Apache Spark, HDFS, Data Science and more.

Kubernetes helps developers get their containers up and running. But what’s the best way to run Kubernetes itself? What if you want to give each of your teams their own Kubernetes cluster?

Come here us talk about why and how we built Kubernetes-as-a-Service. Moreover, we’ll describe the other “as-a-Service” options you can run along with Kubernetes, such as Kafka, Cassandra, HDFS, and Spark, in order to quickly get a stack deployed to help with your data engineering and data science projects. We’ll finish it all off with a live demo that puts all the pieces together!

2) Kubeflow++:

In the second part we will look at how this concept extends to Data Science.

Kubeflow is makes it very easy for data scientist to build their own data science pipeline with Jupyter Notebooks, TensorFlow, TensorBoard and Model serving. However, to build a production grade data science pipeline requires some additional components.

The choice of tools often depends the concrete environment Kubeflow is running in: While Cloud Providers such as Google Cloud, AWS, or Azure have managed solutions for these challenges, an on-prem environment might require alternative Open-Source solutions.

So in this talk we look at building a complete deep learning pipeline around Kubeflow and answer topics such as: