Bias comes in a variety of forms, all of them potentially damaging to the efficacy of your ML algorithm. Read how Alegion's Chief Data Scientist discusses the source of most headlines about AI failures here.

Seldon Core is a machine learning platform that helps your data science team deploy models into production. It provides an open-source data science stack that runs within a Kubernetes cluster. Seldon supports models built with TensorFlow, Keras, Vowpal Wabbit, XGBoost, Gensim, or any other model-building tool.

It includes an API for two key endpoints:

Predict: Build and deploy supervised machine learning models created in any machine learning library or framework at scale using containers and microservices.

Recommend: High-performance user activity and content-based recommendation engine with various algorithms ready to run out of the box.

Weave Cloud and Seldon

Weave Cloud works alongside machine learning platforms such as Seldon’s. It provides the tools to manage, troubleshoot, and monitor machine learning models as well the infrastructure on which it is running

Weave Cloud consists of:

Deploy: Plug the output of your CI system into a cluster so that you can ship features faster.

Explore: Visualize and understand what’s happening so that you can fix problems faster.

Monitor: Understand the behavior of running systems using Prometheus so that you can identify problems faster.

Deploying the MNIST Microservice to Kubernetes

In this tutorial, you will deploy a predictive service that recognizes drawn numbers from 0 to 9. The predictive model was created using TensorFlow. This example describes how to deploy the pre-packaged Docker image that is available in the Seldon server.

We’ll show you how to launch the service to Kubernetes in Google Container Engine and then you’ll manage the outputs of the model and visualize the cluster with Weave Cloud.

Let’s get started!

Create the Cluster in Google Container Engine (GKE)

Connect to your cluster by clicking the Connect button in the GUI by copying the command to install the kubectl binaries in the Google Cloud Shell (or you can set up your command line to use with GKE).

Sign up to Weave Cloud and then install the agents by copying the command shown to you after selecting: Kubernetes > GKE from the setup screens. Run the command in your Cloud Shell.

Next, CD to seldon-server/kubernetes/conf` and run `vi Makefile`. Because you are running in Google Cloud you’ll also need to set the `endpoint to loadbalancer` for the Seldon API before running a make clean:

SELDON_SERVICE_TYPE=LoadBalancer

SELDON_SERVER_SERVICE_TYPE=LoadBalancer

You will also need to change the spark-ui username and password:

SPARK_UI_USERNAME=someone

SPARK_UI_PASSWORD=something

Note: Depending on which shell you’re using, you may need to run `sudo apt-get install apache2-utils` to install .htpassword (if needed).

Run `make clean conf`.

Launch the Seldon server with `seldon-up`.

Depending on the speed of your network, it may take some time for all of the Docker images to download and deploy (up to 10-15 minutes in some cases). In the meantime, go to Weave Cloud and click Explore > Pods to see the Seldon cluster appearing:

Seldon pods deploying to Kubernetes

Deploy the TensorFlow Job and View It in Weave Cloud

The MNIST microservice should already be running once you’ve installed the Seldon service. Next, create the Kubernetes job to load the model:

Test the Output

The microservice takes as input a vector of 784 floats that corresponds to the pixels of a 28x28 image and returns a list of probabilities for each number between 0 and 9. To test it, use the Flask web app created for this purpose.

Load the Demo App

User the <external-IP>: 80to view the client as shown below:

Troubleshoot Predictive Models With Weave Cloud

A useful troubleshooting feature in Weave Cloud is the ability to view real-time logs while you are testing your system. Below, the logs to the Kafka stream predictions remain open so that you can see the parameters being passed back and forth from the model.

KafkaStreaming predictions log available for troubleshooting

Wrapping Up

Your machine learning project needs enormous amounts of training data to get to a production-ready confidence level. Get a checklist approach to assembling the combination of technology, workforce and project management skills you’ll need to prepare your own training data.