In this section, you will find information on how to run
Palladium-based application with another web server instead of Flask’s
built in solution, and how to benchmark your service. Additionally,
you will find information on how to use provided scripts to
automatically generate Docker images of your service and how to deploy
such Docker images using Mesos / Marathon.

Palladium uses HTTP to respond to prediction requests. Through use of the
WSGI protocol (via Flask), Palladium can be used together with a variety
of web servers.

For convenience, a web server is included for development purposes.
To start the built-in web server, use the pld-devserver command.

For production use, you probably want to use something faster and more
robust. Many options are listed in the Flask deployment docs. If you follow any of
these instructions, be aware that the Flask app in Palladium is available as
palladium.wsgi:app. So here’s how you would start an Palladium prediction
server using gunicorn:

An example configuration to use nginx to proxy requests to gunicorn
is also available. It can be used without modification for our example
and has to be made available in the /etc/nginx/sites-enabled/
folder and is active after a restart of nginx. For convenience it is
reprinted here:

In order to benchmark the response time of a service, existing tools
like Apache Benchmark (ab) or siege can be used. They can be
installed using install packages, e.g., for Ubuntu with apt-get
install apache2-utils and sudo apt-get install siege.

If a web server with the Iris predict service is running (either using
the built-in pld-devserver or a faster solution
as described in Web server installation), the ab benchmarking
can be run as follows:

In this ab call, it is assumed that the web server is available at
port 5000 of localhost and 1000 requests with 10 concurrent requests
at a time are sent to the web server. The output provides a number of
statistics about response times of the calls performed.

Note

If there is an error in the sample request used, response times
might be suspiciously low. If very low response times occur, it
might be worth manually checking the corresponding response of used
request URL.

Note

ab does not allow to use different URLs in a benchmark run. If
different URLs are important for benchmarking, either siege or a multiple URL patch for ab
could be used

Palladium has support for quickly building a Docker image to run your
own application based on the Palladium base image. The Palladium base
image can be pulled from Docker Hub as follows:

docker pull ottogroup/palladium-base

As an example, let’s build a Docker image for the Iris example that’s
included in the source. We’ll use the Palladium base image for
version 1.0, and we’ll name our own image my-palladium-app. Thus,
we invoke pld-dockerize like so:

This command will in fact create two images: one that’s called
my-palladium-app, another one that’s called
my-palladium-app-predict. The latter extends the former by adding
calls to automatically fit your model and start a web server.

By default pld-dockerize will create the Dockerfile files and
create the Docker containers. You may want to create the Dockerfile
files only using the -d flag, and then modify files
Dockerfile-app and Dockerfile-predict according to your needs.

Your application’s folder (examples/iris in this case) should look
like this:

This section describes how to setup Mesos / Marathon with a
containerized Palladium application. If you have not built a docker image
with your Palladium application yet, you can follow the instructions that
are provided in the Building a docker image with your Palladium
application section.

An easy way to add a new application to Marathon is to use its REST
API. For this task you need a json file which contains the relevant
information for Marathon. A basic example of the json file could look
like this:

You have to replace the Docker image name, port number (currently set
to 8000) and - if there is any dependency - specify links to other
containers. If you have a Docker image of the Iris service available
(named user/palladium-iris-predict:0.1), you can use this file:

You can see the status of your Palladium service instances using the
Marathon web user interface (available at http://localhost:8080 if
you run the single node installation mentioned above) and can scale
the number of instances as desired. Marathon keeps track of the Palladium
instances. If a service instance breaks down, a new one will be
started automatically.

Sometimes you will want the Palladium web service’s entry points /predict
and /alive to be secured by OAuth2 or similar. Defining
predict_decorators and alive_decorators in the Palladium
configuration file allows you to put any decorators in place to check
authentication.

Let us first consider an example where you want to use HTTP Basic
Auth to guard the entry points. Consider this code taken from the
Flask snippets repository:

# file: mybasicauth.pyfromfunctoolsimportwrapsfromflaskimportrequest,Responsedefcheck_auth(username,password):"""This function is called to check if a username / password combination is valid. """returnusername=='admin'andpassword=='secret'defauthenticate():"""Sends a 401 response that enables basic auth"""returnResponse('Could not verify your access level for that URL.\n''You have to login with proper credentials',401,{'WWW-Authenticate':'Basic realm="Login Required"'})defrequires_auth(f):@wraps(f)defdecorated(*args,**kwargs):auth=request.authorizationifnotauthornotcheck_auth(auth.username,auth.password):returnauthenticate()returnf(*args,**kwargs)returndecorated

The requires_auth can now be used to decorate Flask views to guard
them with basic authentication. Palladium allows us to add decorators to
the /predict and /alive views that it defines itself. To do this,
we only need to add this bit to the Palladium configuration file:

Of course, alternatively, you could set up your mod_wsgi server to
take care of authentication.

Using Flask-OAuthlib to
guard the two views using OAuth2 follows the same pattern. We will
configure and use the flask_oauthlib.provider.OAuth2Provider
for security. In our own package, we might have an instance of
OAuth2Provider and a
require_oauth decorator defined thus:

Alternatively, to get more decoupling from Palladium’s Flask app, you
can use the following snippet inside your Palladium configuration and
assign the Flask app to
OAuth2Provider at application
startup: