Project description

Acumos Python Client User Guide

acumos is a client library that allows modelers to push their Python models
to the Acumos platform.

Installation

You will need a Python 3.4+ environment in order to install acumos.
You can use Anaconda
(preferred) or pyenv to install and
manage Python environments.

If you’re new to Python and need an IDE to start developing, we
recommend using Spyder which
can easily be installed with Anaconda.

The acumos package can be installed with pip:

pip install acumos

Protocol Buffers

The acumos package uses protocol buffers and assumes you have
the protobuf compilerprotocinstalled. Please visit the protobuf
repository
and install the appropriate protoc for your operating system.
Installation is as easy as downloading a binary release and adding it to
your system $PATH. This is a temporary requirement that will be
removed in a future version of acumos.

Acumos Python Client Tutorial

This tutorial provides a brief overview of acumos for creating
Acumos models. The tutorial is meant to be followed linearly, and some
code snippets depend on earlier imports and objects. Full examples are
available in the examples directory.

A Simple Model

Let’s first create a simple model that adds two integers together.
Acumos needs to know what the inputs and outputs of your functions are.
We can use the Python type annotation syntax to specify the function
signature.

Below we define a function add_numbers with int type parameters
x and y, and an int return type. We then build an Acumos
model with an add method.

Note: Function
docstrings are included
with your model and used for documentation, so be sure to include one!

defadd_numbers(x:int,y:int)->int:'''Returns the sum of x and y'''returnx+ymodel=Model(add=add_numbers)

Exporting Models

We can now export our model using the AcumosSession object created
earlier. The push and dump APIs are shown below. The dump method will
save the model to disk so that it can be onboarded via the Acumos web UI. The
push method pushes the model directly to Acumos.

Note: Pushing a model to Acumos will prompt you for your username
and password if you have not previously authenticated. There are two ways to
avoid the interactive prompt:

Export the ACUMOS_USERNAME and ACUMOS_PASSWORD environment variables,
which correspond to your username and password used to log into Acumos website.

Export the ACUMOS_TOKEN environment variable, which corresponds to an
authentication token that can be found in your account settings on the Acumos
website.

Defining Types

In this example, we make a model that can read binary images and output
some metadata about them. This model makes use of a custom type
ImageShape.

We first create a NamedTuple type called ImageShape, which is
like an ordinary tuple but with field accessors. We can then use
ImageShape as the return type of get_shape. Note how
ImageShape can be instantiated as a new object.

importioimportPILImageShape=create_namedtuple('ImageShape',[('width',int),('height',int)])defget_format(data:bytes)->str:'''Returns the format of an image'''buffer=io.BytesIO(data)img=PIL.Image.open(buffer)returnimg.formatdefget_shape(data:bytes)->ImageShape:'''Returns the width and height of an image'''buffer=io.BytesIO(data)img=PIL.Image.open(buffer)shape=ImageShape(width=img.width,height=img.height)returnshapemodel=Model(get_format=get_format,get_shape=get_shape)

Note: Starting in Python 3.6, you can alternatively use this simpler
syntax:

fromacumos.modelingimportNamedTupleclassImageShape(NamedTuple):'''Type representing the shape of an image'''width:intheight:int

Using DataFrames with scikit-learn

In this example, we train a RandomForestClassifier using
scikit-learn and use it to create an Acumos model.

When making machine learning models, it’s common to use a dataframe data
structure to represent data. To make things easier, acumos can
create NamedTuple types directly from pandas.DataFrame objects.

NamedTuple types created from pandas.DataFrame objects store
columns as named attributes and preserve column order. Because
NamedTuple types are like ordinary tuple types, the resulting
object can be iterated over. Thus, iterating over a NamedTuple
dataframe object is the same as iterating over the columns of a
pandas.DataFrame. As a consequence, note how np.column_stack can
be used to create a numpy.ndarray from the input df.

Finally, the model returns a numpy.ndarray of int corresponding
to predicted iris classes. The classify_iris function represents
this as List[int] in the signature return.

Check out the sklearn examples in the examples directory for full
runnable scripts.

Declaring Requirements

Custom Packages

If your model depends on another Python package that you wrote, you can
declare the package via the Requirements class. Note that only pure
Python packages are supported at this time.

Assuming that the package ~/repos/my_pkg contains:

my_pkg/
├── __init__.py
├── bar.py
└── foo.py

then you can bundle my_pkg with your model like so:

frommy_pkg.barimportdo_thingdeftransform(x:int)->int:'''Does the thing'''returndo_thing(x)model=Model(transform=transform)reqs=Requirements(packages=['~/repos/my_pkg'])# using the AcumosSession created earlier:session.push(model,'my-model',reqs)session.dump(model,'my-model','~/',reqs)# creates ~/my-model

Requirement Mapping

Python packaging and PyPI aren’t
perfect, and sometimes the name of the Python package you import in your
code is different than the package name used to install it. One example
of this is the PIL package, which is commonly installed using a fork
called pillow (i.e.
pip install pillow will provide the PIL package).

To address this inconsistency, the acumos.modeling.Requirements
class allows you to map Python package names to PyPI package names. When
your model is analyzed for dependencies by acumos, this mapping is
used to ensure the correct PyPI packages will be used.

In the example below, the req_map parameter is used to declare a
requirements mapping from the PIL Python package to the pillow
PyPI package:

reqs=Requirements(req_map={'PIL':'pillow'})

TensorFlow

Check out the TensorFlow example in the examples/ directory of the Acumos Python
client repository.

Testing Models

The acumos.modeling.Model class wraps your custom functions and
produces corresponding input and output types. This section shows how to
access those types for the purpose of testing. For simplicity, we’ll
create a model using the add_numbers function again:

defadd_numbers(x:int,y:int)->int:'''Returns the sum of x and y'''returnx+ymodel=Model(add=add_numbers)

The model object now has an add attribute, which acts as a
wrapper around add_numbers. The add_numbers function can be
invoked like so:

result=model.add.inner(1,2)print(result)# 3

The model.add object also has a corresponding wrapped function
that is generated by acumos.modeling.Model. The wrapped function is
the primary way your model will be used within Acumos.

We can access the input_type and output_type attributes to test
that the function works as expected:

More Examples

Below are some additional function examples. Note how numpy types
can even be used in type hints, as shown in the numpy_sum function.

fromcollectionsimportCounterimportnumpyasnpdeflist_sum(x:List[int])->int:'''Computes the sum of a sequence of integers'''returnsum(x)defnumpy_sum(x:List[np.int32])->np.int32:'''Uses numpy to compute a vectorized sum over x'''returnnp.sum(x)defcount_strings(x:List[str])->Dict[str,int]:'''Returns a count mapping from a sequence of strings'''returnCounter(x)

Acumos Python Client Developer Guide

Testing

We use a combination of tox, pytest, and flake8 to test
acumos. Code which is not PEP8 compliant (aside from E501) will be
considered a failing test. You can use tools like autopep8 to
“clean” your code as follows: