Using Saved Models

Overview

The main goal of the tfdeploy package is to create models in R and then export, test, and deploy those models to environments without R. However, there may be cases when it makes sense to use a saved model directly from R:

If another R user has saved and/or deployed a model that you would like to use for predictions from R.

If you want to use a saved or deployed model in a Shiny application.

If you want to compare predictions between a saved or deployed model and a new model that is under development.

One way to use a deployed model from R would be to execute HTTP requests using a package like httr. For non-deployed models, it is possible to use serve_savedmodel() - as we did for local testing - along with a tool like httr. However, there is an easier way to make predictions from a saved model using the predict_savedmodel() function.

Example

Using the same MNIST model described previously, we can easily make predictions for new pre-processed images. For example, we can load the MNIST test data set and create predictions for the first 10 images:

Just like the HTTP POST requests, predict_savedmodel() expects the new instance data to be pre-processed.

predict_savedmodel() requires the new data to be in a list, and it always returns a list. This requirement faciliates models with more complex inputs or ouputs.

In the previous example we used predict_savedmodel() with the directory, ‘savedmodel’, which was created with the export_savedmodel() function In addition to providing a path to a saved model directory, predict_savedmodel() can also be used with a deployed model by supplying a REST URL, a CloudML model by supplying a CloudML name and version, or by supplying a graph object loaded with load_savedmodel().

Model Representations

There are a few distinct ways that a model can be represented in R. The most straightforward representation is the in-memory, R model object. This object is what is created and used while developing and training a model.

A second representation is the on-disk saved model. This representation of the model can be used by the *_savedmodel functions. As a special case, load_savedmodel() creates a new R object pointing to the model graph. It is important to keep in mind that these saved models are not the full R model object. For example, you can not update or re-train a graph from a saved model.

Finally, for Keras models there are 2 other representations: HDF5 files and serialized R objects. Each of these represenations captures the entire in-memory R object. For example, using save_model_hdf5() and then load_model_hdf5() will result in a model that can be updated or retrained. Use the serialize_model() and unserialized_model() to save models as R objects.

What represenation should I use?

If you are developing a model and have access to the in-memory R model object, you should use the model object for predictions using R’s predict function.

If you are developing a Keras model and would like to save the model for use in a different session, you should use the HDF5 file or serialize the model and then save it to an R data format like RDS.