Training Flags

Defining Flags

Tuning a model often requires exploring the impact of changes to many hyperparameters. The best way to approach this is generally not to progressively change your source code, but rather to define external flags for key parameters which you may want to vary. This allows you to easily try out different combinations of parameters without changing your code, and makes your training script easier to integrate with hyperparameter tuning tools.

Use the flags() function to define a set of flags (and default values) for a training script. For example,

Note that instead of literal values for the various parameters we want to vary we now reference members of the FLAGS list returned from the flags() function.

Using Flags

When you execute your training script either interactively or using the source() function then the default values defined for your FLAGS will be utilized. There are a number of ways to train with alternate flag values, each of which is described below.

Command Line

If you execute your training script via Rscript, you can vary the values of FLAGS by passing them on the command line.

For example, to run your training script with alternate dropout values you would do this:

Note that for the learning_rate flag we used a dash (“-”) rather than underscore on the command line. Either is valid (dashes are automatically converted to underscores). This accommodation is made because dashes are not valid R identifiers but are commonly used within command line arguments.

Config File

You can also create a YAML configuration file (by default “flags.yml”) with alternate flag values. For example:

See the documentation for the config package for additional details on using named configuration profiles.

tfruns

The tfruns package provides a suite of tools for tracking, visualizing, and managing TensorFlow training runs. The package includes integrated support for both invoking scripts with flags via the training_run() function as well as tracking flag values used within each training run.

Here are some example uses of training_run() that mirror the flag examples from above: