Method Detail

validateDataSets

Validate DataSet objects saved to the specified directory on HDFS by attempting to load them and checking their
contents. Assumes DataSets were saved using DataSet.save(OutputStream).
Note: this method will also consider all files in subdirectories (i.e., is recursive).

validateDataSets

Validate DataSet objects saved to the specified directory on HDFS by attempting to load them and checking their
contents. Assumes DataSets were saved using DataSet.save(OutputStream).
This method (optionally) additionally validates the arrays using the specified shapes for the features and labels.
Note: this method will also consider all files in subdirectories (i.e., is recursive).

featuresShape - May be null. If non-null: feature arrays must match the specified shape, for all values with
shape > 0. For example, if featuresShape = {-1,10} then the features must be rank 2,
can have any size for the first dimension, but must have size 10 for the second dimension.

labelsShape - As per featuresShape, but for the labels instead

Returns:

Results of the validation

deleteInvalidDataSets

Validate DataSet objects - and delete any invalid DataSets - that have been previously saved to the
specified directory on HDFS by attempting to load them and checking their contents. Assumes DataSets were saved
using DataSet.save(OutputStream).
Note: this method will also consider all files in subdirectories (i.e., is recursive).

deleteInvalidDataSets

Validate DataSet objects - and delete any invalid DataSets - that have been previously saved to the
specified directory on HDFS by attempting to load them and checking their contents. Assumes DataSets were saved
using DataSet.save(OutputStream).
This method (optionally) additionally validates the arrays using the specified shapes for the features and labels.
Note: this method will also consider all files in subdirectories (i.e., is recursive).

featuresShape - May be null. If non-null: feature arrays must match the specified shape, for all values with
shape > 0. For example, if featuresShape = {-1,10} then the features must be rank 2,
can have any size for the first dimension, but must have size 10 for the second dimension.

validateMultiDataSets

Validate MultiDataSet objects saved to the specified directory on HDFS by attempting to load them and checking their
contents. Assumes MultiDataSets were saved using MultiDataSet.save(OutputStream).
Note: this method will also consider all files in subdirectories (i.e., is recursive).

validateMultiDataSets

Validate MultiDataSet objects saved to the specified directory on HDFS by attempting to load them and checking their
contents. Assumes MultiDataSets were saved using MultiDataSet.save(OutputStream).
This method additionally validates that the expected number of feature/labels arrays are present in all MultiDataSet
objects
Note: this method will also consider all files in subdirectories (i.e., is recursive).

validateMultiDataSets

Validate MultiDataSet objects saved to the specified directory on HDFS by attempting to load them and checking their
contents. Assumes MultiDataSets were saved using MultiDataSet.save(OutputStream).
This method (optionally) additionally validates the arrays using the specified shapes for the features and labels.
Note: this method will also consider all files in subdirectories (i.e., is recursive).

featuresShape - May be null. If non-null: feature arrays must match the specified shapes, for all values with
shape > 0. For example, if featuresShape = {{-1,10}} then there must be 1 features array,
features array 0 must be rank 2, can have any size for the first dimension, but must have
size 10 for the second dimension.

labelsShape - As per featuresShape, but for the labels instead

Returns:

Results of the validation

deleteInvalidMultiDataSets

Validate MultiDataSet objects - and delete any invalid MultiDataSets - that have been previously saved to the
specified directory on HDFS by attempting to load them and checking their contents. Assumes MultiDataSets were saved
using MultiDataSet.save(OutputStream).
Note: this method will also consider all files in subdirectories (i.e., is recursive).

deleteInvalidMultiDataSets

Validate MultiDataSet objects - and delete any invalid MultiDataSets - that have been previously saved
to the specified directory on HDFS by attempting to load them and checking their contents. Assumes MultiDataSets
were saved using MultiDataSet.save(OutputStream).
This method (optionally) additionally validates the arrays using the specified shapes for the features and labels,
Note: this method will also consider all files in subdirectories (i.e., is recursive).

featuresShape - May be null. If non-null: feature arrays must match the specified shapes, for all values with
shape > 0. For example, if featuresShape = {{-1,10}} then there must be 1 features array,
features array 0 must be rank 2, can have any size for the first dimension, but must have
size 10 for the second dimension.

labelsShape - As per featuresShape, but for the labels instead

Returns:

Results of the validation

deleteInvalidMultiDataSets

Validate MultiDataSet objects - and delete any invalid MultiDataSets - that have been previously saved
to the specified directory on HDFS by attempting to load them and checking their contents. Assumes MultiDataSets
were saved using MultiDataSet.save(OutputStream).
This method (optionally) additionally validates the arrays using the specified shapes for the features and labels.
Note: this method will also consider all files in subdirectories (i.e., is recursive).