For more information on these community-driven changes, be sure to check out the RFCs we have on Github. If you care about details, all of the RFCs are important. Refer to our public project status tracker and issues tagged with 2.0 on GitHub for insight into recent issues and development progress.

And, of course: we would love to have your feedback! If you experience any snags when using TF 2.0, be sure to let us know at the TF 2.0 Testing User Group. We have a support mailing list as well as weekly testing meetings, and would love to hear your migration feedback and questions.

Some highlights:

API clean-up, included removing tf.app, tf.flags, and tf.logging in favor of absl-py.

No more global variables with helper methods like tf.global_variables_initializer and tf.get_global_step.

Functions, not sessions (tf.Session and session.run -> tf.function).

Added support for TensorFlow Lite in TensorFlow 2.0.

Breaking Changes

tf.contrib has been deprecated, and functionality has been either migrated to the core TensorFlow API, to tensorflow/addons, or removed entirely.

Bug Fixes and Other Changes

Use tf.compat.v1.estimator.inputs instead of tf.estimator.inputs in Estimator.

Replace contrib references with tf.estimator.experimental.* for apis in early_stopping.py in Estimator.

Keras & Python API

Added top-k to precision and recall to keras metrics.

Adding public APIs for cumsum and cumprod keras backend functions.

Minor change to SavedModels exported from Keras using tf.keras.experimental.export. (SignatureDef key for evaluation mode is now "eval" instead of "test"). This will be reverted back to "test" in the near future.

Add tf.keras.layers.AbstractRNNCell as the preferred implementation of RNN cell for TF v2. User can use it to implement RNN cell with custom behavior.

Keras training and validation curves are shown on the same plot.

Disable run_eagerly and distribution strategy if there are symbolic tensors added to the model using add_metric or add_loss.

Other:

Only create a GCS directory object if the object does not already exist.

Introduce dynamic constructor argument in Layer and Model, which should be set to True when using imperative control flow in the call method.

ResourceVariable and Variable no longer accepts constraint in the constructor, nor expose it as a @Property.

`ResourceVariab...

Add UnifiedGRU as the new GRU implementation for tf2.0. Change the default recurrent activation function for GRU from 'hard_sigmoid' to 'sigmoid', and 'reset_after' to True in 2.0. Historically recurrent activation is 'hard_sigmoid' since it is fast than 'sigmoid'. With new unified backend between CPU and GPU mode, since the CuDNN kernel is using sigmoid, we change the default for CPU mode to sigmoid as well. With that, the default GRU will be compatible with both CPU and GPU kernel. This will enable user with GPU to use CuDNN kernel by default and get a 10x performance boost in training. Note that this is checkpoint breaking change. If user want to use their 1.x pre-trained checkpoint, please construct the layer with GRU(recurrent_activation='hard_sigmoid', reset_after=False) to fallback to 1.x behavior.

Removing of dtype in the constructor of initializers and partition_info in call.

Add tf.math.nextafter op.

Turn on MKL-DNN contraction kernels by default. MKL-DNN dynamically dispatches the best kernel implementation based on CPU vector architecture. To disable them, build with --define=tensorflow_mkldnn_contraction_kernel=0.

Turn on MKL-DNN contraction kernels by default. MKL-DNN dynamically dispatches the best kernel implementation based on CPU vector architecture. To disable them, build with --define=tensorflow_mkldnn_contraction_kernel=0.

Turn on MKL-DNN contraction kernels by default. MKL-DNN dynamically dispatches the best kernel implementation based on CPU vector architecture. To disable them, build with --define=tensorflow_mkldnn_contraction_kernel=0.

Raw TensorFlow functions can now be used in conjunction with the Keras Functional API during model creation. This obviates the need for users to create Lambda layers in most cases when using the Functional API. Like Lambda layers, TensorFlow functions that result in Variable creation or assign ops are not supported.

Add a ragged size op and register it to the op dispatcher

Transitive dependencies on :pooling_ops were removed. Some users may need to add explicit dependencies on :pooling_ops if they reference the operators from that library.

Updates binary cross entropy logic in Keras when input is probabilities. Instead of converting probabilities to logits, we are using the cross entropy formula for probabilities.

Add CompositeTensor base class.

Malformed gif images could result in an access out of bounds in the color palette of the frame. This has been fixed now

In map_vectorization optimization, reduce the degree of parallelism in the vectorized map node.

Add variant wrapper for absl::string_view.

Post-training quantization tool supports quantizing weights shared by multiple operations. The models made with versions of this tool will use INT8 types for weights and will only be executable interpreters from this version onwards.

Wraps losses passed to the compile API (strings and v1 losses) which are not instances of v2 Loss class in LossWrapper class. => All losses will now use SUM_OVER_BATCH_SIZE reduction as default.

Add OpKernels for some stateless maps

Add v2 APIs for AUCCurve and AUCSummationMethod enums.

Add v2 APIs for AUCCurve and AUCSummationMethod enums.

Allow non-Tensors through v2 losses.

Add v2 sparse categorical crossentropy metric.

DType is no longer convertible to an int. Use dtype.as_datatype_enum instead of int(dtype) to get the same result.

Support both binary and -1/1 label input in v2 hinge and squared hinge losses.

Bug fix: loss and gradients should now more reliably be correctly scaled w.r.t. the global batch size when using a tf.distribute.Strategy.

Added LinearOperator.adjoint and LinearOperator.H (alias).

Switching tf.data functions to use defun, providing an escape hatch to continue using the legacy Defun.

Expose CriticalSection in core as tf.CriticalSection.

Enhanced graphviz output.

The behavior of tf.gather is now correct when axis=None and batch_dims<0.

Add tf.linalg.tridiagonal_solve op.

Add opkernel templates for common table operations.

Fix callbacks do not log values in eager mode when a deferred build model is used.

SignatureDef util functions have been deprecated.

Update Fingerprint64Map to use aliases

Add legacy string flat hash map op kernels

Add support for passing list of lists to the metrics param in Keras `compile.

Keras training and validation curves are shown on the same plot.

Fix: model.add_loss(symbolic_tensor) should work in ambient eager.

Adding clear_losses API to be able to clear losses at the end of forward pass in a custom training loop in eager.

Add support for add_metric in the graph function mode.

Adding clear_losses API to be able to clear losses at the end of forward pass in a custom training loop in eager.

TF 2.0 - Update metric name to always reflect what the user has given in compile. Affects following cases 1. When name is given as 'accuracy'/'crossentropy' 2. When an aliased function name is used eg. 'mse' 3. Removing the weighted prefix from weighted metric names.

Workaround for compiler bug.

Changed default for gradient accumulation for TPU embeddings to true.

Adds summary trace API for collecting graph and profile information.

Support for multi-host ncclAllReduce in Distribution Strategy.

Enable tf.distribute.experimental.MultiWorkerMirroredStrategy working in eager mode.

image.resize now considers proper pixel centers and has new kernels (incl. anti-aliasing).

Thanks to our Contributors

This release contains contributions from many people at Google, as well as:

Release 1.13.1

Major Features and Improvements

TensorFlow Lite has moved from contrib to core. This means that Python modules are under tf.lite and source code is now under tensorflow/lite rather than tensorflow/contrib/lite.

TensorFlow GPU binaries are now built against CUDA 10 and TensorRT 5.0.

Support for Python3.7 on all operating systems.

Moved NCCL to core.

Behavioral changes

Disallow conversion of python floating types to uint32/64 (matching behavior of other integer types) in tf.constant.

Make the gain argument of convolutional orthogonal initializers (convolutional_delta_orthogonal, convolutional_orthogonal_1D, convolutional_orthogonal_2D, convolutional_orthogonal_3D) have consistent behavior with the tf.initializers.orthogonal initializer, i.e. scale the output l2-norm by gain and NOT by sqrt(gain). (Note that these functions are currently in tf.contrib which is not guaranteed backward compatible).

Bug Fixes and Other Changes

Documentation

Update the doc with the details about the rounding mode used in quantize_and_dequantize_v2.

Clarify that tensorflow::port::InitMain() should be called before using the TensorFlow library. Programs failing to do this are not portable to all platforms.

Add tf.dtypes. endpoint for every constant in dtypes.py; moving endpoints in versions.py to corresponding endpoints in tf.sysconfig. and tf.version.; moving all constants under tf.saved_model submodules to tf.saved_model module. New endpoints are added in V1 and V2 but existing endpoint removals are only applied in V2.

Add "unit" attribute to the substr op, which allows obtaining the substring of a string containing unicode characters.

Broadcasting support for Ragged Tensors.

SpaceToDepth supports uint8 data type.

Support multi-label quantile regression in estimator.

We now use "div" as the default partition_strategy in tf.nn.safe_embedding_lookup_sparse, tf.nn.sampled_softmax and tf.nn.nce_loss. hyperparameter are ignored.

Performance

Improve performance of GPU cumsum/cumprod by up to 300x.

Added support for weight decay in most TPU embedding optimizers, including AdamW and MomentumW.

TensorFlow 2.0 Development

Add a command line tool to convert to TF2.0, tf_upgrade_v2

Merge tf.spectral into tf.signal for TensorFlow 2.0.

Change the default recurrent activation function for LSTM from 'hard_sigmoid' to 'sigmoid' in 2.0. Historically recurrent activation is 'hard_sigmoid' since it is fast than 'sigmoid'. With new unified backend between CPU and GPU mode, since the CuDNN kernel is using sigmoid, we change the default for CPU mode to sigmoid as well. With that, the default LSTM will be compatible with both CPU and GPU kernel. This will enable user with GPU to use CuDNN kernel by default and get a 10x performance boost in training. Note that this is checkpoint breaking change. If user want to use their 1.x pre-trained checkpoint, please construct the layer with LSTM(recurrent_activation='hard_sigmoid') to fallback to 1.x behavior.

Enable map_and_batch_fusion and noop_elimination optimizations by default. They can be disabled by configuring tf.data.experimental.OptimizationOptions to set map_and_batch = False or noop_elimination = False respectively. To disable all default optimizations, set apply_default_optimizations = False.

Support parallel map in map_and_filter_fusion.

Disable static optimizations for input pipelines that use non-resource tf.Variables.

Add NUMA-aware MapAndBatch dataset.

Deprecate tf.data.Dataset.make_one_shot_iterator() in V1, removed it from V2, and added tf.compat.v1.data.make_one_shot_iterator()`.

Deprecate tf.data.Dataset.make_initializable_iterator() in V1, removed it from V2, and added tf.compat.v1.data.make_initializable_iterator().

A new environment variable TF_XLA_DEBUG_OPTIONS_PASSTHROUGH set to "1" or "true" allows the debug options passed within an XRTCompile op to be passed directly to the XLA compilation backend. If such variable is not set (service side), only a restricted set will be passed through.

Allow the XRTCompile op to return the ProgramShape resulted form the XLA compilation as a second return argument.

XLA HLO graphs can now be rendered as SVG/HTML.

Estimator

Replace all occurences of tf.contrib.estimator.BaselineEstimator with tf.estimator.BaselineEstimator

Replace all occurences of tf.contrib.estimator.DNNLinearCombinedEstimator with tf.estimator.DNNLinearCombinedEstimator

Replace all occurrences of tf.contrib.estimator.DNNEstimator with tf.estimator.DNNEstimator

Replace all occurrences of tf.contrib.estimator.LinearEstimator with tf.estimator.LinearEstimator

Users of tf.contrib.estimator.export_all_saved_models and related should switch to tf.estimator.Estimator.experimental_export_all_saved_models.

Update regression_head to the new Head API for Canned Estimator V2.

Switch multi_class_head to Head API for Canned Estimator V2.

Replace all occurences of tf.contrib.estimator.InMemoryEvaluatorHook and tf.contrib.estimator.make_stop_at_checkpoint_step_hook with tf.estimator.experimental.InMemoryEvaluatorHook and tf.estimator.experimental.make_stop_at_checkpoint_step_hook

Migrate linear optimizer from contrib to core.

Thanks to our Contributors

This release contains contributions from many people at Google, as well as:

Release 1.13.0 RC2

Major Features and Improvements

TensorFlow Lite has moved from contrib to core. This means that Python modules are under tf.lite and source code is now under tensorflow/lite rather than tensorflow/contrib/lite.

TensorFlow GPU binaries are now built against CUDA 10 and TensorRT 5.0.

Support for Python3.7 on all operating systems.

Moved NCCL to core.

Behavioral changes

Disallow conversion of python floating types to uint32/64 (matching behavior of other integer types) in tf.constant.

Make the gain argument of convolutional orthogonal initializers (convolutional_delta_orthogonal, convolutional_orthogonal_1D, convolutional_orthogonal_2D, convolutional_orthogonal_3D) have consistent behavior with the tf.initializers.orthogonal initializer, i.e. scale the output l2-norm by gain and NOT by sqrt(gain). (Note that these functions are currently in tf.contrib which is not guaranteed backward compatible).

Bug Fixes and Other Changes

Documentation

Update the doc with the details about the rounding mode used in quantize_and_dequantize_v2.

Clarify that tensorflow::port::InitMain() should be called before using the TensorFlow library. Programs failing to do this are not portable to all platforms.

Add tf.dtypes. endpoint for every constant in dtypes.py; moving endpoints in versions.py to corresponding endpoints in tf.sysconfig. and tf.version.; moving all constants under tf.saved_model submodules to tf.saved_model module. New endpoints are added in V1 and V2 but existing endpoint removals are only applied in V2.

Add "unit" attribute to the substr op, which allows obtaining the substring of a string containing unicode characters.

Broadcasting support for Ragged Tensors.

SpaceToDepth supports uint8 data type.

Support multi-label quantile regression in estimator.

We now use "div" as the default partition_strategy in tf.nn.safe_embedding_lookup_sparse, tf.nn.sampled_softmax and tf.nn.nce_loss. hyperparameter are ignored.

Performance

Improve performance of GPU cumsum/cumprod by up to 300x.

Added support for weight decay in most TPU embedding optimizers, including AdamW and MomentumW.

TensorFlow 2.0 Development

Add a command line tool to convert to TF2.0, tf_upgrade_v2

Merge tf.spectral into tf.signal for TensorFlow 2.0.

Change the default recurrent activation function for LSTM from 'hard_sigmoid' to 'sigmoid' in 2.0. Historically recurrent activation is 'hard_sigmoid' since it is fast than 'sigmoid'. With new unified backend between CPU and GPU mode, since the CuDNN kernel is using sigmoid, we change the default for CPU mode to sigmoid as well. With that, the default LSTM will be compatible with both CPU and GPU kernel. This will enable user with GPU to use CuDNN kernel by default and get a 10x performance boost in training. Note that this is checkpoint breaking change. If user want to use their 1.x pre-trained checkpoint, please construct the layer with LSTM(recurrent_activation='hard_sigmoid') to fallback to 1.x behavior.

Enable map_and_batch_fusion and noop_elimination optimizations by default. They can be disabled by configuring tf.data.experimental.OptimizationOptions to set map_and_batch = False or noop_elimination = False respectively. To disable all default optimizations, set apply_default_optimizations = False.

Support parallel map in map_and_filter_fusion.

Disable static optimizations for input pipelines that use non-resource tf.Variables.

Add NUMA-aware MapAndBatch dataset.

Deprecate tf.data.Dataset.make_one_shot_iterator() in V1, removed it from V2, and added tf.compat.v1.data.make_one_shot_iterator()`.

Deprecate tf.data.Dataset.make_initializable_iterator() in V1, removed it from V2, and added tf.compat.v1.data.make_initializable_iterator().

A new environment variable TF_XLA_DEBUG_OPTIONS_PASSTHROUGH set to "1" or "true" allows the debug options passed within an XRTCompile op to be passed directly to the XLA compilation backend. If such variable is not set (service side), only a restricted set will be passed through.

Allow the XRTCompile op to return the ProgramShape resulted form the XLA compilation as a second return argument.

XLA HLO graphs can now be rendered as SVG/HTML.

Estimator

Replace all occurences of tf.contrib.estimator.BaselineEstimator with tf.estimator.BaselineEstimator

Replace all occurences of tf.contrib.estimator.DNNLinearCombinedEstimator with tf.estimator.DNNLinearCombinedEstimator

Replace all occurrences of tf.contrib.estimator.DNNEstimator with tf.estimator.DNNEstimator

Replace all occurrences of tf.contrib.estimator.LinearEstimator with tf.estimator.LinearEstimator

Users of tf.contrib.estimator.export_all_saved_models and related should switch to tf.estimator.Estimator.experimental_export_all_saved_models.

Update regression_head to the new Head API for Canned Estimator V2.

Switch multi_class_head to Head API for Canned Estimator V2.

Replace all occurences of tf.contrib.estimator.InMemoryEvaluatorHook and tf.contrib.estimator.make_stop_at_checkpoint_step_hook with tf.estimator.experimental.InMemoryEvaluatorHook and tf.estimator.experimental.make_stop_at_checkpoint_step_hook

Migrate linear optimizer from contrib to core.

Thanks to our Contributors

This release contains contributions from many people at Google, as well as:

gunan released this
Feb 23, 2019
· 1 commit to master since this release

Release 1.13.0

Major Features and Improvements

TensorFlow Lite has moved from contrib to core. This means that Python modules are under tf.lite and source code is now under tensorflow/lite rather than tensorflow/contrib/lite.

TensorFlow GPU binaries are now built against CUDA 10 and TensorRT 5.0.

Support for Python3.7 on all operating systems.

Moved NCCL to core.

Behavioral changes

Disallow conversion of python floating types to uint32/64 (matching behavior of other integer types) in tf.constant.

Make the gain argument of convolutional orthogonal initializers (convolutional_delta_orthogonal, convolutional_orthogonal_1D, convolutional_orthogonal_2D, convolutional_orthogonal_3D) have consistent behavior with the tf.initializers.orthogonal initializer, i.e. scale the output l2-norm by gain and NOT by sqrt(gain). (Note that these functions are currently in tf.contrib which is not guaranteed backward compatible).

Bug Fixes and Other Changes

Documentation

Update the doc with the details about the rounding mode used in quantize_and_dequantize_v2.

Clarify that tensorflow::port::InitMain() should be called before using the TensorFlow library. Programs failing to do this are not portable to all platforms.

Add tf.dtypes. endpoint for every constant in dtypes.py; moving endpoints in versions.py to corresponding endpoints in tf.sysconfig. and tf.version.; moving all constants under tf.saved_model submodules to tf.saved_model module. New endpoints are added in V1 and V2 but existing endpoint removals are only applied in V2.

Add "unit" attribute to the substr op, which allows obtaining the substring of a string containing unicode characters.

Broadcasting support for Ragged Tensors.

SpaceToDepth supports uint8 data type.

Support multi-label quantile regression in estimator.

We now use "div" as the default partition_strategy in tf.nn.safe_embedding_lookup_sparse, tf.nn.sampled_softmax and tf.nn.nce_loss. hyperparameter are ignored.

Performance

Improve performance of GPU cumsum/cumprod by up to 300x.

Added support for weight decay in most TPU embedding optimizers, including AdamW and MomentumW.

TensorFlow 2.0 Development

Add a command line tool to convert to TF2.0, tf_upgrade_v2

Merge tf.spectral into tf.signal for TensorFlow 2.0.

Change the default recurrent activation function for LSTM from 'hard_sigmoid' to 'sigmoid' in 2.0. Historically recurrent activation is 'hard_sigmoid' since it is fast than 'sigmoid'. With new unified backend between CPU and GPU mode, since the CuDNN kernel is using sigmoid, we change the default for CPU mode to sigmoid as well. With that, the default LSTM will be compatible with both CPU and GPU kernel. This will enable user with GPU to use CuDNN kernel by default and get a 10x performance boost in training. Note that this is checkpoint breaking change. If user want to use their 1.x pre-trained checkpoint, please construct the layer with LSTM(recurrent_activation='hard_sigmoid') to fallback to 1.x behavior.

Enable map_and_batch_fusion and noop_elimination optimizations by default. They can be disabled by configuring tf.data.experimental.OptimizationOptions to set map_and_batch = False or noop_elimination = False respectively. To disable all default optimizations, set apply_default_optimizations = False.

Support parallel map in map_and_filter_fusion.

Disable static optimizations for input pipelines that use non-resource tf.Variables.

Add NUMA-aware MapAndBatch dataset.

Deprecate tf.data.Dataset.make_one_shot_iterator() in V1, removed it from V2, and added tf.compat.v1.data.make_one_shot_iterator()`.

Deprecate tf.data.Dataset.make_initializable_iterator() in V1, removed it from V2, and added tf.compat.v1.data.make_initializable_iterator().

A new environment variable TF_XLA_DEBUG_OPTIONS_PASSTHROUGH set to "1" or "true" allows the debug options passed within an XRTCompile op to be passed directly to the XLA compilation backend. If such variable is not set (service side), only a restricted set will be passed through.

Allow the XRTCompile op to return the ProgramShape resulted form the XLA compilation as a second return argument.

XLA HLO graphs can now be rendered as SVG/HTML.

Estimator

Replace all occurences of tf.contrib.estimator.BaselineEstimator with tf.estimator.BaselineEstimator

Replace all occurences of tf.contrib.estimator.DNNLinearCombinedEstimator with tf.estimator.DNNLinearCombinedEstimator

Replace all occurrences of tf.contrib.estimator.DNNEstimator with tf.estimator.DNNEstimator

Replace all occurrences of tf.contrib.estimator.LinearEstimator with tf.estimator.LinearEstimator

Users of tf.contrib.estimator.export_all_saved_models and related should switch to tf.estimator.Estimator.experimental_export_all_saved_models.

Update regression_head to the new Head API for Canned Estimator V2.

Switch multi_class_head to Head API for Canned Estimator V2.

Replace all occurences of tf.contrib.estimator.InMemoryEvaluatorHook and tf.contrib.estimator.make_stop_at_checkpoint_step_hook with tf.estimator.experimental.InMemoryEvaluatorHook and tf.estimator.experimental.make_stop_at_checkpoint_step_hook

Migrate linear optimizer from contrib to core.

Thanks to our Contributors

This release contains contributions from many people at Google, as well as:

Release 1.13.0 RC2

Major Features and Improvements

TensorFlow Lite has moved from contrib to core. This means that Python modules are under tf.lite and source code is now under tensorflow/lite rather than tensorflow/contrib/lite.

TensorFlow GPU binaries are now built against CUDA 10 and TensorRT 5.0.

Support for Python3.7 on all operating systems.

Moved NCCL to core.

Behavioral changes

Disallow conversion of python floating types to uint32/64 (matching behavior of other integer types) in tf.constant.

Make the gain argument of convolutional orthogonal initializers (convolutional_delta_orthogonal, convolutional_orthogonal_1D, convolutional_orthogonal_2D, convolutional_orthogonal_3D) have consistent behavior with the tf.initializers.orthogonal initializer, i.e. scale the output l2-norm by gain and NOT by sqrt(gain). (Note that these functions are currently in tf.contrib which is not guaranteed backward compatible).

Bug Fixes and Other Changes

Documentation

Update the doc with the details about the rounding mode used in quantize_and_dequantize_v2.

Clarify that tensorflow::port::InitMain() should be called before using the TensorFlow library. Programs failing to do this are not portable to all platforms.

Add tf.dtypes. endpoint for every constant in dtypes.py; moving endpoints in versions.py to corresponding endpoints in tf.sysconfig. and tf.version.; moving all constants under tf.saved_model submodules to tf.saved_model module. New endpoints are added in V1 and V2 but existing endpoint removals are only applied in V2.

Add "unit" attribute to the substr op, which allows obtaining the substring of a string containing unicode characters.

Broadcasting support for Ragged Tensors.

SpaceToDepth supports uint8 data type.

Support multi-label quantile regression in estimator.

We now use "div" as the default partition_strategy in tf.nn.safe_embedding_lookup_sparse, tf.nn.sampled_softmax and tf.nn.nce_loss. hyperparameter are ignored.

Performance

Improve performance of GPU cumsum/cumprod by up to 300x.

Added support for weight decay in most TPU embedding optimizers, including AdamW and MomentumW.

TensorFlow 2.0 Development

Add a command line tool to convert to TF2.0, tf_upgrade_v2

Merge tf.spectral into tf.signal for TensorFlow 2.0.

Change the default recurrent activation function for LSTM from 'hard_sigmoid' to 'sigmoid' in 2.0. Historically recurrent activation is 'hard_sigmoid' since it is fast than 'sigmoid'. With new unified backend between CPU and GPU mode, since the CuDNN kernel is using sigmoid, we change the default for CPU mode to sigmoid as well. With that, the default LSTM will be compatible with both CPU and GPU kernel. This will enable user with GPU to use CuDNN kernel by default and get a 10x performance boost in training. Note that this is checkpoint breaking change. If user want to use their 1.x pre-trained checkpoint, please construct the layer with LSTM(recurrent_activation='hard_sigmoid') to fallback to 1.x behavior.

Enable map_and_batch_fusion and noop_elimination optimizations by default. They can be disabled by configuring tf.data.experimental.OptimizationOptions to set map_and_batch = False or noop_elimination = False respectively. To disable all default optimizations, set apply_default_optimizations = False.

Support parallel map in map_and_filter_fusion.

Disable static optimizations for input pipelines that use non-resource tf.Variables.

Add NUMA-aware MapAndBatch dataset.

Deprecate tf.data.Dataset.make_one_shot_iterator() in V1, removed it from V2, and added tf.compat.v1.data.make_one_shot_iterator()`.

Deprecate tf.data.Dataset.make_initializable_iterator() in V1, removed it from V2, and added tf.compat.v1.data.make_initializable_iterator().

A new environment variable TF_XLA_DEBUG_OPTIONS_PASSTHROUGH set to "1" or "true" allows the debug options passed within an XRTCompile op to be passed directly to the XLA compilation backend. If such variable is not set (service side), only a restricted set will be passed through.

Allow the XRTCompile op to return the ProgramShape resulted form the XLA compilation as a second return argument.

XLA HLO graphs can now be rendered as SVG/HTML.

Estimator

Replace all occurences of tf.contrib.estimator.BaselineEstimator with tf.estimator.BaselineEstimator

Replace all occurences of tf.contrib.estimator.DNNLinearCombinedEstimator with tf.estimator.DNNLinearCombinedEstimator

Replace all occurrences of tf.contrib.estimator.DNNEstimator with tf.estimator.DNNEstimator

Replace all occurrences of tf.contrib.estimator.LinearEstimator with tf.estimator.LinearEstimator

Users of tf.contrib.estimator.export_all_saved_models and related should switch to tf.estimator.Estimator.experimental_export_all_saved_models.

Update regression_head to the new Head API for Canned Estimator V2.

Switch multi_class_head to Head API for Canned Estimator V2.

Replace all occurences of tf.contrib.estimator.InMemoryEvaluatorHook and tf.contrib.estimator.make_stop_at_checkpoint_step_hook with tf.estimator.experimental.InMemoryEvaluatorHook and tf.estimator.experimental.make_stop_at_checkpoint_step_hook

Migrate linear optimizer from contrib to core.

Thanks to our Contributors

This release contains contributions from many people at Google, as well as:

Release 1.13.0

Major Features and Improvements

TensorFlow Lite has moved from contrib to core. This means that Python modules are under tf.lite and source code is now under tensorflow/lite rather than tensorflow/contrib/lite.

TensorFlow GPU binaries are now built against CUDA 10 and TensorRT 5.0.

Moved NCCL to core.

Behavioral changes

Disallow conversion of python floating types to uint32/64 (matching behavior of other integer types) in tf.constant.

Make the gain argument of convolutional orthogonal initializers (convolutional_delta_orthogonal, convolutional_orthogonal_1D, convolutional_orthogonal_2D, convolutional_orthogonal_3D) have consistent behavior with the tf.initializers.orthogonal initializer, i.e. scale the output l2-norm by gain and NOT by sqrt(gain). (Note that these functions are currently in tf.contrib which is not guaranteed backward compatible).

Bug Fixes and Other Changes

Documentation

Update the doc with the details about the rounding mode used in quantize_and_dequantize_v2.

Clarify that tensorflow::port::InitMain() should be called before using the TensorFlow library. Programs failing to do this are not portable to all platforms.

Add tf.dtypes. endpoint for every constant in dtypes.py; moving endpoints in versions.py to corresponding endpoints in tf.sysconfig. and tf.version.; moving all constants under tf.saved_model submodules to tf.saved_model module. New endpoints are added in V1 and V2 but existing endpoint removals are only applied in V2.

Add "unit" attribute to the substr op, which allows obtaining the substring of a string containing unicode characters.

Broadcasting support for Ragged Tensors.

SpaceToDepth supports uint8 data type.

Support multi-label quantile regression in estimator.

We now use "div" as the default partition_strategy in tf.nn.safe_embedding_lookup_sparse, tf.nn.sampled_softmax and tf.nn.nce_loss. hyperparameter are ignored.

Performance

Improve performance of GPU cumsum/cumprod by up to 300x.

Added support for weight decay in most TPU embedding optimizers, including AdamW and MomentumW.

TensorFlow 2.0 Development

Add a command line tool to convert to TF2.0, tf_upgrade_v2

Merge tf.spectral into tf.signal for TensorFlow 2.0.

Change the default recurrent activation function for LSTM from 'hard_sigmoid' to 'sigmoid' in 2.0. Historically recurrent activation is 'hard_sigmoid' since it is fast than 'sigmoid'. With new unified backend between CPU and GPU mode, since the CuDNN kernel is using sigmoid, we change the default for CPU mode to sigmoid as well. With that, the default LSTM will be compatible with both CPU and GPU kernel. This will enable user with GPU to use CuDNN kernel by default and get a 10x performance boost in training. Note that this is checkpoint breaking change. If user want to use their 1.x pre-trained checkpoint, please construct the layer with LSTM(recurrent_activation='hard_sigmoid') to fallback to 1.x behavior.

Enable map_and_batch_fusion and noop_elimination optimizations by default. They can be disabled by configuring tf.data.experimental.OptimizationOptions to set map_and_batch = False or noop_elimination = False respectively. To disable all default optimizations, set apply_default_optimizations = False.

Support parallel map in map_and_filter_fusion.

Disable static optimizations for input pipelines that use non-resource tf.Variables.

Add NUMA-aware MapAndBatch dataset.

Deprecate tf.data.Dataset.make_one_shot_iterator() in V1, removed it from V2, and added tf.compat.v1.data.make_one_shot_iterator()`.

Deprecate tf.data.Dataset.make_initializable_iterator() in V1, removed it from V2, and added tf.compat.v1.data.make_initializable_iterator().

A new environment variable TF_XLA_DEBUG_OPTIONS_PASSTHROUGH set to "1" or "true" allows the debug options passed within an XRTCompile op to be passed directly to the XLA compilation backend. If such variable is not set (service side), only a restricted set will be passed through.

Allow the XRTCompile op to return the ProgramShape resulted form the XLA compilation as a second return argument.

XLA HLO graphs can now be rendered as SVG/HTML.

Estimator

Replace all occurences of tf.contrib.estimator.BaselineEstimator with tf.estimator.BaselineEstimator

Replace all occurences of tf.contrib.estimator.DNNLinearCombinedEstimator with tf.estimator.DNNLinearCombinedEstimator

Replace all occurrences of tf.contrib.estimator.DNNEstimator with tf.estimator.DNNEstimator

Replace all occurrences of tf.contrib.estimator.LinearEstimator with tf.estimator.LinearEstimator

Users of tf.contrib.estimator.export_all_saved_models and related should switch to tf.estimator.Estimator.experimental_export_all_saved_models.

Update regression_head to the new Head API for Canned Estimator V2.

Switch multi_class_head to Head API for Canned Estimator V2.

Replace all occurences of tf.contrib.estimator.InMemoryEvaluatorHook and tf.contrib.estimator.make_stop_at_checkpoint_step_hook with tf.estimator.experimental.InMemoryEvaluatorHook and tf.estimator.experimental.make_stop_at_checkpoint_step_hook

Migrate linear optimizer from contrib to core.

Thanks to our Contributors

This release contains contributions from many people at Google, as well as:

Release 1.13.0

Major Features and Improvements

TensorFlow Lite has moved from contrib to core. This means that Python modules are under tf.lite and source code is now under tensorflow/lite rather than tensorflow/contrib/lite.

TensorFlow GPU binaries are now built against CUDA 10.

Moved NCCL to core.

Behavioral changes

Disallow conversion of python floating types to uint32/64 (matching behavior of other integer types) in tf.constant.

Make the gain argument of convolutional orthogonal initializers (convolutional_delta_orthogonal, convolutional_orthogonal_1D, convolutional_orthogonal_2D, convolutional_orthogonal_3D) have consistent behavior with the tf.initializers.orthogonal initializer, i.e. scale the output l2-norm by gain and NOT by sqrt(gain). (Note that these functions are currently in tf.contrib which is not guaranteed backward compatible).

Bug Fixes and Other Changes

Documentation

Update the doc with the details about the rounding mode used in quantize_and_dequantize_v2.

Clarify that tensorflow::port::InitMain() should be called before using the TensorFlow library. Programs failing to do this are not portable to all platforms.

Add tf.dtypes. endpoint for every constant in dtypes.py; moving endpoints in versions.py to corresponding endpoints in tf.sysconfig. and tf.version.; moving all constants under tf.saved_model submodules to tf.saved_model module. New endpoints are added in V1 and V2 but existing endpoint removals are only applied in V2.

Add "unit" attribute to the substr op, which allows obtaining the substring of a string containing unicode characters.

Broadcasting support for Ragged Tensors.

SpaceToDepth supports uint8 data type.

Support multi-label quantile regression in estimator.

We now use "div" as the default partition_strategy in tf.nn.safe_embedding_lookup_sparse, tf.nn.sampled_softmax and tf.nn.nce_loss. hyperparameter are ignored.

Performance

Improve performance of GPU cumsum/cumprod by up to 300x.

Added support for weight decay in most TPU embedding optimizers, including AdamW and MomentumW.

TensorFlow 2.0 Development

Add a command line tool to convert to TF2.0, tf_upgrade_v2

Merge tf.spectral into tf.signal for TensorFlow 2.0.

Change the default recurrent activation function for LSTM from 'hard_sigmoid' to 'sigmoid' in 2.0. Historically recurrent activation is 'hard_sigmoid' since it is fast than 'sigmoid'. With new unified backend between CPU and GPU mode, since the CuDNN kernel is using sigmoid, we change the default for CPU mode to sigmoid as well. With that, the default LSTM will be compatible with both CPU and GPU kernel. This will enable user with GPU to use CuDNN kernel by default and get a 10x performance boost in training. Note that this is checkpoint breaking change. If user want to use their 1.x pre-trained checkpoint, please construct the layer with LSTM(recurrent_activation='hard_sigmoid') to fallback to 1.x behavior.

A new environment variable TF_XLA_DEBUG_OPTIONS_PASSTHROUGH set to "1" or "true" allows the debug options passed within an XRTCompile op to be passed directly to the XLA compilation backend. If such variable is not set (service side), only a restricted set will be passed through.

Allow the XRTCompile op to return the ProgramShape resulted form the XLA compilation as a second return argument.

XLA HLO graphs can now be rendered as SVG/HTML.

Thanks to our Contributors

This release contains contributions from many people at Google, as well as:

Release 1.12.0

Major Features and Improvements

Keras models can now be directly exported to the SavedModel format(tf.contrib.saved_model.save_keras_model()) and used with Tensorflow Serving.

Keras models now support evaluating with a tf.data.Dataset.

TensorFlow binaries are built with XLA support linked in by default.

Ignite Dataset added to contrib/ignite that allows to work with Apache Ignite.

Bug Fixes and Other Changes

tf.data:

tf.data users can now represent, get, and set options of TensorFlow input pipelines using tf.data.Options(), tf.data.Dataset.options(), and tf.data.Dataset.with_options() respectively.

New tf.data.Dataset.reduce() API allows users to reduce a finite dataset to a single element using a user-provided reduce function.

New tf.data.Dataset.window() API allows users to create finite windows of input dataset; when combined with the tf.data.Dataset.reduce() API, this allows users to implement customized batching.

All C++ code moves to the tensorflow::data namespace.

Add support for num_parallel_calls to tf.data.Dataset.interleave.

tf.contrib:

Remove tf.contrib.linalg. tf.linalg should be used instead.

Replace any calls to tf.contrib.get_signature_def_by_key(metagraph_def, signature_def_key) with meta_graph_def.signature_def[signature_def_key]. Catching a ValueError exception thrown by tf.contrib.get_signature_def_by_key should be replaced by catching a KeyError exception.

tf.contrib.data

Deprecate, and replace by tf.data.experimental.

Other:

Improved XLA stability and performance.

Fix single replica TensorBoard summary stats in Cloud ML Engine.

TPUEstimator: Initialize dataset iterators in parallel.

Keras on TPU model quality and bug fixes.

Instead of jemalloc, revert back to using system malloc since it simplifies build and has comparable performance.

Remove integer types from tf.nn.softplus and tf.nn.softsign OpDefs. This is a bugfix; these ops were never meant to support integers.

Allow subslicing Tensors with a single dimension.

Add option to calculate string length in Unicode characters

Add functionality to SubSlice a tensor.

Add searchsorted (ie lower/upper_bound) op.

Add model explainability to Boosted Trees.

Support negative positions for tf.substr

There was previously a bug in the bijector_impl where the _reduce_jacobian_det_over_event does not handle scalar ILDJ implementations properly.

In tf eager execution, allow re-entering a GradientTape context

Add tf_api_version flag. If --define=tf_api_version=2 flag is passed in, then bazel will build TensorFlow API version 2.0. Note that TensorFlow 2.0 is under active development and has no guarantees at this point.

Add additional compression options to TfRecordWriter

Performance improvements for regex full match operations.

Replace tf.GraphKeys.VARIABLES with tf.GraphKeys.GLOBAL_VARIABLES

Remove unused dynamic learning rate support.

Thanks to our Contributors

This release contains contributions from many people at Google, as well as:

Differences from 1.12.0-rc0

Major Features and Improvements

Keras models can now be directly exported to the SavedModel format(tf.contrib.saved_model.save_keras_model()) and used with Tensorflow Serving.

Keras models now support evaluating with a tf.data.Dataset.

TensorFlow binaries are built with XLA support linked in by default.

Ignite Dataset added to contrib/ignite that allows to work with Apache Ignite.

Bug Fixes and Other Changes

tf.data:

tf.data users can now represent, get, and set options of TensorFlow input pipelines using tf.data.Options(), tf.data.Dataset.options(), and tf.data.Dataset.with_options() respectively.

New tf.data.Dataset.reduce() API allows users to reduce a finite dataset to a single element using a user-provided reduce function.

New tf.data.Dataset.window() API allows users to create finite windows of input dataset; when combined with the tf.data.Dataset.reduce() API, this allows users to implement customized batching.

All C++ code moves to the tensorflow::data namespace.

Add support for num_parallel_calls to tf.data.Dataset.interleave.

tf.contrib:

Remove tf.contrib.linalg. tf.linalg should be used instead.

Replace any calls to tf.contrib.get_signature_def_by_key(metagraph_def, signature_def_key) with meta_graph_def.signature_def[signature_def_key]. Catching a ValueError exception thrown by tf.contrib.get_signature_def_by_key should be replaced by catching a KeyError exception.

tf.contrib.data

Deprecate, and replace by tf.data.experimental.

Other:

Instead of jemalloc, revert back to using system malloc since it simplifies build and has comparable performance.

Remove integer types from tf.nn.softplus and tf.nn.softsign OpDefs. This is a bugfix; these ops were never meant to support integers.

Allow subslicing Tensors with a single dimension.

Add option to calculate string length in Unicode characters

Add functionality to SubSlice a tensor.

Add searchsorted (ie lower/upper_bound) op.

Add model explainability to Boosted Trees.

Support negative positions for tf.substr

There was previously a bug in the bijector_impl where the _reduce_jacobian_det_over_event does not handle scalar ILDJ implementations properly.

In tf eager execution, allow re-entering a GradientTape context

Add tf_api_version flag. If --define=tf_api_version=2 flag is passed in, then bazel will build TensorFlow API version 2.0. Note that TensorFlow 2.0 is under active development and has no guarantees at this point.

Add additional compression options to TfRecordWriter

Performance improvements for regex full match operations.

Replace tf.GraphKeys.VARIABLES with tf.GraphKeys.GLOBAL_VARIABLES

Remove unused dynamic learning rate support.

Thanks to our Contributors

This release contains contributions from many people at Google, as well as:

Assets

Release 1.10.0

Major Features And Improvements

The tf.lite runtime now supports complex64.

Initial Bigtable integration for tf.data.

Improved local run behavior in tf.estimator.train_and_evaluate which does not reload checkpoints for evaluation.

RunConfig now sets device_filters to restrict how workers and PS can communicate. This can speed up training and ensure clean shutdowns in some situations. But if you have jobs that require communication between workers, you will have to set custom session_options in your RunConfig.

Moved Distributions and Bijectors from tf.contrib.distributions to Tensorflow Probability (TFP). tf.contrib.distributions is now deprecated and will be removed by the end of 2018.

Estimators now use custom savers included in EstimatorSpec scaffolds for saving SavedModels during export.

EstimatorSpec will now add a default prediction output for export if no export_output is provided, eliminating the need to explicitly include a PredictOutput object in the model_fn for simple use-cases.

Adding new endpoints for existing tensorflow symbols. These endpoints are going to be the preferred endpoints going forward and may replace some of the existing endpoints in the future. List of new endpoints:

Assets

Release 1.10.0

Major Features And Improvements

The tf.lite runtime now supports complex64.

Initial Bigtable integration for tf.data.

Improved local run behavior in tf.estimator.train_and_evaluate which does not reload checkpoints for evaluation.

RunConfig now sets device_filters to restrict how workers and PS can communicate. This can speed up training and ensure clean shutdowns in some situations. But if you have jobs that require communication between workers, you will have to set custom session_options in your RunConfig.

Moved Distributions and Bijectors from tf.contrib.distributions to Tensorflow Probability (TFP). tf.contrib.distributions is now deprecated and will be removed by the end of 2018.

Estimators now use custom savers included in EstimatorSpec scaffolds for saving SavedModels during export.

EstimatorSpec will now add a default prediction output for export if no export_output is provided, eliminating the need to explicitly include a PredictOutput object in the model_fn for simple use-cases.

Adding new endpoints for existing tensorflow symbols. These endpoints are going to be the preferred endpoints going forward and may replace some of the existing endpoints in the future. List of new endpoints:

Assets

Release 1.10.0

Major Features And Improvements

The tf.lite runtime now supports complex64.

Initial Bigtable integration for tf.data.

Improved local run behavior in tf.estimator.train_and_evaluate which does not reload checkpoints for evaluation.

RunConfig now sets device_filters to restrict how workers and PS can communicate. This can speed up training and ensure clean shutdowns in some situations. But if you have jobs that require communication between workers, you will have to set custom session_options in your RunConfig.

Moved Distributions and Bijectors from tf.contrib.distributions to Tensorflow Probability (TFP). tf.contrib.distributions is now deprecated and will be removed by the end of 2018.

Estimators now use custom savers included in EstimatorSpec scaffolds for saving SavedModels during export.

EstimatorSpec will now add a default prediction output for export if no export_output is provided, eliminating the need to explicitly include a PredictOutput object in the model_fn for simple use-cases.

Adding new endpoints for existing tensorflow symbols. These endpoints are going to be the preferred endpoints going forward and may replace some of the existing endpoints in the future. List of new endpoints:

Update benchmark for tf.scan to match ranges across eager and graph modes.

Fixed bug in tf.reduce_prod gradient for complex dtypes.

Allow the use of '.' in variables (e.g. "hparams.parse('a.b=1.0')"), which would previously raise an error. This will correspond to an attribute name with an embedded '.' symbol (e.g. 'a.b'), which can only be accessed indirectly (e.g. through getattr and setattr). To set this up the user will first need to explicitly add the variable to the hparam object (e.g. "hparams.add_hparam(name='a.b', value=0.0)").

Added LinearOperatorKronecker, a dense-free implementation of the Kronecker Product.

Allow LinearOperator to broadcast.

SavedModelBuilder will now deduplicate asset names that point to files with the same basename and the same contents. Note that this may result in new asset files included in SavedModels in cases where assets with the same name but different contents were previously overwriting each other.

Thanks to our Contributors

This release contains contributions from many people at Google, as well as:

Update benchmark for tf.scan to match ranges across eager and graph modes.

Fixed bug in tf.reduce_prod gradient for complex dtypes.

Add optional args argument to Dataset.from_generator().

Allow the use of '.' in variables (e.g. "hparams.parse('a.b=1.0')"), which would previously raise an error. This will correspond to an attribute name with an embedded '.' symbol (e.g. 'a.b'), which can only be accessed indirectly (e.g. through getattr and setattr). To set this up the user will first need to explicitly add the variable to the hparam object (e.g. "hparams.add_hparam(name='a.b', value=0.0)").

Dataset.list_files() now produces determinstic results when shuffle=False or a seed is passed.

Added LinearOperatorKronecker, a dense-free implementation of the Kronecker Product.

Allow LinearOperator to broadcast.

SavedModelBuilder will now deduplicate asset names that point to files with the same basename and the same contents. Note that this may result in new asset files included in SavedModels in cases where assets with the same name but different contents were previously overwriting each other.

Thanks to our Contributors

This release contains contributions from many people at Google, as well as:

Update benchmark for tf.scan to match ranges across eager and graph modes.

Fixed bug in tf.reduce_prod gradient for complex dtypes.

Allow the use of '.' in variables (e.g. "hparams.parse('a.b=1.0')"), which would previously raise an error. This will correspond to an attribute name with an embedded '.' symbol (e.g. 'a.b'), which can only be accessed indirectly (e.g. through getattr and setattr). To set this up the user will first need to explicitly add the variable to the hparam object (e.g. "hparams.add_hparam(name='a.b', value=0.0)").

Added LinearOperatorKronecker, a dense-free implementation of the Kronecker Product.

Allow LinearOperator to broadcast.

SavedModelBuilder will now deduplicate asset names that point to files with the same basename and the same contents. Note that this may result in new asset files included in SavedModels in cases where assets with the same name but different contents were previously overwriting each other.

Thanks to our Contributors

This release contains contributions from many people at Google, as well as:

Update benchmark for tf.scan to match ranges across eager and graph modes.

Fixed bug in tf.reduce_prod gradient for complex dtypes.

Add optional args argument to Dataset.from_generator().

Allow the use of '.' in variables (e.g. "hparams.parse('a.b=1.0')"), which would previously raise an error. This will correspond to an attribute name with an embedded '.' symbol (e.g. 'a.b'), which can only be accessed indirectly (e.g. through getattr and setattr). To set this up the user will first need to explicitly add the variable to the hparam object (e.g. "hparams.add_hparam(name='a.b', value=0.0)").

Dataset.list_files() now produces determinstic results when shuffle=False or a seed is passed.

Added LinearOperatorKronecker, a dense-free implementation of the Kronecker Product.

Allow LinearOperator to broadcast.

SavedModelBuilder will now deduplicate asset names that point to files with the same basename and the same contents. Note that this may result in new asset files included in SavedModels in cases where assets with the same name but different contents were previously overwriting each other.

Thanks to our Contributors

This release contains contributions from many people at Google, as well as:

Assets

Release 1.7.1

Bug Fixes and Other Changes

The TensorFlow Lite TOCO compiler did not perform correct boundary checks when reading from some fields within TFLite files.

The block size in meta file could contain a large int64 value which causes an integer overflow upon addition. Subsequent code using n as index may cause an out-of-bounds read.

TensorFlow checkpoint meta file uses Google's https://github.com/google/snappy compression/decompression library. There is a memcpy-param-overlap issue in the version of snappy currently used by TensorFlow.

Bug Fixes and Other Changes

Add tf.contrib.data.AUTOTUNE, which allows the tf.data runtime to automatically tune the prefetch buffer sizes based on your system and environment.

Add tf.contrib.data.make_csv_dataset for building datasets of CSV files.

Eager Execution:

With eager execution Datasets can now be used as standard python iterators (for batch in dataset:). Both Dataset.__iter__() and Dataset.make_one_shot_iterator() can now be used to create iterators when eager execution is enabled.

Automatic device placement has been enabled (i.e., use a GPU if available automatically, without requiring an explicit with tf.device(“/gpu:0”)) (Fixes #14133)

Fix bug in tf.contrib.opt.MultitaskOptimizerWrapper where types of tensors were mismatched.

Other:

Low-level graph construction now calls the TensorFlow C API. This change should be invisible to most users, but can be disabled by setting the environment variable TF_C_API_GRAPH_CONSTRUCTION=0 in this release. Future releases will remove the ability to disable this change. Please file a bug if you find yourself using this escape hatch.

Add description of shapes and a pointer to tutorial notebook in tf.distributions.Distribution.

Update scatter operations:

Add tf.scatter_min and tf.scatter_max

Extend scatter operations to work with a scalar update parameter.

Move cuDNN RNN ops to core for use in TensorFlow codebase only.

Add float64 support for Conv2d, Conv2dBackpropInput, and Conv2dBackpropFilter.

Add float64 support for AvgPool/AvgPoolGrad.

Make graph name scope thread local so that they work correctly in multi-threaded environments.

Bug Fixes and Other Changes

Add tf.contrib.data.AUTOTUNE, which allows the tf.data runtime to automatically tune the prefetch buffer sizes based on your system and environment.

Add tf.contrib.data.make_csv_dataset for building datasets of CSV files.

Eager Execution:

With eager execution Datasets can now be used as standard python iterators (for batch in dataset:). Both Dataset.__iter__() and Dataset.make_one_shot_iterator() can now be used to create iterators when eager execution is enabled.

Automatic device placement has been enabled (i.e., use a GPU if available automatically, without requiring an explicit with tf.device(“/gpu:0”)) (Fixes #14133)

Fix bug in tf.contrib.opt.MultitaskOptimizerWrapper where types of tensors were mismatched.

Other:

Low-level graph construction now calls the TensorFlow C API. This change should be invisible to most users, but can be disabled by setting the environment variable TF_C_API_GRAPH_CONSTRUCTION=0 in this release. Future releases will remove the ability to disable this change. Please file a bug if you find yourself using this escape hatch.

Add description of shapes and a pointer to tutorial notebook in tf.distributions.Distribution.

Update scatter operations:

Add tf.scatter_min and tf.scatter_max

Extend scatter operations to work with a scalar update parameter.

Move cuDNN RNN ops to core for use in TensorFlow codebase only.

Add float64 support for Conv2d, Conv2dBackpropInput, and Conv2dBackpropFilter.

Add float64 support for AvgPool/AvgPoolGrad.

Make graph name scope thread local so that they work correctly in multi-threaded environments.

Bug Fixes and Other Changes

Add tf.contrib.data.AUTOTUNE, which allows the tf.data runtime to automatically tune the prefetch buffer sizes based on your system and environment.

Add tf.contrib.data.make_csv_dataset for building datasets of CSV files.

Eager Execution:

With eager execution Datasets can now be used as standard python iterators (for batch in dataset:). Both Dataset.__iter__() and Dataset.make_one_shot_iterator() can now be used to create iterators when eager execution is enabled.

Automatic device placement has been enabled (i.e., use a GPU if available automatically, without requiring an explicit with tf.device(“/gpu:0”)) (Fixes #14133)

Fix bug in tf.contrib.opt.MultitaskOptimizerWrapper where types of tensors were mismatched.

Other:

Low-level graph construction now calls the TensorFlow C API. This change should be invisible to most users, but can be disabled by setting the environment variable TF_C_API_GRAPH_CONSTRUCTION=0 in this release. Future releases will remove the ability to disable this change. Please file a bug if you find yourself using this escape hatch.

Add description of shapes and a pointer to tutorial notebook in tf.distributions.Distribution.

Update scatter operations:

Add tf.scatter_min and tf.scatter_max

Extend scatter operations to work with a scalar update parameter.

Move cuDNN RNN ops to core for use in TensorFlow codebase only.

Add float64 support for Conv2d, Conv2dBackpropInput, and Conv2dBackpropFilter.

Add float64 support for AvgPool/AvgPoolGrad.

Make graph name scope thread local so that they work correctly in multi-threaded environments.

Known Bugs

Google discovered in mid-December 2017 that the PTX-to-SASS compiler in CUDA 9 and CUDA 9.1 sometimes does not properly compute the carry bit when decomposing 64-bit address calculations with large offsets (e.g. load [x + large_constant]) into 32-bit arithmetic in SASS.

As a result, these versions of ptxas miscompile most XLA programs which use more than 4GB of temp memory. This results in garbage results and/orCUDA_ERROR_ILLEGAL_ADDRESS failures.

A fix in CUDA 9.1.121 is expected in late February 2018. We do not expect a fix for CUDA 9.0.x. Until the fix is available, the only workaround is todowngrade to CUDA 8.0.x or disable XLA:GPU.

TensorFlow will print a warning if you use XLA:GPU with a known-bad version of CUDA; see e00ba24.

The tensorboard command or module may appear to be missing after certain upgrade flows. This is due to pip package conflicts as a result of changing the TensorBoard package name. See the TensorBoard 1.6.0 release notes for a fix.

Thanks to our Contributors

This release contains contributions from many people at Google, as well as:

Known Bugs

Google discovered in mid-December 2017 that the PTX-to-SASS compiler in CUDA 9 and CUDA 9.1 sometimes does not properly compute the carry bit when decomposing 64-bit address calculations with large offsets (e.g. load [x + large_constant]) into 32-bit arithmetic in SASS.

As a result, these versions of ptxas miscompile most XLA programs which use more than 4GB of temp memory. This results in garbage results and/orCUDA_ERROR_ILLEGAL_ADDRESS failures.

A fix in CUDA 9.1.121 is expected in late February 2018. We do not expect a fix for CUDA 9.0.x. Until the fix is available, the only workaround is todowngrade to CUDA 8.0.x or disable XLA:GPU.

TensorFlow will print a warning if you use XLA:GPU with a known-bad version of CUDA; see e00ba24.

The tensorboard command or module may appear to be missing after certain upgrade flows. This is due to pip package conflicts as a result of changing the TensorBoard package name. See the TensorBoard 1.6.0 release notes for a fix.

Thanks to our Contributors

This release contains contributions from many people at Google, as well as:

Add streaming_precision_recall_at_equal_thresholds, a method for computing streaming precision and recall with O(num_thresholds + size of predictions) time and space complexity.

Change RunConfig default behavior to not set a random seed, making random behavior independently random on distributed workers. We expect this to generally improve training performance. Models that do rely on determinism should set a random seed explicitly.

Replaced the implementation of tf.flags with absl.flags.

Add support for CUBLAS_TENSOR_OP_MATH in fp16 GEMM

Add support for CUDA on NVIDIA Tegra devices

Bug Fixes and Other Changes

Documentation updates:

Clarified that you can only install TensorFlow on 64-bit machines.

Added a short doc explaining how Estimators save checkpoints.

Add documentation for ops supported by the tf2xla bridge.

Fix minor typos in the doc of SpaceToDepth and DepthToSpace.

Updated documentation comments in mfcc_mel_filterbank.h and mfcc.h to clarify that the input domain is squared magnitude spectra and the weighting is done on linear magnitude spectra (sqrt of inputs).

Change tf.contrib.distributions docstring examples to use tfd alias rather than ds, bs.

Fix docstring typos in tf.distributions.bijectors.Bijector.

tf.assert_equal no longer raises ValueError. It now raises InvalidArgumentError, as documented.

Fix a bug in import_meta_graph's handling of partitioned variables when importing into a scope. WARNING: This may break loading checkpoints of graphs with partitioned variables saved after using import_meta_graph with a non-empty import_scope argument.

Fix bug in offline debugger which prevented viewing events.

Added the WorkerService.DeleteWorkerSession method to the gRPC interface, to fix a memory leak. Ensure that your master and worker servers are running the same version of TensorFlow to avoid compatibility issues.

Fix bug in peephole implementation of BlockLSTM cell.

Fix bug by casting dtype of log_det_jacobian to match log_prob in TransformedDistribution.

Fix a bug in import_meta_graph's handling of partitioned variables when

Ensure tf.distributions.Multinomial doesn't underflow in log_prob. Before this change, all partitions of an integer variable were initialized with the shape of the unpartitioned variable; after this change they are initialized correctly.

Other:

Add necessary shape util support for bfloat16.

Add a way to run ops using a step function to MonitoredSession.

Add DenseFlipout probabilistic layer.

A new flag ignore_live_threads is available on train. If set to True, it will ignore threads that remain running when tearing down infrastructure after successfully completing training, instead of throwing a RuntimeError.

Restandardize DenseVariational as simpler template for other probabilistic layers.