tf.contrib.rnn.IntersectionRNNCell

Class IntersectionRNNCell

Architecture with coupled recurrent gate as well as coupled depth
gate, designed to improve information flow through stacked RNNs. As the
architecture uses depth gating, the dimensionality of the depth
output (y) also should not change through depth (input size == output size).
To achieve this, the first layer of a stacked Intersection RNN projects
the inputs to N (num units) dimensions. Therefore when initializing an
IntersectionRNNCell, one should set num_in_proj = N for the first layer
and use default settings for subsequent layers.

Returns:

Methods

__init__

Args:

num_units: int, The number of units in the +RNN cell

num_in_proj: (optional) int, The input dimensionality for the RNN.
If creating the first layer of an +RNN, this should be set to
num_units. Otherwise, this should be set to None (default).
If None, dimensionality of inputs should be equal to num_units,
otherwise ValueError is thrown.

initializer: (optional) The initializer to use for the weight matrices.

forget_bias: (optional) float, default 1.0, The initial bias of the
forget gates, used to reduce the scale of forgetting at the beginning
of the training.

y_activation: (optional) Activation function of the states passed
through depth. Default is 'tf.nn.relu`.

reuse: (optional) Python boolean describing whether to reuse variables
in an existing scope. If not True, and the existing scope already has
the given variables, an error is raised.

__call__

__call__(
inputs,
state,
scope=None
)

Run this RNN cell on inputs, starting from the given state.

Args:

inputs: 2-D tensor with shape [batch_size, input_size].

state: if self.state_size is an integer, this should be a 2-D Tensor
with shape [batch_size, self.state_size]. Otherwise, if
self.state_size is a tuple of integers, this should be a tuple
with shapes [batch_size, s] for s in self.state_size.

scope: VariableScope for the created subgraph; defaults to class name.

Returns:

A pair containing:

Output: A 2-D tensor with shape [batch_size, self.output_size].

New state: Either a single 2-D tensor, or a tuple of tensors matching
the arity and shapes of state.

__deepcopy__

__deepcopy__(memo)

add_loss

add_loss(
losses,
inputs=None
)

Add loss tensor(s), potentially dependent on layer inputs.

Some losses (for instance, activity regularization losses) may be dependent
on the inputs passed when calling a layer. Hence, when reusing the same
layer on different inputs a and b, some entries in layer.losses may
be dependent on a and some on b. This method automatically keeps track
of dependencies.

The get_losses_for method allows to retrieve the losses relevant to a
specific set of inputs.

Note that add_loss is not supported when executing eagerly. Instead,
variable regularizers may be added through add_variable. Activity
regularization is not supported directly (but such losses may be returned
from Layer.call()).

Arguments:

losses: Loss tensor, or list/tuple of tensors.

inputs: If anything other than None is passed, it signals the losses
are conditional on some of the layer's inputs,
and thus they should only be run where these inputs are available.
This is the case for activity regularization losses, for instance.
If None is passed, the losses are assumed
to be unconditional, and will apply across all dataflows of the layer
(e.g. weight regularization losses).

Raises:

RuntimeError: If called in Eager mode.

add_update

add_update(
updates,
inputs=None
)

Add update op(s), potentially dependent on layer inputs.

Weight updates (for instance, the updates of the moving mean and variance
in a BatchNormalization layer) may be dependent on the inputs passed
when calling a layer. Hence, when reusing the same layer on
different inputs a and b, some entries in layer.updates may be
dependent on a and some on b. This method automatically keeps track
of dependencies.

The get_updates_for method allows to retrieve the updates relevant to a
specific set of inputs.

This call is ignored in Eager mode.

Arguments:

updates: Update op, or list/tuple of update ops.

inputs: If anything other than None is passed, it signals the updates
are conditional on some of the layer's inputs,
and thus they should only be run where these inputs are available.
This is the case for BatchNormalization updates, for instance.
If None, the updates will be taken into account unconditionally,
and you are responsible for making sure that any dependency they might
have is available at runtime.
A step counter might fall into this category.

add_variable

Adds a new variable to the layer, or gets an existing one; returns it.

Arguments:

name: variable name.

shape: variable shape.

dtype: The type of the variable. Defaults to self.dtype or float32.

initializer: initializer instance (callable).

regularizer: regularizer instance (callable).

trainable: whether the variable should be part of the layer's
"trainable_variables" (e.g. variables, biases)
or "non_trainable_variables" (e.g. BatchNorm mean, stddev).
Note, if the current variable scope is marked as non-trainable
then this parameter is ignored and any added variables are also
marked as non-trainable.

constraint: constraint instance (callable).

partitioner: (optional) partitioner instance (callable). If
provided, when the requested variable is created it will be split
into multiple partitions according to partitioner. In this case,
an instance of PartitionedVariable is returned. Available
partitioners include tf.fixed_size_partitioner and
tf.variable_axis_size_partitioner. For more details, see the
documentation of tf.get_variable and the "Variable Partitioners
and Sharding" section of the API guide.

Returns:

The created variable. Usually either a Variable or ResourceVariable
instance. If partitioner is not None, a PartitionedVariable
instance is returned.

Raises:

RuntimeError: If called with partioned variable regularization and
eager execution is enabled.

apply

apply(
inputs,
*args,
**kwargs
)

Apply the layer on a input.

This simply wraps self.__call__.

Arguments:

inputs: Input tensor(s).

*args: additional positional arguments to be passed to self.call.

**kwargs: additional keyword arguments to be passed to self.call.

Returns:

Output tensor(s).

build

build(_)

call

call(
inputs,
state
)

Run one step of the Intersection RNN.

Args:

inputs: input Tensor, 2D, batch x input size.

state: state Tensor, 2D, batch x num units.

Returns:

new_y: batch x num units, Tensor representing the output of the +RNN
after reading inputs when previous state was state.

new_state: batch x num units, Tensor representing the state of the +RNN
after reading inputs when previous state was state.