Dynamic-Net: We propose an approach that enables traversing the “objective-space”, spanned by two different
objectives, at test-time, without re-training, as illustrated by the blue dot moving along the blue curve in the plot. This is
different from the common practice of training a separate network for each objective, represented by X’s on the plot. Using a
single Dynamic-Net we can tune the level of stylization of an image, monitor completion quality per image, or control facial
attributes, all interactively at test-time, without re-training.

Abstract

One of the key ingredients for successful optimization of
modern CNNs is identifying a suitable objective. To date,
the objective is fixed a-priori at training time, and any variation
to it requires re-training a new network. In this paper
we present a first attempt at alleviating the need for
re-training. Rather than fixing the network at training time,
we train a “Dynamic-Net” that can be modified at inference
time. Our approach considers an “objective-space” as the
space of all linear combinations of two objectives, and the
Dynamic-Net can traverse this objective-space at test-time,
without any further training. We show that this upgrades
pre-trained networks by providing an out-of-learning extension,
while maintaining the performance quality. The
solution we propose is fast and allows a user to interactively
modify the network, in real-time, in order to obtain
the result he/she desires. We show the benefits of such an
approach via several different applications.

Proposed framework: Our training has two steps: (i) First the “main” network \(\theta\) (green blocks) is trained to
minimize \({\cal O}_0\). (ii) Then \(\theta\) is fixed, one or more tuning-blocks \(\psi\) are added (orange block), and trained to minimize \({\cal O}_1\). The
output \(\hat y_1\) approximates the output \(y_1\) one would get from training the main network \(\theta\) with objective \({\cal O}_1\). At test-time, we
can emulate results equivalent to a network trained with objective \({\cal O}_m\) by tuning the parameter \(\alpha_m\) (in blue) that determines
the latent representation \(z_m\). Our method can be applied as (a) a single-block framework or as (b) multi-block framework.

Applications

Dynamic Style Transfer

Control over Stylization level

Interpolation between two Styles

Dynamic DC-GAN: Controlled Image Generation

The proposed method allow us to generate faces with control over the facial attributes e.g gender or hair color.

Image Completion

Input

\(\alpha=0\)

\(\alpha=0.4\)

\(\alpha=1\)

Input

\(\alpha=0\)

\(\alpha=0.5\)

\(\alpha=1\)

Original result

Improved result

Original result

Improved result

Dynamic-Net allows the user to select the best working point for each image, improving results of networks that were trained with sub-optimal objectives.