Slideshare uses cookies to improve functionality and performance, and to provide you with relevant advertising. If you continue browsing the site, you agree to the use of cookies on this website. See our User Agreement and Privacy Policy.

Slideshare uses cookies to improve functionality and performance, and to provide you with relevant advertising. If you continue browsing the site, you agree to the use of cookies on this website. See our Privacy Policy and User Agreement for details.

3.
Motivation & Contribution
“One of the challenges in the study of generative
adversarial networks is the instability of its training.”
: Proposed a novel weight normalization technique called
spectral normalization to stabilize the training of the
discriminator of GANs.
• Lipschitz constant is the only hyper-parameter to be tuned,
and the algorithm does not require intensive tuning of the
only hyper-parameter for satisfactory performance.
• Implementation is simple and the additional computational
cost is small.

8.
GANs is hard to train… why?
WGAN, GAN-GP
While input based regularizations allow for relatively easy formulations
based on samples, they also suffer from the fact that, they cannot
impose regularization on the space outside of the supports of the
generator and data distributions without introducing somewhat
heuristic means.

15.
Why SN is better than…
… Gradient Penalty
• The approach has an obvious weakness of being heavily dependent on the support of
the current generative distribution. As a matter of course, the generative distribution
and its support gradually changes in the course of the training, and this can
destabilize the effect of such regularization.
• While this seems to serve the same purpose as spectral normalization, orthonormal
regularization are mathematically quite different from our spectral normalization
because the orthonormal regularization destroys the information about the spectrum
by setting all the singular values to one. On the other hand, spectral normalization
only scales the spectrum so that the its maximum will be one.
… Orthonormal Regularization

22.
Summary
They proposed a novel weight normalization technique
called spectral normalization to stabilize the training of the
discriminator of GANs
• in various network architectures
• in various hyperparameter settings
• in various datasets
• with an intuitive and straight forward idea
• only using a relatively easy linear algebra knowledge
Practicality
Principled way