Two time-scale update rule for training GANs

Fréchet Inception Distance (FID)

The FID is the performance measure used to evaluate the experiments in the paper. There, a detailed description can be found
in the experiment section as well as in the the appendix in section A1.

In short:
The Fréchet distance between two multivariate Gaussians X_1 ~ N(mu_1, C_1) and X_2 ~ N(mu_2, C_2) is

d^2 = ||mu_1 - mu_2||^2 + Tr(C_1 + C_2 - 2*sqrt(C_1*C_2)).

The FID is calculated by assuming that X_1 and X_2 are the activations of the coding layer pool_3 of the inception model (see below) for generated samples and real world samples respectivly. mu_n is the mean and C_n the covariance of the activations of the coding layer over all real world or generated samples.

IMPORTANT: The number of samples to calculate the Gaussian statistics (mean and covariance) should be greater than the
dimension of the coding layer, here 2048 for the Inception pool 3 layer. Otherwise the covariance is not full rank resulting in complex numbers and nans by calculating the square root.

We recommend using a minimum sample size of 10,000 to calculate the FID otherwise the true FID of the generator is
underestimated.

Compatibility notice

Previous versions of this repository contained two implementations to calculate the FID, a "unbatched" and a "batched" version.
The "unbatched" version should not be used anymore. If you've downloaded this code previously, please update it immediately to
the new version. The old version included a bug!

Provided Code

Requirements: TF 1.1+, Python 3.x

fid.py

This file contains the implementation of all necessary functions to calculate the FID. It can be used either
as a python module imported into your own code, or as a standalone
script to calculate the FID between precalculated (training set) statistics and a directory full of images, or between
two directories of images.