Using SRGANs to Generate Photo-realistic Images [Tutorial]

Super-Resolution Generative Adversarial Network, or SRGAN, is a Generative Adversarial Network (GAN) that can generate super-resolution images from low-resolution images, with finer details and higher quality. CNNs were earlier used to produce high-resolution images that train quicker and achieve high-level accuracy. However, in some cases, they are incapable of recovering finer details and often generate blurry images.

This tutorial is an excerpt taken from the book Generative Adversarial Networks Projects written by Kailash Ahirwar. The book explores unsupervised techniques for training neural networks and includes seven end-to-end projects in the GAN domain.

Downloading the CelebA dataset

For this tutorial, we will use the large-scale CelebFaces Attributes (CelebA) dataset, which is available here. The dataset contains 202, 599 face images of celebrities.

The dataset is available for non-commercial research purposes only and can’t be used for commercial purposes. If you intend to use the dataset for commercial purposes, seek permissions from the owners of the images.

We will use the CelebA dataset to train our SRGAN network. Perform the following steps to download and extract the dataset:

Extract images from the downloaded img_align_celeba.zip by executing the following command:

unzip img_align_celeba.zip

We have now downloaded and extracted the dataset. We can now start working on the Keras implementation of SRGAN.

The Keras implementation of SRGAN

SRGAN has three neural networks, a generator, a discriminator, and a pre-trained VGG19 network on the Imagenet dataset. In this section, we will write the implementation for all the networks. Let’s start by implementing the generator network.

Before starting to write the implementations, create a Python file called main.py and import the essential modules, as follows:

Now, add an Add layer to take the sum of the output from the pre-residual block, which is gen1, and the output from the post-residual block, which is gen2. This layer generates another tensor of similar shape.

Once you have defined all the layers in the network, you can create a Keras model. We have defined a Keras sequential graph using Keras’s functional API. Let’s create a Keras model by specifying the input and output for the network.

Now, create a Keras model and specify the inputs and the outputs for the model, as follows:

In this section, we have successfully created a Keras model for the discriminator network.

The adversarial network

The adversarial network is a combined network that uses the generator, the discriminator, and VGG19. In this section, we will create an adversarial network.
Perform the following steps to create an adversarial network:

Start by creating an input layer for the network:

input_low_resolution = Input(shape=(64, 64, 3))

The adversarial network will receive an image of a shape of (64, 64, 3), which is why we have created an input layer.

We have now successfully implemented the networks in Keras. Next, we train the network on the dataset that we downloaded.

Training the SRGAN

Training the SRGAN network is a two-step process. In the first step, we train the discriminator network. In the second step, we train the adversarial network, which eventually trains the generator network. Let’s start training the network.
Perform the following steps to train the SRGAN network:

The code for the sample_images function is as follows. It is quite descriptive and can be understood by going through it. It contains different steps to load and resize the images to generate high-resolution as well as low-resolution images:

It is very important to convert the pixel values to a range of between -1 to 1. Our generator network has tanh at the end of the network. The tanh activation function squashes values to the same range. While calculating the loss, it is necessary to have all values in the same range. After this step is complete, we train the discriminator network, generator network, and then further visualize the images and evaluate the model.

In this tutorial, we learned how to download the CelebA dataset, and implemented the project in Keras before training the SRGAN. If you want to learn more about how to evaluate the trained SRGAN network, and optimizing the trained model, be sure to check out the book Generative Adversarial Networks Projects.