Simple RMSProp distributed learning

This post is the first one in the series of posts where I will describe the creation of a neural network, modelling the behaviour of a small organism. Why model such a system, what functions it will perform and what the results I will get will be described in the next posts.

The only thing I want to describe in this post is the solution which helps accelerate the training of neural networks.

In order to train the neural network, one needs to collect a large pool of data. But if you work with systems that are training in real time, it is impossible to collect the data beforehand (e.g. Markov decision process). Every action of the neural network influences its environment and changes it. It makes it very difficult to generate a necessary amount of data. In this case I decided to use parallel training by dividing the network.

I divided the system into two components: a Trainer and a Player.

A Trainer is a class incapsulating the functions of backward training and actual weights. A Trainer decides how many Players can play simultaneously.

A Player is a class that plays a game by getting a snapshot of actual weights from the Trainer.

After finishing the game a Player returns a set of hidden layer states and input data to the Trainer. As all Players play in the same homogenous environment we can assume that their games are of equal value.

There exists some discrepancy related to the lag of players from actual weights but it is negligible compared to the increase in speed of training.

Below you can see the diagrams of the weights distribution before and after the training and the diagram of the speed of training of such model.