1 Answer

The difficulty of training artificial recurrent neural networks has to do with their complexity.

One of the simplest ways to explain why recurrent neural networks are hard to train is that they are not feedforward neural networks.

In feedforward neural networks, signals only move one way. The signal moves from an input layer to various hidden layers, and forward, to the output layer of a system.

By contrast, recurrent neural networks and other different types of neural networks have more complex signal movements. Classed as “feedback” networks, recurrent neural networks can have signals traveling both forward and back, and may contain various “loops” in the network where numbers or values are fed back into the network. Experts associate this with the aspect of recurrent neural networks that's associated with their memory.