Neural Network Tutorial:

Installation

The quickest way to install is with easy_install. Since this is a Python library,
at the Python prompt put:

easy_installpyneurgen

This section will go through an example to get acquainted with the
software. To illustrate what is happening here, we will also use a separate
Python software package called matplotlib. If you are not already acquainted
with the package, you will find that it is very helpful for 2d plotting and well
worth adding to your system. It can be gotten at
http://matplotlib.sourceforge.net

We will start with a simple neural network that calculates a sine
related function with a slight random component to make it slightly more
interesting:

Inputs and Targets

Now, we will set up inputs and targets. The following code just creates a
list of inputs and targets. The random component is added as an input to make
the overall result a little more interesting or challenging. The network must
learn to ignore the random input and will just have to live with trying to
understand the random component in the target data. One important thing to
remember is that the input should be normalized so that it is in the vicinity of
-1 to +1. That helps to avoid overflow issues related to weights trying to
adjust for errors. This package does not normalize data automatically, because
it should be done individually for each project. For example, if timeseries
data was normalized automatically, it might have a different starting point
between testing and putting the neural network into production. The chart at
the right shows the data that is created.

The population is built and then sorted randomly. The data is placed
into the inputs and targets for the neural net. The last 20% of the data
will be used for testing.

# all samples are drawn from this populationpop_len=200factor=1.0/float(pop_len)population=[[i,math.sin(float(i)*factor*10.0)+ \
random.gauss(float(i)*factor,.2)]foriinrange(pop_len)]all_inputs=[]all_targets=[]defpopulation_gen(population):""" This function shuffles the values of the population and yields the items in a random fashion. """pop_sort=[itemforiteminpopulation]random.shuffle(pop_sort)foriteminpop_sort:yielditem# Build the inputsforposition,targetinpopulation_gen(population):pos=float(position)all_inputs.append([random.random(),pos*factor])all_targets.append([target])

Network Structure

After importing and instantiating the main class NeuralNet
called net. Then, it specifies a network structure of 2 inputs,
1 hidden layer of 10 nodes, and finally, 1 output. Note that this hidden
layer is a list and can contain any number of hidden layers, although more than
two hidden layers would be unusual.

The default values initiate a network structure with a set of network nodes.
Each node is fully connected to the layer below. In addition, the learn rate is
set to .20. That means that 20% of the error between each instance of target
and output will be communicated back down the network during training. The next
step in the process is to randomize the weights of each connection. Without
starting random values, the process of search is not able to differentiate one
set of nodes from another.

net=NeuralNet()net.init_layers(2,[10],1)net.randomize_network()net.set_halt_on_extremes(True)# Set to constrain beginning weights to -.5 to .5# Just to show we cannet.set_random_constraint(.5)net.set_learnrate(.1)

Now that the network structure is created, the inputs and targets are
loaded into the system.

net.set_all_inputs(all_inputs)net.set_all_targets(all_targets)

The first 80% of the data will be used for learning and the rest will be
set aside for testing. We will skip validation testing for these purposes.
The net object is notified of that in the following:

The activation types for a network default to 'linear' for the input
layer, 'sigmoid' for the hidden layers, and 'linear' for the output. For
this example, the hidden layer will be set to 'tanh'. Layers are organized in
a list with the input layer as 0, hidden layers 1 and above, ending in the
output layer as the last.

net.layers[1].set_activation_type('tanh')

At this point we have the data loaded, the network structure
defined and ready. We can now start the network learn process which sets the
appropriate weights for each connection. This process will be run in this
case with 10 trips through the data. Each trip is called an epoch. There
is also an option of loading in the data in a random fashion.

net.learn(epochs=125,show_epoch_results=True,random_testing=False)

With learning complete, it is time to test and evaluate the results.

mse=net.test()

To view the results we will use some matplotlib charts. The first chart
compares the test versus actual for the 20% that was tested. You can see that
the general shape of the population shows in the sample. In addition, the
actual values are mostly representative of the target values. Because of the
random component of the population data, there is no expectation that it would
exactly match.

The second chart shows the mean squared errors for each epoch.
Ideally, the progress of reducing error should be steadily downwards.

Below, is the code used to generate the charts:

test_positions=[item[0][1]*1000.0foriteminnet.get_test_data()]all_targets1=[item[0][0]foriteminnet.test_targets_activations]allactuals=[item[1][0]foriteminnet.test_targets_activations]# This is quick and dirty, but it will show the resultssubplot(3,1,1)plot([i[1]foriinpopulation])title("Population")grid(True)subplot(3,1,2)plot(test_positions,all_targets1,'bo',label='targets')plot(test_positions,allactuals,'ro',label='actuals')grid(True)legend(loc='lower left',numpoints=1)title("Test Target Points vs Actual Points")subplot(3,1,3)plot(range(1,len(net.accum_mse)+1,1),net.accum_mse)xlabel('epochs')ylabel('mean squared error')grid(True)title("Mean Squared Error by Epoch")show()