Sunday, December 12, 2010

R Code Example for Neural Networks

In this past June's issue of R journal, the 'neuralnet' package was introduced. I had recently been familiar with utilizing neural networks via the 'nnet' package (see my post on Data Mining in A Nutshell) but I find the neuralnet package more useful because it will allow you to actually plot the network nodes and connections. (it may be possible to do this with nnet, but I'm not aware of how).The neuralnet package was written primarily for multilayer perceptron architectures, which may be a limitation if you are interested in other architectures.

The data set used was a default data set found in the package 'datasets' and consisted of 248 observations and 8 variables:

The following code runs the network (with 2 hidden layers) classifying 'case' (a binary variable) as a function of several independent varaibles. The neural network is estimated, and the results are stored in the data frame 'nn.'

nn <- neuralnet(

case~age+parity+induced+spontaneous,

data=infert, hidden=2, err.fct="ce",

linear.output=FALSE)

The weight estimates can be obtained with the following command:

nn$result.matrix

And, the network can be plotted or visualized with the simple command:

plot(nn)

As can be seen below, the output weights correspond directly with the visualized network. While this tool may be more difficult to interpret with more complex network architectures, I find this simplified version very useful for demonstration/instructional purposes.

Example: The weight for the path from input 'age' to the first hidden layer is -3.0689 (age.to.1layhid1) which can easily be found in the network diagram. After all inputs feed into hidden layer 1, the weight associated with the path from hidden layer 1(1layhid.1.to.case) to the output layer (which along with information from the other layers of the network will give us the classification of 'case') is -1001.15.

Estimated Network Weights

error

123.811

reached.threshold

0.009853

steps

16822

Intercept.to.1layhid1

0.535237

age.to.1layhid1

-3.0689

parity.to.1layhid1

2.262792

induced.to.1layhid1

30.50824

spontaneous.to.1layhid1

0.000961

Intercept.to.1layhid2

-5.52491

age.to.1layhid2

0.11538

parity.to.1layhid2

-1.98372

induced.to.1layhid2

2.550932

spontaneous.to.1layhid2

3.829613

Intercept.to.case

-1.7714

1layhid.1.to.case

-1001.15

1layhid.2.to.case

4.570324

Neural Network Visualization (click to enlarge)

There are many other options avaiable with the neural net package, I encourage you to read the article referenced below for more details.Reference: neuralnet: Training of Neural Networks by Frauke G¸nther and Stefan Fritsch The R Journal Vol. 2/1, June 2010

I've now included the code I used, but I have to say, each time I run it I get different results. I'm not sure if there is an option for specifying a seed or some way to get consistent results. there is also a lot more I need to know about interpreting the output beyond what I've already posted. Probably time to give the journal article and documentation a second read.

@Gyathri- I'm not sure about this. I first learned about Neural nets via the Rattle Package in R, and the journal article I cited above. I would like to know more if I get time to dig into it. If you use twitter, you might send out a tweet with #rstats hash tag and ask about this. I might if I get a chance in the future. You could try the R help listserv, but I've never used it as I've heard so much about it not being user friendly or welcoming to anyone that is not an expert in R.

@ Anonymous & Anonymous- I'll have to look at these issues- I'm not sure I'll get the answer. I'm still just learning these techniques and the code too! But my purpose of this blog is to facilitate more learning. I can barely keep up with new posts, ( I have a lot more I want to write) not to mention addressing these issues.

In the mean time, typically what I do when I have an issue like this, is I'll make a blog post and ask the question via twitter (with a link to the post like this one) using the #rstats hash tag. Stack overlfow is another web site that is useful to post questions about R-code, as well as the R-user group. If you have a linkedin profile, I notice that R-users are often using the linkedin R user interest group for these kind of discussions.

If feel bad for not providing you with a direct answer, but hopefully one of these resources will be helpful.

Anonymous is not using the correct package ('neuralnet', rather than 'nnet' - I am guessing nnet doesn't have the dataset... After loading nnet I got the same error, but the post's neuralnet example ran fine.

You're right that people, notably Brian Ripley, can get snarky about poorly written questions, but I found a lot of people willing to help me when I was first learning R and most beginner questions already have answers. Try a google search like "R how to install package" etc. Also take a look at the posting guide before posting:

I am trying to use Nnet for predicting continous data, wil it work? my dependent is a gamma distributed varible, so i used ur code but with little tweek for linout=true. but i am not getting results. please see the error below and advice me what other changes can be made

From the perspective of R, how do we use the result of the neural network exercise? Do we come up with an equation? Does it work like a regression equation where, after the procedure, we can just plug in numbers to get a predicted value?

Are we sure that this package works properly? I am trying to figure it out with the simplest of data, and even though it gives the very known error: Warning message:algorithm did not converge in 1 of 1 repetition(s) within the stepmax

or a huge Error in the network prediction that makes outputs have no sense. It doesn't even working with the simple data table when you have a first column a number and the second column is numbers' 2X and as output you want the sum of these two columns...

About Me

My primary interest is applied econometrics, machine learning, and predictive modeling. Other interests include applied economics with applications related to the interrelationships between biotechnology, nutrition and health, big data, and genomics.