I am not a mathematician. I enjoy a good math puzzle, but I admit my weaknesses whole heartedly. That said, I've always had an interest in Neural Networks, and while I understand them enough to implement them from scratch, I hit a wall when I need to understand any concept that I can only find mathematic proofs for. Where is the programmer's guide to neural networks, using code instead of formula to explain the practical reasonings?

Neural networks are just fancy interpolation functions. I doubt you can get an even remotely accurate description of a neural network without explaining what interpolation is.
–
Alexandre C.May 3 '11 at 20:50

5 Answers
5

Another alternative is a non-math, non-programming explanation. The book Blondie24: Playing at the Edge of AI contains a really great explanation of neural networks. It's about a checkers-playing AI developed by the author. It's not completely without programming references, but it does a great job of explaining how the algorithms work without getting into the code of the solution.

The best way to have a "programmer's understanding" of neural networks is not so much by examining the code, but in the problem and the correct results. So, if you don't want to look at math, I recommend you look at a given problem. For example, consider the XOR problem as an example of why you need non-linear activation functions, look at the number of variables and their possible values for understanding why a neural network needs to be of a certain size and toplogy to be effective, and split your data into train/test regimes and do studies to see why overfitting is dangerous. Examine the code with the data.

I also recommend not getting too hung up, but reading further. Certain practices in feed-forward networks become more clear once you see their generalization in recurrent and constructive neural networks. I also recommend going wider: Bayesian networks, fuzzy cognitive maps, SOM, Boltzman machines, simulated annealing, and reinforcement learning all have intuitions.

I understand it better with a small problem like a XOR network. My grasping of the networks behavior falls apart with problems I couldn't solve myself, and backpropagation becomes my wall. I could write it, but I don't understand the details of how it works.
–
ironfroggyNov 23 '08 at 15:53

I was in the same boat w/ backprop, I couldn't understand it, but over time I saw so many related ideas, that I finally got it. Let me try a couple of explanations.
–
John with waffleNov 23 '08 at 16:04

The internal nodes are features. So, first you learn what inputs together make a feature, and then then which features influence an output. The problem is at the output, you don't know if the feature is unrelated, or whether the feature is poorly determined.
–
John with waffleNov 23 '08 at 16:06

So, if it is unrelated, that means other examples will compensate with correcting the weights of the inputs later. But, if the feature is poorly determined, then the other examples will not compensate, and the features will change.
–
John with waffleNov 23 '08 at 16:06

There is a closely related technique in reinforcement learning called value propagation. In reinforcement learning, you are trying to optimize some reward, but if you do something good, you don't know which action led to your reward, so you have to spread the reward back, and average it out.
–
John with waffleNov 23 '08 at 16:09

You need a bit of understanding in either Pascal or Delphi but this overview from ThinkQuest is pretty helpful from a programming stand-point. It also explains some of the difficulties and why the mathematics looks a bit intimidating. (I'm not a mathematician either.)

I was pretty interested in this sort of thing a while back (still am for the most part) and in search of some walk-throughs that I can follow pretty quickly.

The author in my opinion does not fully utilize the more powerful functionality of C++, in many cases it reads more like traditional C with classes. The book is also a little dated by now.

HOWEVER - if you need explanations of the algorithms and techniques used in neural networks, explained in a way that an intelligent layperson could understand, so that you can go away and try these things for yourself, then I would certainly give this book a try. Not much navel-gazing goes on here, which is what I liked.

It takes you through all the main things needed to program a neural network - how to compare the actual output with the desired in order to obtain an error signal, and then use this error signal in conjunction with back propagation algorithms to modify the network link connection strengths, doing this iteratively so that gradually the neural network 'learns' the task.