Introduction

Artificial Neural Networks are a recent development tool that are modeled from biological neural networks. The powerful side of this new tool is its ability to solve problems that are very hard to be solved by traditional computing methods (e.g. by algorithms). This work briefly explains Artificial Neural Networks and their applications, describing how to implement a simple ANN for image recognition.

Background

I will try to make the idea clear to the reader who is just interested in the topic.

About Artificial Neural Networks (ANNs)

Artificial Neural Networks (ANNs) are a new approach that follow a different way from traditional computing methods to solve problems. Since conventional computers use algorithmic approach, if the specific steps that the computer needs to follow are not known, the computer cannot solve the problem. That means, traditional computing methods can only solve the problems that we have already understood and knew how to solve. However, ANNs are, in some way, much more powerful because they can solve problems that we do not exactly know how to solve. That's why, of late, their usage is spreading over a wide range of area including, virus detection, robot control, intrusion detection systems, pattern (image, fingerprint, noise..) recognition and so on.

ANNs have the ability to adapt, learn, generalize, cluster or organize data. There are many structures of ANNs including, Percepton, Adaline, Madaline, Kohonen, BackPropagation and many others. Probably, BackPropagation ANN is the most commonly used, as it is very simple to implement and effective. In this work, we will deal with BackPropagation ANNs.

BackPropagation ANNs contain one or more layers each of which are linked to the next layer. The first layer is called the "input layer" which meets the initial input (e.g. pixels from a letter) and so does the last one "output layer" which usually holds the input's identifier (e.g. name of the input letter). The layers between input and output layers are called "hidden layer(s)" which only propagate the previous layer's outputs to the next layer and [back] propagates the following layer's error to the previous layer. Actually, these are the main operations of training a BackPropagation ANN which follows a few steps.

A typical BackPropagation ANN is as depicted below. The black nodes (on the extreme left) are the initial inputs. Training such a network involves two phases. In the first phase, the inputs are propagated forward to compute the outputs for each output node. Then, each of these outputs are subtracted from its desired output, causing an error [an error for each output node]. In the second phase, each of these output errors is passed backward and the weights are fixed. These two phases is continued until the sum of [square of output errors] reaches an acceptable value.

Implementation

The network layers in the figure above are implemented as arrays of structs. The nodes of the layers are implemented as follows:

Compare the resulting output with the desired output for the given input. This is called the error.

Modify the weights for all neurons using the error.

Repeat the process until the error reaches an acceptable value (e.g. error < 1%), which means that the NN was trained successfully, or if we reach a maximum count of iterations, which means that the NN training was not successful.

Great job buddy i need a bit help, how can i decide the architecture of a nueral network? means for how many inputs, outputs etc for a particular problem? can u suggest me architecture for face detection in an image plzz? or any other resource u can suggest which can ans my question?
Thanx in advance

We have reviewed your image recognition work. We want to discuss with you regarding development of this per our needs with some advanced features. Please advise in case you are interested in providing commerical services for this technology.

This is by far the best resource I've been able to find online. It's a really exciting area with many possibilities. I'm really interested in combining neural nets with genetic algorithms. It feels like an area of computer science that is "on the brink" - very close to becoming something amazing.

Anyway, well done for being selfless and making this available for people! Good Karma to you!

i'm sorry if my question seems so stupid but i totally new in this.i just started to learn a few weeks ago.but i still don't understand if we want to recognize any pattern or picture we should use any tools or completely coding.What is the tool if i have to use tool and what is the language that i should use to do it.Can java make it?

The problem is not the programming language you use, but the kind of pattern you want to recognize. a pattern can be speech, picture, character and many others. There are not much present tool to handle these and you should never write yourself if you just need to recognize a kind of pattern. you may use libraries. For example, if you look for OCR, MODI or tesseract are perfect.

As I know, in your program the number of input units of network is determined by the number of pixels of pattern. But I found that the patterns in your file do not have the same size. There are 23x33, 29x33, 19x33... I would like to know when I train the network, how many input units are there indeed? It is impossible to train the same network if the patterns have different dimension.
Thank you for your response.

Hi everybody!
My name is Khiem , I'm from Vietnam.
I downloaded the demo project including the source code 5 days ago for research but I got a problem installing the software in my PC. I do not install the C# program for my PC yet , do I need to install it?
The fact that I used my colleges computer to download it , and this computer doesn't have C# yet , ironically , it works .
So please tell me what steps I should do to make it work . Thanks so much for your help .

Hi
u have provided a very good neural network system.
But i did not understand the logic of how did u decide the number of input units, hidden units and output units. and how does the output units determine which image it is liek suppose u gave 34 output units then how do those 34 units wil decide whch image of training set it is
pls reply as soon as possible

if the bitmap is 20x30 then there are 600 inputs (each pixel is mapped to one input). and if there are 34 training pattern there are 34 outputs (each pattern is mapped to one output, say "0" refers to 1st output, "1" refers to 2nd output.. "z" refers to 34th output..) hidden unit number may differ between number of input and output units.

I would like to train the network with many tags, which means many similar vectors with little variance. When the training is over, I want to pass a test tag in front of the device, obtain a vector of data, and get from the network if the data somehow "fits" to the trained data.
For a better understanding, the data in the vector changes if the tag goes left-right or up-down. I would train using the same "direction" and test with a tag in the other direction, leading to a "rejection" of the item.

My question is :
Does it make sense to use a neural network (even yours) for such a case?
My training data might be composed of 5000 vectors. The way I see how I could use your application library is to define each of these vectors as "pattern", and then look at the probabilities if the test vector reaches for example >60% similarities with one of the vectors.

Hi, your source code is easy to understand and compile. Thanks.
However I think there is an error in the Backpropagation function.
It is missing a factor of "Output*(1-Output) for all hidden layers.
The link below gives a very good explanation of the math and easy to follow.

http://www.speech.sri.com/people/anand/771/html/node37.html

I am trying to use the structure of your code to do some real work and can't afford to have any error in the algorithm.
It would be great if you could clarify if I am wrong.
Thanks.

You are right. I removed Output*(1-Output) factor from hidden layers. Since, unfortunately, it causes trainig process an "endless process". Maybe it stems from selecting wrong initial values, or wrong choice of hidden layer number or anything else. But, I think, if the purpose of traing network is to decrease the error, this also works fine(I can not prove ) .If you want to apply original theory do that:

In your application , is it only restricted to 0/1 in value of input or output ? can I use this to identify something colorful ?

If not I remember wrong , some application may have threshold value , implying that it seems to turn out 0/1
result . If I want to apply this on continious values ,
do I need to implement the threshould ?

By the way , Is there any simple neural network tutorial
website recommended , I want to learn more .... >_
Thanks You Very Much
Your Product is worthy of full mark ( 100 )

Actually, selecting proper numbers for hidden layer's nodes is confusing. there are some theoritical expressions about it but, nevertheless, not so clear.
As I said at previous comment, if there is an input having 600 nodes, first hidden sholud have 250 nodes, second hidden should have 100 nodes and output have 34 units([from input to output] node numbers should be choosen in descending order).

I dont use 0-1 as input but double. the following method, for example, converts a bitmap(which is input) to double array:

I want to ask you about your aplication "C# Implementation of BackPropagation Neural Network For Pattern Recognition".

can you explain me more details about the tab settings.I don't understand what for is "Number of layers", "Maximum Error", "Number of Input Unit", "Number of Hidden Unit", why default value for "Number of Input Unit" is 250, and why default value for "Number of Hidden Unit" is 100.

A neural network structure contains 1 or more layers. the first layer is called "input layer" and the last one is called "output layer". Layers between the first and last layer called "hidden layers".
A layer contains many nodes. If input layer contains 600 nodes, then we say number of input unit is 600. Dont keep your mind on default values of layer units. they are just "default" and can be changed. but if network contains 600 inputs and 34 outputs, hidden layers (which are between input and output layer) should have unit numbers between 600 and 34. in this example, input have 600, first hidden have 250, second hidden have 100 and output have 34 units.
Finally, "Maximum error" indicates how well the network will be trained.(a smaller value of "Maximum error" means, network will be trained better)

I am a student from Croatia who studies Computer science. I have to write cca. 20 pages about: Use of neural networks for pattern recognition.
describe: -types of networks which are used in this purpose
-commercial tools
-achieved results

I found your work on http://www.codeproject.com/ so I thought that you could help me beacuse you are probably good in this field.

If I have an image with a string of characters like ABC123, will this work on that image also? Do I need to normalize and wash out the image to make the letters appear black and white before this will work??

An image containing ABC123 should be divided into images as A,B,D,1,2,3 to be recognized. It is also possible to train the network to recognize images containing 6 letters but probably no any PC would train such a network. (As only (28+10) output is enought for the first case, however, there must be 38*38*38*38*38*38 outputs for the second case.) Probably, it would take a simple PC days to train such a network

It does'nt required to map pixels to 0 and 1 or wash out. Gray level pixels also work fine.

I am not one to comment normally, but good job, I have been timidly browsing the net for something like this a couple of days now, exactly what I was looking for. Could use a couple more comments =], I am just sitting around here looking at your code, and I was wondering, what do I need to change to have the program check just say the border of an image? Again, Great Work, thanks!

To Check the Borders of Images:
1-Insert the pixels [in the borders of images] into TrainingSet as array of double.
2-Train Network
3-Send the borders of an image [as the parameter] to Recognize method.