Introduction

Artificial Neural Networks are a recent development tool that are modeled from biological neural networks. The powerful side of this new tool is its ability to solve problems that are very hard to be solved by traditional computing methods (e.g. by algorithms). This work briefly explains Artificial Neural Networks and their applications, describing how to implement a simple ANN for image recognition.

Background

I will try to make the idea clear to the reader who is just interested in the topic.

About Artificial Neural Networks (ANNs)

Artificial Neural Networks (ANNs) are a new approach that follow a different way from traditional computing methods to solve problems. Since conventional computers use algorithmic approach, if the specific steps that the computer needs to follow are not known, the computer cannot solve the problem. That means, traditional computing methods can only solve the problems that we have already understood and knew how to solve. However, ANNs are, in some way, much more powerful because they can solve problems that we do not exactly know how to solve. That's why, of late, their usage is spreading over a wide range of area including, virus detection, robot control, intrusion detection systems, pattern (image, fingerprint, noise..) recognition and so on.

ANNs have the ability to adapt, learn, generalize, cluster or organize data. There are many structures of ANNs including, Percepton, Adaline, Madaline, Kohonen, BackPropagation and many others. Probably, BackPropagation ANN is the most commonly used, as it is very simple to implement and effective. In this work, we will deal with BackPropagation ANNs.

BackPropagation ANNs contain one or more layers each of which are linked to the next layer. The first layer is called the "input layer" which meets the initial input (e.g. pixels from a letter) and so does the last one "output layer" which usually holds the input's identifier (e.g. name of the input letter). The layers between input and output layers are called "hidden layer(s)" which only propagate the previous layer's outputs to the next layer and [back] propagates the following layer's error to the previous layer. Actually, these are the main operations of training a BackPropagation ANN which follows a few steps.

A typical BackPropagation ANN is as depicted below. The black nodes (on the extreme left) are the initial inputs. Training such a network involves two phases. In the first phase, the inputs are propagated forward to compute the outputs for each output node. Then, each of these outputs are subtracted from its desired output, causing an error [an error for each output node]. In the second phase, each of these output errors is passed backward and the weights are fixed. These two phases is continued until the sum of [square of output errors] reaches an acceptable value.

Implementation

The network layers in the figure above are implemented as arrays of structs. The nodes of the layers are implemented as follows:

Compare the resulting output with the desired output for the given input. This is called the error.

Modify the weights for all neurons using the error.

Repeat the process until the error reaches an acceptable value (e.g. error < 1%), which means that the NN was trained successfully, or if we reach a maximum count of iterations, which means that the NN training was not successful.

Comments and Discussions

Hi,
I'm somewhat of a newbie.. I trying to tinker with the code
and upon compiling I'm getting a "System.NullReferenceException"
When I click on browse and try to access a file to recognize.
Why is this?

I tried to use your code to find similar images. I have a database containing a lot of images and I have to decide whether another image is part of the database or not. My intention is to create an application which is able to decide whether an image has been stolen (i.e. copyright infringement) or not. I have a huge database of images (>10,000 images) in different sizes (10x10px - 5,000x5,000px). It is very difficult to remember every image in the database, so I tried to use an ANN. Your code resizes the images to an average size and uses this number to calculate the number of input and hidden nodes. Assuming you have 1,000 images with an average size of 1,000x1,000px, you have about (1,000*1,000 + 1,000) * 0.5 input nodes + (1,000*1,000 + 1,000) * .3 output nodes. Every input node has a double array with connection weights to the hidden nodes. I.e. there are (1,000*1,000 + 1,000) * 0.5 input nodes, each with a double array of (1,000*1,000 + 1,000) * .3 elements. One double has 8 bytes. We need 500500 double arrays with 300300 elements. But the maximum size of a double array is 65536 (in C#)... We would need about 500500*300300*8 bytes (= 1.093 terabytes) of space for this artificial neural network... I think this can't be done...

I tried to set a "maximum average size" of 128*128px but the accuracy of the network died with its waste of space... Do you know a method do increase accuracy and decrease the usage of space?

10000 pattern with an average image size of 128*128px is too big and 1,000*1,000px is crazy. For this case I think pattern count and average image size should be decreased or feature extraction methods be used.

Actually it means if the current output is equal to mapped output then assign 1 to current output's target.
About the number of hidden & input neurons, there is no specific rule to decide. But a general form is that; the more complex input data requires the more number of neurons on the layers..

This is enlightenment!!!
Thanks for the wonderful article
Regarding fingerprint matching, is it enough that we just have the, say, 100 fingerprint image from our neighborhood, and do just what your program does? pattern matching, without extracting the properties of finger print itself?

I read a little about properties of fingerprint, do we have to extract those attribute, and train the network using those features?

yes, I think so. But I'll try with your image detection first. After that, I'll try with the feature extraction. Actually, I am very new to this field. So, if you had any references that you think will help me, I'll be very thankfull

First, it is a great aplication
Well, now I have a question:
is possible to get generalization (recognize similar patterns)?, I try with BP simple layer, but I can't get good results.
or
Is possible with the others (2 layers or 3 layers)?

thanks.why method ToImage for grey level never been used?
and
at method initializeNetwork where computing weight with formula:
SeMasukanLayer[i].Weights[j] = 0.01 + ((double)rand.Next(0, 2) / 100);?
why use random number between 0.00 - 0.02
and at layer 2 and layer 3 use random number between 0.00 - 0.05?

we are extremely thankful for this damn awesome project.it wll bw very kind of you if u could please asist me to know
"OutputLayer[k++].Value = p.Key;"
1)what is the initial value of output layer???
2}what is the meaning of "p.key" and "p.value"..plese elaborate.

Sorry for late reply. That code is to map each pattern id to [each] output at the initialization step. "OutputLayer[i].Value" holds the id of pattern, however, p is KeyValuePair type and p.Key holds the pattern name (or id) that will be mapped to the output.

p.value holds the matrix of the image and it is not used at the initialization phase.