I've implemented a neural network (using the back-propagation algorithm) in a digital marketing application, where the algorithm classifies words according to quality, price and another characteristics in an AdWord Campaign. I would like to know if there is a paper or book (preferably free) that discusses the accuracy rate of acceptance that a neural network should have.

$\begingroup$I think this question would be a better fit for CS.SE since I am not sure that you are asking a research level question in theoretical computer science.$\endgroup$
– Artem Kaznatcheev♦Jun 4 '12 at 22:22

$\begingroup$Not clear. Maybe it's an open area of research ?$\endgroup$
– Suresh VenkatJun 29 '12 at 6:14

2 Answers
2

In principle you could train a neural network (a multi-layer perceptron, I assume) to perfectly classify the training set (provided that it doesn't contain conflicting examples), however doing so might require a long time (exponential in the size of the training set) and more importantly will usually impair the neural network ability to generalise to unseen data points.

In practice you usually divide your training examples in a training set proper and a validation set: train on the training set and after each epoch you measure accuracy on the validation set, repeat until the accuracy on the validation doesn't increase for a small number of epochs and keep the network with the best accuracy on the validation set. You might also reiterate the whole training process a few times starting from random weights each time, in order to avoid being trapped in local optima.
Moreover, you can experiment varying parameters such as the number of hidden layers, the number of nodes in each hidden layer, and the learning rate, changing the data encoding, and carefully selecting the input features.

Eventually your neural network will reach a maximum accuracy which depends on the problem difficulty.

A baseline against which the performance of a classifier can be compared is a naive classifier that only exploits obvious biases in the data. The idea is that if your classifier doesn't do much better than a random biased guess, it is not a very good one. Or it might be that the data doesn't allow you to extract much information.

In general, the answer as to what is a good test performance is problem and application dependent. A useful approach is therefore to apply one's classifiers and algorithms to well-known problems and benchmarks, so that it's possible to compare one's performance to the current state-of-the-art. An example of such a standard benchmark is for instance the MNIST data base of handwritten digits.