Posted
by
timothy
on Monday October 12, 2009 @03:37AM
from the don't-tell-hal-about-this dept.

sevarac writes "The latest release of Java Neural Network Framework Neuroph v2.3 comes with ready-to-use image recognition support. It provides GUI tool for training multi layer perceptrons for image recognition, and an easy-to-use API to deploy these neural networks in end-user applications. This image recognition approach has been successfully used by DotA AutoScript tool, and now it is released as open source. With this Java library basic image recognition can be performed in just few lines of code. The developers have published a howto article and an online demo which can be used to train image recognition neural networks online."

Neural nets aren't a good solution for all problems (nothing is, of course).

Furthermore you need plenty of training data partitioned into training and test sets so you can evaluate the generalisation error. If you don't do that you can have absolutely no confidence about the performance of your system.

Working with neural nets has a mostly experimental feel to them (I am a final year AI student). I'm not sure application developers have the discipled methodology to use them correctly (though maybe this framework helps with that) as these behave quite differently from standard computer vision recognition techniques.

Furthermore you need plenty of training data partitioned into training and test sets so you can evaluate the generalisation error. If you don't do that you can have absolutely no confidence about the performance of your system.

Furthermore you need plenty of training data partitioned into training and test sets so you can evaluate the generalisation error.

Yes. What's the alternative?

Working with neural nets has a mostly experimental feel to them

Most of software development is "experimental" because most software works with real-world data and because you have an endless cycle of implementing new features, testing, bug reporting, and bug fixing. With neural networks (and similar methods), at least people quantify their error rate

wouldnt it be cool if there was a neural net search engine, so rather than everyone training up their own individual image recognition bots in isolation they cook work together in a supernet when training but still be disconnected when in use. Prior art would be the matrix: "I know Kung Fu!"...

Is there a FORTRAN port or a COBOL port? Java is a perfectly good language but it seems that there are a group of Slashdotters that automatically dismiss anything written in Java. Generally their views seem to be based on false rumour put about when Java first got off the ground.

For the record I have been coding for the last 30 years in just about every major language (with the notable exception of COBOL) on every platform from Mainframes to embedded. I still use C/C++ when I have to but default to Java wh

In Java everything is passed by VALUE (yes, that's not a mistake) - both primitive types and references.When you pass the 'obj' variable from the example above as a method parameter,you actually pass the reference (or pointer in other words) by value.

My issues with Java (and especially with the containers layered on top like Tomcat, J2EE etc) is simply resource usage. It can use up a lot of memory just to put a tiny little java app on a server. VMWare found this out when they rewrte their management utility using Tomcat, a lot of users started to complain that it took up 500mb, which was enough to run another guest image!

My other issue is the 100% pure java nonsense. Anyone that tries to tell me what languages I should code in gets that language added t

My issues with Java (and especially with the containers layered on top like Tomcat, J2EE etc) is simply resource usage. It can use up a lot of memory just to put a tiny little java app on a server. VMWare found this out when they rewrte their management utility using Tomcat, a lot of users started to complain that it took up 500mb, which was enough to run another guest image!

What has a neural network library to do with a web server like tomcat? So you want to use/run a C/C++ Web Server and link a C++ neural

Java is a perfectly good language but it seems that there are a group of Slashdotters that automatically dismiss anything written in Java

Have you ever tried going to the limit in optimizing code? An SSE2 machine can do a multiply-and-sum operation on an array of four floating point numbers in one clock cycle. Multiply-and-sum is, basically, what's needed in an artificial neuron network. Get your best effort in java programming and count the cycles. I have seen papers showing an improvement of about 200 thou

meh, not sure about that. You keep forgetting that you don't have to write the whole program in Java, use the JNI and you could implement the parts that really need to be fast in C/C++... Java has some obvious advantages when it comes to portability, out-of-the-box supported libs, database connectivity, even some GUI, which might leave impression that it should be used everywhere. As you pointed out, it doesn't have to, but also the reciprocal is true, one shouldn't dismiss it just because it doesn't do wel

Java is a perfectly good language but it seems that there are a group of Slashdotters that automatically dismiss anything written in Java. Generally their views seem to be based on false rumour put about when Java first got off the ground.

In all fairness, when Java first got off the ground, those rumors weren't false!

But yes, most all of the issues with speed and JiTC have not been problems for some time now, and the compatibility issue, while still exists, has been reduced to "Works with Suns JVM" or "Works with MS JVM" etc, and are now just 'annoying' implementation details instead of actual 'problems'.

Personally, I avoid Java apps because for run anywhere they sure are awfully picky about how anywhere is set up. I find that they rarely want to run right and when they do in spite of claims that the resource hogging is a myth, I notice that the disk spins and spins while it's loading. Meanwhile, C, python, Perl, and FORTRAN just work.

Meanwhile, C libs are easy to use in nearly anything else. It's easier to call Python from C than it is to call Java from anything else.

You remind me of Cato the Elder [wikipedia.org] who had the habit of ending all his speeches with the words Furthermore, I think Carthage must be destroyed [wikipedia.org]. I think he was right, and I think you're right, too. But that's slashdot: What earned the old Roman eternal reputation gives you only a "-1, Troll". Hang on, though.

Furthermore, I think that the CO2 level in the atmosphere must be reduced.

To open source thesis work, you will have to get approval from the university your are working at. So I think it probably got open-sourced later then it was developed at least. Often (at my faculty at least) implementations are protected aggressively, to keep other people from getting the same performance.

To open source thesis work, you will have to get approval from the university your are working at. So I think it probably got open-sourced later then it was developed at least. Often (at my faculty at least) implementations are protected aggressively, to keep other people from getting the same performance.

Hi, you're right when you say the Neuroph has always been open source, but the image recognition used by http://dota-autoscript.com/ [dota-autoscript.com] has not been open source. With this release of Neuroph we added the image recognition support based on ideas and original code from http://dota-autoscript.com./ [dota-autoscript.com.]

This is hardly news-worthy. Having looked at their web-site and having done object recognition myself I can tell you, that this wont work for several reasons. From their FAQ it seems that they are just taking the image and create very long vector from it and then feed that into an ANN.

The following problems exist with this approach (which has been known and addressed in at least the last 20 years):1. What if you want to classify pictures that have different sizes (not too uncommon)? Wont work because you first have to set a fixed number of neurons in your first layer.2. What about different locations of the same object? Even if you move the whole image just one image to the right this will dramatically change your flattened vector and can yield completely different classification. This is called location invariance and is solved in the computer vision community by extracting a lot of features on "interesting" regions in the image. With that you can also have different sized images. One famous extraction method is the scale invariant feature transform (SIFT [wikipedia.org]).

I could go on but you get the point. As for using ANNs instead of other classification methods, it does not really matter that much what kind of classificator you use as the pre-processing is much more important. Support-Vector-Machines has been hyped and work pretty well, but it has it's own caveats. However, using a SVM on the raw-image data produces the same horrible results that I would expect here.

There's also the brute-force approach of sampling sub-images from the image at a ton of different sizes and locations and then feeding the sub-images to your classifier. And for controlled images (such as in face recognition) normalizing the image isn't too difficult (by scaling the image to normalize the distance between the eyes). Even then, dimension reduction by PCA and/or LDA is useful to reduce the needed complexity of the classifier.

However, using a SVM on the raw-image data produces the same horrible results that I would expect here.

SVMs would probably work a lot better. SVMs perform excellent in high dimensional datasets.

Simple feed forward ANNs screws up – if you have a large amount of input values (e.g. treating the whole image as a vector) there will be a lot of weights on the input layer. So I really don’t know why you would use a normal ANN.

Yes, they seem to sample the image as a grid, and feed it straight into
the neural network. Its a really dumb way to do image recognition. However
a neural network library is useful for many identifcation tasks, and its free.
Even if its going to be useless for image recognition, it may be useful
for how general ai tasks.

Unless the water gun has an extremely high muzzle velocity, she would need to lead the target bugs by a non-trivial angle. The flightpath prediction algorithms required for reliable hits would probably be prohibitively complex since they would have to take species, sex and several environmental factors into account (and math is HARD).

She might get better results adapting this project [slashdot.org] to her purposes. Please invest in a high-powered laser for the pr

Unless the water gun has an extremely high muzzle velocity, she would need to lead the target bugs by a non-trivial angle. The flightpath prediction algorithms required for reliable hits would probably be prohibitively complex since they would have to take species, sex and several environmental factors into account (and math is HARD).

Great analogies to cluster computing design issues. Really expensive perfect bullets, or lots of cheap bullets... I'd try the lots of cheap bullets approach since water is cheap. I'd hook up the garden hose thru one of those lawn sprinkler solenoid valves, attach hose to RC car servos, and work the software for two things:

1) Image detection before and after water spray. If you can't tell if you've hit anything, then what you do doesn't matter. That in itself would be an interesting science fair project

Actually the plan is to use a waterpik which has a very high velocity (and won't require any change the the eletronics, just mash in the power button and apply power to the feeds to get it to fire) and short range (not trying to wet people across the room).

Sheesh, I got a '0-off topic' but the replies all got '2 - informative'?

After reading that article about seeing through walls with mesh networks, I wonder if you couldn't implement something like this image recognition system on a cluster. The machines could locate a subject with their "sixth sense" and then point cameras at it to track, and none of the nodes would have to be spectacularly powerful.

If you want to see a neural network (-like) implementation that actually works well, you should look at Numenta's NuPic [numenta.com] application and their vision toolkit. It works pretty amazingly well. They also have a nice architecture, in that the core is c++ but they use python for UI, file IO, and other utility stuff.
It's under a 'research license' which means that you are allowed to do research and play with it, but if you want to sell a product with their technology, you need to get a real license.

Did you actually try it out yourself? I bought into Numenta's hype and downloaded this program [numenta.com], and it was laughably bad. It "learns" a list of objects and shows it can recognize them, but when I modified the test images just a little bit and fed them to it, it failed miserably. That's especially bad considering how low res the images are and how they're the same size.

Numenta is only different in having a disprportionately large marketing budget. (Yes, I took the plunge and read On Intelligence.)

probably not any better than anything else...this is not a new technology, it's just an open implementation to save people the time coding. Although, once you read the right literature, a neural net like this isn't really rocket science to implement in the first place, so really it's just a time saver/crutch for students.

Well that was my first thought too, but then we may be past that point already.
A few days ago I read about a Firefox extension someone has sent a cease-and-desist over called Skipscreen which lets you avoid all those pesky waits when using Rapidshare type sites.
Now of course I never use these sites myself and certainly not to download anything illegal, but thought I'd install it for the convenience of anybody else who happened to use use my computer, I mean, who knows what my cat gets up to in the midd

Back in 1987 I was working in the Multimission Image Processing Lab at JPL/Nasa, and the whole Neural Network fad was in its second resurgence, after the original "perceptron" work of Minsky and Papert caught some attention in the early 70's. The group that I was in [nasa.gov] was mostly interesting in mapping, and in particular identifying features (e.g. soil types, vegetation, etc) from both spatial and spectral signatures. As usual, the military was also interested in this, and so Darpa was throwing a lot of loose money around, some of which fell our way.

I spent a lot of time on this project, writing a lot of neural net simulations, supervised and unsupervised learning, back-prop, Hopfield nets, reproducing a lot of Terry Sejnowski's and Grossman's work, taking trips over to Caltech to see what David Van Essen and his crew were doing with their analysis of the visual cortex of Monkey brains, trying to understand how "wetware" neural nets can so quickly identify features in a visual field, resolve 3D information from left-right pairs, and the like. For the most part, all of the neural net models were really programmable dynamical systems, and the big trick was to find ways (steepest descent, simulated annealing, lagrangian analysis) of computing a set of synaptic parameters whose response minimizes an error function. That, and figuring out the "right" error function and training sets (assuming you are doing supervised learning).

Bottom line was, not much came of all this, beyond a few research grants and published papers. The one thing that we do know now is, real biological neural networks do not learn by backward-error propagation. If they did, the learning would be so slow that we would all still be no smarter than trilobites if that. Most learning does appear to be "connectionist" and is stored in the synaptic connections between nodes, and that those connections are strengthened when the nodes that they connect often fire simultaneously. There is some evidence now of "grandmother cells" which are hypothetical cells that fire when, e.g. your grandmother comes into the room. But other than that, most of the real magic of biological vision appears to be in the pre-processing stages of the retinal signals, which are hardcoded layer upon layer of edge-detectors, color mapping, and some really amazing slicing, dicing and discrete FFT transforms of the orginal data into small enough and simple enough pieces that the cognitive part of the brain can make sense of the information.

It's pretty easy to train a small back-prop net to "spot" a silhouette of a cat and distinguish it from a triangle and a doughnut. It is not so easy to write a network simulation that can pick out a small cat in a busy urban scene, curled up beneath a dumpster, batting at a mouse....

Yes, there's more than back-prop for succesfull neural processing. One of the most instereseting approaches IMO were done by Liaw and Berger [ieee.org] on adaptable sinaptic training models, but that was more than ten years ago!

"Bottom line was, not much came of all this, beyond a few research grants and published papers."

I am constantly reading about successful applications of neural networks and svm's in medical, industrial control and financial applications. Why do you say not much came of it, are you referring to vision only?

I was referring to our own work on using neural networks for pattern recognition in images.

The research we were doing was in fact prompted by the well-documented success of neural networks in other nonlinear problems. One of the very first good examples of an applied adaptive neural network was in the standard modem of the time, which used a very small neural network to optimize the equalizer settings on each end.

Neural nets appear to have a lot more success with constructing nonlinear maps from su