In the recent Kickstarter updates, we've been discussing various options for running the machine vision system on Spirit. I built the prototype using SimpleCV, which works, but it's certainly not ideal. Much better solutions exist but as I'm not much of a software guy, I'm hoping for some help from the rest of you.

A few options I've had a quick look at:

OpenCV: This appears to be the go-to standard for this kind of work, though at first glance, it looks complicated.

OpenMV: This looks like a possibly great solution, but I haven't dug into it yet.https://openmv.io/

We would really like a solution that runs on Python 2.7. As far as I can tell, the latest OpenCV only runs on Python 3.0, but we have a few other things running on the robot requiring 2.7. I tried to move those to 3.0 a while back and it was a huge mess so if we can stick with 2.7 for the vision, so much the better.

The second big requirement is that it can be made easy. All of these bits of software appear quite complex, but if I can understand it well enough and I can write some simple functions that are "user friendly for the masses" then it'll work.

The production hardware will be a Raspberry Pi B3 with the original Rev1 Raspberry Pi camera (the 5MP design), though we will eventually move to the new Rev2 (8MP) Pi Camera, so if we could create a vision solution that is easy to swap between cameras, this would also be a huge plus. Eventually we will need to run it on the Rev2 8MP camera one way or the other as the Rev01 cams are harder to find these days.

So I'll open it up for discussion below. If you have a Pi and a camera module, and you have more advanced understanding of how all this stuff works, we would really appreciate your help. Share your thoughts on the different options, discuss between yourselves, play with some demo code, etc.

In the recent Kickstarter updates, we've been discussing various options for running the machine vision system on Spirit. I built the prototype using SimpleCV, which works, but it's certainly not ideal. Much better solutions exist but as I'm not much of a software guy, I'm hoping for some help from the rest of you.

A few options I've had a quick look at:

OpenCV: This appears to be the go-to standard for this kind of work, though at first glance, it looks complicated.

OpenMV: This looks like a possibly great solution, but I haven't dug into it yet.https://openmv.io/

We would really like a solution that runs on Python 2.7. As far as I can tell, the latest OpenCV only runs on Python 3.0, but we have a few other things running on the robot requiring 2.7. I tried to move those to 3.0 a while back and it was a huge mess so if we can stick with 2.7 for the vision, so much the better.

Kevin,

You can stick with Python 2.7 and push off a code overhaul (not sure how massive... would need to see what you're working with there) if you allow your product to use an earlier version of OpenCV.

Can you get away with running OpenCV 2.4? There are a bunch of openCV 2.4 components and python-opencv available via the Debian package manager under Raspbian...

So it's the latest version of Raspbian Jessie (as of the writing of this post).

With all the above installed (and any required dependencies that the package manager determines), I can get open CV to work using the tutorials found here for both stills and video using the RasPi camera:

Kevin wrote:The second big requirement is that it can be made easy. All of these bits of software appear quite complex, but if I can understand it well enough and I can write some simple functions that are "user friendly for the masses" then it'll work.

Computer vision isn't simple by its very nature. It requires some knowledge of vector and matrix mathematics (linear algebra) for it all to make sense. We should think about tutorials that get people thinking in terms of vectors/rays in order for the applications to make sense. Most of the OpenCV tutorials are pretty good, but they're a bit hodge-podge in terms of the path you want to take in terms of how to use the toolkit, because it's just that... a toolkit that has differnet tools that may or may not be used together.

Since your camera is on a pan/tilt head, a really simple application would be to track a blob of a specific color, and then command the pan/tilt head to track it. If you want the robot to follow it, have the robot close a control loop on "nulling" the pan channel error to zero by turning (relative to pointing "straight ahead" in the robot body space), regardless of the tilt angle. That would be a great exercise for the users to do, actually. That way you could have the robot "chase" someone holding a blue ball, or something like that. You could have it close a speed loop on tilt angle relative to horizontal such that looking horizontal would be full blast speed, and it would be throttled back to zero as the tilt angle increased away from the horizontal angle in the robot body space, all the while commanding the pan/tilt head to put the blob in the center of the camera field of view.

As far as making things simple, I'm not trying to teach the underlying theory of computer vision (at least not to start). I'm looking to create some simple python functions that do simple things, like find the biggest blue blob and return its center coordinates. That's what I'm doing with SimpleCV right now, though it's slow and difficult to tune.

I'll check into the links. The book looks like a good reference as well.

When writing code I always ask myself if I can make the functions simple enough that a 3rd grader can understand and use them. I think we'll loose a lot of people if we jump straight into vectors.

Does OpenCV 3.0 have significant benefits over the previous versions?

Have you guys looked at OpenMV? I'm all about using whatever software is most easy for a novice and some of the other options out there seem to advertise great performance and ease of use.