Integrating the FlyCapture SDK for use with OpenCV

A recent stab at grabbing images from the Flea2 camera using APIs from the FlyCapture2 SDK by Point Gray Research (PGR). Additionally, the camera was to be used in “Format 7 mode”, so that we may grab partial regions of the complete image.

This application builds on the AsynchTriggerEx C++ example provided by PGR, so that the user can grab images from the camera either by way of software triggers, or from an external hardware triggers.

Using Format7 mode to grab partial images

The Format7ImageSettings data structure is modified in order to set the desired output image size, pixel formats etc. Standard FlyCapture API calls are used to perform all necessary camera actions: connect to camera, disconnect from camera, start image capturing, retrieve image buffers etc. When the using the default equipment setup, the input images obtained are grayscales of 24 x 480 pixels consisting of an array of dark gray spots against a lighter shade of gray background.

Converting FlyCapture images into OpenCV images

In the image grabbing loop, FlyCapture2-obtained images are converted into OpenCV-compatible images by copying the pixel data from one to the other:

memcpy( img->imageData,
image.GetData(),
data_size );

Thresholding the grayscale image into binary (black & white)

The OpenCV grayscale image is then thresholded to produce a black and white (binary) image, from which we can use for things like detecting the number and location of the spots. Depending on the equipment used and lighting arrangement, some experimentation is probably needed to come up with an optimal threshold value.

Related Posts

About The Author

Latest Comments

Pkout23 February 2012

Does this work for you when you try more complicated opencv functions such as camshift() or getting image first moment? I am successfully getting the frames and converting them to opencv format. I can even use cvShowImage() to show what the camera sees, but as soon as I try more advanced functions like the ones mentioned above, the program crashes without any valuable error message. I am developing in c++.

Hi Pkout, I have never tried the more advanced stuff like camshift, so I’m not sure what the cause of your crash is. Would it help if you would try a bare minimum prototype that uses this camshift function and see if it still crashes then? It might be a more fundamental issue. I remember having to overcome all kinds of gotchas when trying out this opencv stuff for the first time. If you could point people towards some code that might help.

Thanks for the response! I know the problem isn’t in my implementation of OpenCV because the same code works fine when I either load the video from a file (even if the file is an uncompressed RAW8 format video from FlyCapture) or when I connect a webcam and, therefore, avoid using the flycapture-to-opencv memcpy() conversion. It starts crashing only when I implement the memcpy() on the flycapture image object. Weird. I suspect its something silly, like incorrect matrix data alignment or something of that sort. But I have no idea how to fix it :(. Thanks!

If in Microsoft Visual Studio, you start by creating an empty
project and paste in the code I have given then yes, the compiler in all probability will complain along the lines of ‘fatal error C1083: Cannot open include file: ‘stdafx.h’: No such file or directory’. This is because it being an empty project with nothing in or no settings will need to know where ‘stdafx.h’ lives. Sorry about that.

In your Project properties, go to C/C++ -> General -> Additional Include Directories, and tell it where this file lives. In my VS 2010, this was located in:

C:\Program Files (x86)\Microsoft Visual Studio 10.0\VC\atlmfc\src\mfc

It is probably different for yours. Once this is set and you re-compile, you might then get the following error message :

Instead of using an empty project, in your Visual Studio create a new console project, while making sure to leave the ‘Precompiled Header’ checkbox in the
checked (with tick) state, and you should not get any of these set up / compilation problems.

To do this In Visual Studio, create a new project, choosing ‘Win32 Console Applicatio’n among the list of available options. Once this problem is out of the way, it is then a matter of setting up your FlyCapture / OpenCV settings. This I originally found to be a little trickier! Let me know how you get on.

in your code? It might be a good idea to strip out all the other code, and start with an empty main and any #includes / namespaces you need. I haven’t visited this area for a while, but I will re-examine it when I get the chance. If I see any new problems or get any more insights I will update this post accordingly.

In general, a ‘Compiler Error C2871’ will occur when you pass an identifier that is not a namespace to a using directive:

The problem I had was in choosing infeasible height/width settings. I found that it worked for (say) 24 x 480 or 56 x 482 pixels but not 52 x 484 or 24 x 483 – because the height had to be a multiple of 8.

It took me ages to realize that this was the problem. I guess Point Grey have documented this somewhere, but I had overlooked it. I’m not sure if there are any other considerations for when using the chameleon range. It may also be worth Googling ‘Format 7 error FlyCapture’ or similar.

Hi Marcus
I think this may be an issue with certain code symbols not being displayed correctly.
I think an ampersand (‘&’) is missing in various sections of the code. Can you please try placing a ‘&’ behind regVal, as in:

Hi Andy,
Thank you very much for your instructions.
I was a lit bit confused, because I had FlyCap for x64 and x86 at same computer.
Maybe I was charge for dll x86 and lib x64, or vice versa
Thank you very much, because the program works very well now..

This has been extremely useful for a project I am working on involving a 3D scanner with openCV and a Point Grey camera.

The program takes a live stream from the camera, displays it and gives the user more options for calibration/scanning/ect. Before we got the point grey camera, we had the code set up to use the camera as a webcam (for testing phases) since openCV plays very well with them (using cvcapturefromCAM()). Point grey has directshow dlls which allows openCV to *attempt* to use the ptgrey camera, but my program crashes and it seems that many people have issues upon camera connection.

My question is:

Have you been able to use a live stream from the point grey camera to openCV? My code is entirely set up to use a livestream from the camera and it would take a huge amount of work the change everything to image by image capture.

One thought I have had is to use Flycapture2 to continuously capture to an AVI file and have openCV use capturefromAVI(), but simultaneously reading and writing to this file has proved somewhat difficult.

Also, I am using Win7 x64, VS2010 x86, FlyCapture2/camera driver x86, and openCV 2.4.4 and compile everything in 32bit. I have also tried every camera driver(x64 and x86) that point grey has to offer for my camera(FireFlyMV) with no luck.

Hi Max
Everything I’ve done so far with FlyCapture / OpenCV has been on an image-by-image basis. I’ve been lucky in that my images were also quite simple – rows of inkjet printed spots, albeit quite tiny in the order of microns.

My only advice is to keep experimenting to find out what works reliably and what doesn’t. I agree that some aspects of FlyCapture and OpenCV are more flakey than others.

Maybe it would be beneficial if (one day) I could experiment with using livestreams and share my experiences on this blog, but don’t hold your breath just yet. So much to do, so little time…

I created a new Wings console project and copied the copied code along with including the FlyCapture SDK and OpenCV libraries inthe project. However, on running the executable,windows just crashes and restarts. I am not able to figure out the problem for it.

I implemented your code. When I run it, it freezes at retrieveBuffer function. Then, I close the program manually. Then, I run again and close it manually again. After two freezes, it runs successfully. How can I solve that problem?