Introduction

All my life, I have been possessed with anything that has something to do with robots and computer vision. So lately, I have been looking for some simple code to read images from movies in any format and from the webcam. Here, I want to share my experiences in this API jungle with you. Also, please consider voting using the bar bellow so I get an idea if this information was useful to anyone.

Background

As hard as I was looking, all I found were samples for the VFW interface, which is a practically dead API with a lot of issues. First of all, it's only for AVI files, and most of the codecs (like divx) just don't work in it anyway (those who have tried it knows), and also in some resolutions, on most webcams, it just crashes. There are a few samples for the newer DirectShow API but all the important code is just buried in the zillion unimportant wrappers/support classes/files. Not to mention that the informational value of the samples themselves is lost in often redundant, error handling jungle. And finally, they are based on using/compiling additional MS DirectShow filter samples, like SampleGrabber or its bug fixed version, GrabberSample, making the whole approach non-portable between the SDK versions. They also seem just disappeared from the last SDK and the Web itself. So here is what I think is a much simpler/versatile alternative to those.

New method not requiring DirectShow SDK

Lately I added to article this new method because a lot of people had problems compiling DirectShow SDK related stuff. Most of them just need to get the RGB data and forget the whole DirectShow thing so I got an idea. If we know how data flows inside directshow graph why don't we just redirect particular method to grab data ?
Now in Directshow data can flow either via:

In our case (last filter renderer) we capture data of Push method. you can find sample capturing Pull mode in my other article Fun with Google TTS.
I tested Following code on VisualStudio 2008 and 2010 and it compiled without DirectShow SDK just fine. Also remember that you will probably need to convert received data to RGB if you wish but it's trivial.

Previous method requiring DirectShow SDK:

Starting with DirectShow

Most users who for the first time try to compile any DirectShow sample from web will spend 90% of time searching for the SDK itself (as for now it has been moved to the Platform SDK from the DirectX SDK, so be sure to get latest) and then spend the rest of the day solving the undefined variables (add YOUR_PLATFORMSDK_DIR\Samples\Multimedia\DirectShow\BaseClasses to the compiler include paths) and unresolved externals (add strmiids.lib, winmm.lib, and strmbasd.lib (strmbase.lib for the release build) to the linker input). But the latter two libs are not redistributed by Microsoft and must usually be built manually (just and run vcvars32 and nmake (nmake nodebug=1 for release lib) in the include directory mentioned above and copy there the libs from the subdirectories created by nmake).

Using the code

So here is the sample. Just create an empty Windows C++ application in whatever development environment you have (Disable UNICODE in VC++ or you get linking errors), and copy/paste it there. I stripped it to a bare minimum to run, so please forgive its compressed formatting for the unimportant code parts. And as usual, you can find information on any used API simply by searching for its name in Google and using the first found (usually MSDN) page.

Points of interest

The most important lines are in DoRenderSample since there you get your image in RGB format in every frame. The sample itself tries to find and connects to the first found pin on the first found video capturing device. And if none is found, then it tries to open and run a video file from the provided path. So in fact, these are two tutorials in one. The cool side-feature is that you can use the Alt+PrintScreen key to grab a screenshot from (any?) movie/ webcam and paste/save it to your favorite image editor.

Place for enhancements

Consider using some form of smart pointers like CComPtr<IGRAPHBUILDER> instead of raw pointers to interfaces like IGraphBuilder*. I removed them to keep the sample buildable by people who don't have the ATL libs. Example code to have more control over the video playback (auto-rewind in this case).

Some valuable tips from the Forum users.

Unresolved externals: Make sure you did build libs mentioned in article.
VS6: Get Platform SDK and don't forget to set standard SDK include / lib paths
VS2005: unresolved externals
Go to "Configuration Properties" -> "General" -> "Character Set"
By default, it is set to "Use Unicode Character Set". Change it to "Not Set" and the project should link.

So enjoy it, and don't forget that most of the advanced error handling etc. is up to you to implement, and this is just very basic sample so you can focus on how it works.

Thanks for clarifying how this sh*t works ... most of the online documentation (especially the "official" documentation) is fragmented, mindlessly recursive, and useless. Excellent examples, I really like the minimalist appraoch.

Also huge thanks for putting in print how to get the mysterious DirectShow SDK functional

hey Ladislav. I have been tried VFW capture, but it dosen't support my webcam driver. and in directshow I have trouble when i want to save webcam data to an avi file. I tried your first sample but I can't save data to disk. Would you please show me how can I save it to disk.

hello.. i'm trying to use your code(New method not requiring DirectShow SDK) by copying it to visual studio 2005(i started using win32 project) . but when i run it, it has so many errors.. can you help me to solve this problem?

Thank you very much for great example! Tons of Microsoft examples can be deleted forever!
I need you advise now. If I try to display video over 2 monitors (in "Extended Desktop" mode under XP) I can see video only on the first monitor and the second one displays just black window. I've found that this problem is related to DirectShow, and again there is no clear solution from MS and lots of redundant and strange examples in the net... What would be the best way to solve this problem?
Thanks a lot

Thanks for the great example!
However, I have a small question about getting frame data.
I found that it getting quite slow if I copy buf to a array.
even I use memcpy, it still affect the preview performance a lot.
It may because by the manipulations to buf.
But It works great if I write the value to buf. writting value to buf won't affect the performance.

Reading is slow because the code is taking the frame bits from the Frame Renderer, that buffer is on the GPU.

In order to make it work faster, you need to connect to an earlier filter with a buffer in the RAM.

However, earlier filters sometimes are in wierd formats so iterating backwards on the graph might be difficult.

My solution was adding a pass through filter (in my case, DMO color control) to the graph, disconnecting the old connection between the Frame Renderer and the filter before it. Finally, I connected the new filter to the pins I just freed.

On my machine the new filter flipped the image vertically, so I flipped it back.