Capturing window in linux using Gstreamer for OpenCV

One of the projects that I am working “CloudBank” required that I somehow capture the frames from a video game that was displayed in a separate window on the screen. I needed to do this so that I could analyse and process the frames using OpenCV. A few searches on google shows that this is one of the most frustrating things to accomplish so in this post, I am going to show you how I used the media streaming framework GStreamer to accomplish this. By using GStreamer I was able to accomplish the following:

Capture the video game frame that is being played in a separate window and then save that frame into a file of any format that can then be analyzed and processed by an OpenCV program.

Capture the video game frame in a way that DOES NOT require the window to be placed in any specific position of the screen.

xname: The name of the window you want to capture. (IMPORTANT NOTE: This will capture any window that possesses the name of the window you want to capture. Ensure that your desired window is THE ONLY ONE with that name on your screen )

Running the cmd

On Linux, you can run a command in a variety of different ways. In my project, I run the Gstreamer cmd (written within a .sh file) within my C++ using the function “system” and pass in the cmd to run the script as the argument. For example to run the script/file that holds the GStreamer cmd mentioned before I used:

system("<path_to_file> &");

the “&” symbol runs the command in a terminal in the background.

The image below shows an example of the game “Transistor” that is being run on a Steam and a separate video output produced by GStreamer. each frame is saved into a separate file at a location that I have specified.