Firstly the program has to register the TaskbarButtonCreated message. This will create a taskbar button for this application. Without this it is impossible to use the new Windows 7 features.

After having the taskbar button created the application has to initialize an ITaskbarList3 or ITaskbarList4 interface in order to be able to access the new features.

In our event filter:

bool MyClass::eventFilter(void *message_, long *result)
{
static unsigned int taskBarCreatedId = WM_NULL;
MSG* message = static_cast(message_);
if (taskBarCreatedId == WM_NULL) {
taskBarCreatedId = RegisterWindowMessage(L"TaskbarButtonCreated");
return false;
}
if (message->message == taskBarCreatedId &&
message->hwnd == parent->winId())
//very important to check to which window this message is address
//since it is possible to get a dozen of them
//the parent can be a QMainWindow or any other QWidget which acts as a
//tab container
{
//init the ITaskbarList3 interface
//announce that the interface is ready
return true;
}
//...
}

Adding tabs

The most important ITaskbarList3 functions to work with in this case are: RegisterTab, SetTabActive, SetTabOrder and UnregisterTab.

You should not register the window or widget which contains these tabs as a tab.

First thing to do is to capture all messages sent to the application. This can be achieved by substituting the current application's event filter with ours.
This will route all messages to our own function.

Before registering a widget as a tab it is necessary to set two attributes to the window handle. This can be achieved by setting the DWMWA_FORCE_ICONIC_REPRESENTATION and DWMWA_HAS_ICONIC_BITMAP to true:

This code will make the application receive WM_DWMSENDICONICTHUMBNAIL and WM_DWMSENDICONICLIVEPREVIEWBITMAP messages when a thumbnail is requested for the registered widget.

Another important thing which is related to Qt, you SHOULD NOT REGISTER AS A TAB A WIDGET WHICH IS BEING USED IN THE MAINWINDOW OR DIALOG.
This will not work.
The solution is to create a new blank widget, register it, and map the real widget to this blank widget. The thumbnails in the taskbar will be provided by the real widgets but the messages will be addressed to the blank ones.

So each time when a WM_DWMSENDICONICTHUMBNAIL or WM_DWMSENDICONICLIVEPREVIEWBITMAP message is received, it is addressed to the blank registered widget, not the real one.

The steps:
1) Create a new QWidget*
2) Map this created QWidget to the real QWidget is being used. This is apt to you.
3) Set iconic preview enabled to the created QWidget
4) Register the created QWidget as a tab
5) Set tab order
6) Set it as the active tab

In our own event filter we'll receive many messages, the most important ones are:

The message code provided by the RegisterWindowMessage function. When the "TaskbarButtonCreated" message was registered - When this happens our tab manager will initialize the ITaskbarList3 interface.

WM_DWMSENDICONICTHUMBNAIL - from MSDN: "instructs a window to provide a static bitmap to use as a thumbnail representation of that window."

WM_DWMSENDICONICLIVEPREVIEWBITMAP - from MSDN: "Instructs a window to provide a static bitmap to use as a live preview (also known as a Peek preview) of that window."

WM_ACTIVATE - when a thumbnail was clicked.

WM_CLOSE - when a thumbnail is about to be closed.

Providing the static bitmap

When a WM_DWMSENDICONICTHUMBNAIL message is received first thing it is checked if this is addressed to one of our registered widgets. This check can be done by comparing widget->winId() to message->hwnd. If they match we have to provide a static bitmap.
The created bitmap is set by calling the DwmSetIconicThumbnail function

Providing the live preview

Basically is almost the same as in the first case, except that the DwmSetIconicLivePreviewBitmap function is called.

An example:

case WM_DWMSENDICONICTHUMBNAIL:
//check if this is message is addressed to one of our widgets
if (!m_widgetMap.contains(message.hwnd)) return false;
//get the real widget
widget = m_widgetMap[message.hwnd];
QPixmap thumbnail = QPixmap::grabWidget(widget).scaled(size, Qt::KeepAspectRatio);
//QPixmap::Alpha in case the image has transparent regions
HBITMAP hbitmap = thumbnail.toWinHBITMAP(QPixmap::Alpha);
DwmSetIconicThumbnail(id, hbitmap, 0);
DeleteObject(hbitmap);
return true;
case WM_DWMSENDICONICLIVEPREVIEWBITMAP:
//check if this is message is addressed to one of our widgets
if (!m_widgetMap.contains(message.hwnd)) return false;
//we want to grap the main window and show as a live preview
widget = parent;
QPixmap thumbnail = QPixmap::grabWidget(widget).scaled(size, Qt::KeepAspectRatio);
HBITMAP hbitmap = thumbnail.toWinHBITMAP(QPixmap::NoAlpha);
DwmSetIconicLivePreviewBitmap(id, hbitmap, 0);
DeleteObject(hbitmap);
return true;
case WM_ACTIVATE :
if (LOWORD(message->wParam) == WA_ACTIVE) {
//check if this is message is addressed to one of our widgets
if (!m_widgetMap.contains(message->hwnd)) return false;
//get the real widget
widget = m_widgetMap[message->hwnd];
//announce that widget was activated
//..
}
//route message further
return false;
case WM_CLOSE :
//The same as for WM_ACTIVE except ..
//announce that widget is about to be removed
//..
return false;

Updating the tab

When the content of the tab has changed, the thumbnail bitmap will not change by itself. That is why is needed to call DwmInvalidateIconicBitmaps. This will update the taskbar thumbnail for the specified tab.

You need to do the following:
1) Call the DwmSetWindowAttribute function to set window attributes for non-client rendering to DWMWA_FORCE_ICONIC_REPRESENTATION and DWMWA_HAS_ICONIC_BITMAP
2) Catch WM_DWMSENDICONICTHUMBNAIL and WM_DWMSENDICONICLIVEPREVIEWBITMAP Windows events
3) Call DwmSetIconicThumbnail, on WM_DWMSENDICONICTHUMBNAIL event, to set thumbnail bitmap
4) Call DwmSetIconicLivePreviewBitmap, on DWMSENDICONICLIVEPREVIEWBITMAP event, to set the window live preview bitmap

In this tutorial I'll explain how to create a Windows 7 thumbnail toolbar using Qt with MINGW GCC. At the end of the post you'll find the link to the project's code.

Missing definitions and declarations

MINGW GCC doesn't have all needed definitions and declarations to create a windows 7 thumbnail toolbar. That is why they need to be defined. I've explained how I did it in my previous post. You can browse the definitions file here.

Project settings

It is needed to add extra linkage libraries options to the linker.

LIBS += libcomctl32 libole32

Creating the thumbnail toolbar

In short you need to do the following:
1) Create an instance to the ITaskbarList3 COM interface
2) Create a list of images and add it to the thumbnail toolbar
3) Create a list of thumbnail toolbar buttons and add it to the toolbar
4) Update the buttons or the images when needed

What you can do:
1) Show/hide, disable/enable, change button details, update the image list
2) Have a thumbnail toolbar per program's window

What you can't do:
1) Add more than 7 buttons to the toolbar
2) Remove the toolbar (it will be destroyed when the window associated to it will be destroyed.)

Initialize the ITaskbarList3 COM interface

Firstly you need to implement your own winEvent function. This can be done by overriding the QMainWindow's winEvent protected function.

To access the Taskbar button from Qt with MINGW it is needed to declare the ITaskbarList3 COM interface, define CLSID_TaskbarList and IID_ITaskbarList3. And to declare used structs (THUMBBUTTON) and enums (TBPFLAG, THUMBBUTTONFLAGS, THUMBBUTTONMASK). All this can be done by finding their declarations in the Windows 7 SDK header file ShObjIdl.h, and adapting them for MINGW GCC in a separate header file.
In my previous post I have explained how this can be accomplished.
The result can be viewed here.