It is statically built so after compiling you end up with a single exe that you can run without the need of DLLs.
I have just tested this on OS X 10.9, Qt Creator 3.0.1 and the application was tested on Windows XP SP3 inside Parallels.

A quick tutorial:

If you download it (gcc-4.8.0-qt-4.8.4-for-mingw32.dmg) you will have a DMG file, inside a PKG which you can install, it will install MinGW and Qt 4.8.4 on this directory:
/usr/local/gcc-4.8.0-qt-4.8.4-for-mingw32/