But this forced me to switch to vivtc, and after some conditionalDeintStillCombed magic I got similar/better results.

Can you put all the scripts and a sample script and the plugins into an archive? That way I can test with the same versions as you and don't need to go on a treasure hunt. And reveal your avisynth version too while you're at it. I'd really like to get to the bottom of this...

same error in vspipe and vs editor, only that the mmx line is only shown in the vs editor. Tested with colorbars() + tfm(mode=6)

At least we're making progress. It happens because of a bug in one of the avisynth plugins. Comment out the script gradually and locate which one it is. Someone forgot to use the emms instruction. Find the plugin author and shame them, or update to the latest version of whatever plugin it is.

EDIT: currently an avs error msg is not show like in virtualdub (or any other tool). It would be nice to see the actual avs error rather than "vapoursynth.Error: Avisynth 32-bit proxy: command failed". It's a nice to have feature (and if it isn't too hard to implement)