The Problem

Recently I had to package up a wxPython and VTK-based app for standalone deployment on Windows. Because of great experience with PyInstaller, I opted to use this tool.

With the first try with the freshly built package on the deployment machine, it refused to start up due to an ImportError: DLL load failed: The specified module could not be found., and specifically with the vtk.vtkCommonCorePython.pyd Python extension DLL.

What was frustrating, is that the relevant file was definitely present and in the right place, namely the same folder as the exe file.

A test app that imports only wx(4.0.0rc) and vtk(8.0.1) generated the following traceback:

However, using the same invocation on the offending PYD would still raise an error. By the way, I had to disable PyInstaller’s silly exceptions, because they masked the underlying OSError exception.

Great deal of good that did me, because Windows 10 gets me no additional information other than reporting “error 126” on the top-level DLL. Why can’t such a mature system not offer a little more guidance?

(Pdb) ctypes.WinDLL("vtk.vtkCommonCorePython.pyd")
>>> OSError: [WinError 126] The specified module could not be found
(Pdb) ctypes.WinDLL("C:\\Users\\cpbotha\\Downloads\\t1\\vtk.vtkCommonCorePython.pyd")
>>> OSError: [WinError 126] The specified module could not be found
(Pdb)import os
(Pdb) os.path.exists("C:\\Users\\cpbotha\\Downloads\\t1\\vtk.vtkCommonCorePython.pyd")True(Pdb) ctypes.WinDLL("vtkCommonCorePython.pyd")
>>> OSError: [WinError 126] The specified module could not be found
(Pdb) os.path.exists("vtkCommonCorePython.pyd")True

I did spend more time than I should have in pdb tracing through the whole complicated PyInstaller and Python import code innards. The fact that some PYDs loaded and some did not should have more quickly pushed me in the direction of nested dependencies.

Seeing the light with lucasg’s Dependencies

Although I had at an earlier stage checked DLL loading first with Dependency Walker (this gets very confused by the new Windows api-ms-win-* DLLs) and later with lucasg’s improved utility, I did not drill down far enough into the dependency tree.

It’s important to keep on drilling down until you see missing DLLs, the application won’t automatically traverse the tree.

Anyways, drilling down from the offending vtk.vtkCommonCorePython.pyd soon enough led me to the culprit: The Intel Threading Building Blocks (TBB) DLL conda package I was using was accidentally built in debug mode, and the debug runtimes it relied one were obviously not being deployed:

After switching to a TBB conda package from a different channel, the app was finally able to start up and run on the deployment machine.

TL;DR: Parallels Desktop 13 only supports OpenGL 3.2 on an extremely limited subset of mostly games. VMWare Fusion 10 has full OpenGL 3.3 support. In my case, this made the difference between being able to work on a VTK-based client project (VMWare Fusion 👍👍) or NOT being able to work the project (Parallels 👎👎).

I bought a Parallels Desktop Pro 13 subscription to be able to do Linux and Windows development on my MacBook Pro.

Although PD is extremely well done otherwise, they seem to have been dragging their feet with rolling out full OpenGL 3.2 support, as can be seen in a number of threads on their forums, e.g. here, here and here.

For a client project, we are currently working on a cross-platform VTK-based app which targets Windows as its main platform. I was looking forward to using my Parallels Windows 10 guest to test the prototype out. Unfortunately, when trying to run a simple VTK sample I was greeted with this error message:

Warning: In ..\Rendering\OpenGL2\vtkOpenGLRenderWindow.cxx, line 647
vtkWin32OpenGLRenderWindow (0000018AAABA44B0): VTK is designed to work
with OpenGL version 3.2 but it appears it has been given a context
that does not support 3.2. VTK will run in a compatibility mode
designed to work with earlier versions of OpenGL but some features may
not work.

The app then reproducibly crashes hard.

Further investigation with the OpenGL Extension Viewer showed that there were three different OpenGL renderers: Two of them are OpenGL 2.1 capable, and one is 3.2 capable. However, as a user, you can’t decide which app gets which renderer.

Further digging, also with the glewinfo app, reveals that Parallels only very selectively supports certain games and apps. See the list at the end of this knowledge base article.

I logged a support issue re the VTK 7 and later OpenGL 3.2+ requirement. The bug was acknowledged, and Parallels confirmed that it was now on their backlog, but that they could give no indication of when this would be available.

Client projects really can’t wait for this, so based on good reviews of the OpenGL support in VMWare Fusion 10, I made use of their Cyber Monday special to purchase a license for the Pro version.

After installation, I imported the VM from Parallels Desktop:

… after which I was greeted with the VM configuration screen where I could configure the 3D support:

After which I could boot up the same Windows VM and run the VTK sample app without any issues whatsoever:

So there you go: If you need good OpenGL support in your VMs, prefer VMWare Fusion over Parallels Desktop.

We’ve all been there: You’re used to the terminal on Linux or OSX, and then for some or other reason you need to work on Windows and you’re confronted with the half-baked monstrosity that is cmd.exe:

It’s 2011 and this is Windows 7: Why does the console still make me want to gnaw off my fingers?

To summarise: You can only resize the window vertically (so you’re always restricted to 80 characters width which is just idiotic on modern widescreen displays), copy and paste is so painful that you’ll wish that they’d rather just not bothered with it in the first place and the cmd.exe interpreter itself is primitive when compared with any modern unix shell.

Enter the two-pronged solution of Console2 and Git bash! Console2 is a console that solves the UI problems (resizing! copy and paste! terminal transparency! tabs!) and you can use it together with any combination of command line interpreters, such as for example the existing cmd.exe, bash or anything else.

Here’s my setup, configured with Git bash (you get this for free with the Windows git installer, for this purpose it’s better than Cygwin as it’s a touch closer to Windows) as well as plain old cmd.exe. To configure a Git bash tab, set Shell to “C:\Windows\SysWOW64\cmd.exe /c “”C:\Program Files (x86)\Git\bin\sh.exe” –login -i” (including all quotes) and Startup dir to %HOMEDRIVE%%HOMEPATH%. I’ve also configured Ctrl-Alt-T to invoke Console2. With shift-mouse-drag I can select text (nice editor select, not cmd.exe block select, ptooey!), and I’ve set Ctrl-Shift-C and Ctrl-Shift-V to copy and paste so it’s like the Gnome terminal on Linux. Behold:

Console2 makes the pain go away.

My Windows experience is a lot more bearable now, perhaps even slightly pleasant! Did I just say that?!

Alternatives

PowerShell replaces cmd.exe, but by default still runs inside the Windows console. Still much suckage in other words, plus that I don’t feel compelled to learn PS yes.

Cygwin mintty is also often cited as a console replacement, but seems to cause problems (for example with git) as it’s not a full Windows console.

Cygwin itself is really fantastic, but due to that POSIX compatibility layer which is otherwise a useful thing, differs too much from the native Windows goodies so that sometimes native Windows behaviour is interfered with.