Surprisingly, google couldn't help me. But, it turned out to be a very simple fix.

The problem was the space in the GIT_SSH environment variable pointing to TortoisePlink.exe. I copied the "C:\Program Files\TortoiseGit\bin\" directory to "C:\TortoiseGit\bin\" and set GIT_SSH to "C:\TortoiseGit\bin\". I actually created a symbolic link, instead of actually copying it. I didn't want to risk breaking TortoiseGit, so I didn't actually move the directory. Of course, the right answer would be uninstall TortoiseGit and reinstall it in the above folder without a space in the path, but I really wasn't in the mood for yet another can of worms today.

Update Oct 15 2015:
This happened to me again on a new machine.

error: cannot spawn C:\Program Files (x86)\PuTTY\plink.exe;

Git_SSH was already set to the right path. Changing it to use %ProgramFiles(x86)% instead of the full spelled out path didn't help. After making a symbolic link to Putty on the root folder and pointing Git_SSH there, it started working.