Keyloggers would get the key passphrase too. And the USB stick
would have its contents pilfered. So, keys don't appear to give any
advantage over passwords on an untrusted machine.

combined with `cryptographic smart card', it is great.

For the "connect from untrusted computers" there are one-time-passwords.
I've used libpam-opie in the past with great success for the occasional
connection from internet cafe's for example.
Cheers
Wolfgang