I might just be dumb, but according to http://www.gnupg.org/ there is a standalone (or portable) version that one can get of GnuPG. That is, there is a version of GnuPG that doesn't require any installation and which should be able to run from for example a USB-key.