Accessing IMAP/SMTP via HTTPS Proxy

In some places Internet access is available only via proxy, which in practice means that you are limited to HTTP and HTTPS protocols only. But if you have external email accounts like Gmail, this is bit limiting, because you cannot access your email via IMAP protocol from your email client.

However there can be a solution – if proxy support HTTPS protocol, it means also that it supports CONNECT method that tunnels a connection to remote server unchanged. This method could be used to tunnel any protocol, so basically it could tunnel also IMAP ( and SMTP for outgoing email).

The only piece that is needed is either support from client (but Thunderbird unfortunately does not support this) or a tunnel program running on local machine + slight change in email account – to use local connection to tunnel proxy instead of direct connection to server. But there is still one small glitch – if your computer is notebook and you come home, there is no proxy, and you want to connect to mail server directly again. To handle this scenario local tunnelling proxy should be smart and fall- back to direct connection to server, if HTTPS proxy is not available. I have not found any program like this, so I’ve written mine own – called ptunnel. Recently I returned to this problem and created new ptunnel in Rust – it’s much faster and less resource intensive, so I recommend this one.

Now how to do this in practice:

Get ptunnel from links above (old one requires python, new one must be compiled by locally first).

Run it with appropriate parameters – for Gmail it should be something like this:
Rust program:

You should assure that ptunnel is automatically started, when computer starts – the ways how you can do this are various – I prefer to start it with my desktop (Startup Applications in Gnome, Unity, Cinnamon), you can use /etc/rc.local or some other ways – I believe it may work fine even under other platforms like Win or Mac, even I never tried it.

You have to reconfigure your email account – IMAP server should be localhost:9993 and smtp server localhost:5587.

After reconfiguration, when you’ll try to receive or send email, you will get security warning – that’s because SSL certificates are not matching. You now connecting to localhost, but certificates are issued to mail server DNS name – you have to add security exceptions for these certificates.

This tunnelling works for me fine for quite some time ( a year or so). Tunnelling is not limited to IMAP /SMTP only so it can be used for SSH etc.

Post navigation

25 thoughts on “Accessing IMAP/SMTP via HTTPS Proxy”

Wow! this looks cool: Unfortunately I can’t make it work with my versions of Python. I’m having Python 2.4.3, and it complains socket doesn’t have a ‘create_connection’ attribute. When trying with Python 3.2 (alter running 2to3), I get an error from sock.sendall(), TypeError: ‘str’ does not support the buffer interface. Do you have any idea what to do to make it work with Python 3.2?

That would relate to fact that in python 3 str type is basically unicode – so we need to encode to bytes before sending
try change
sock.sendall(Tunnel.CONNECT % tuple(self.server.tunnel[1:]))
to
sock.sendall(Tunnel.CONNECT % tuple(self.server.tunnel[1:]).encode(‘ascii’))

You made a great work !
Your python solution works perfectly with current Thunderbird version also (at least for windows)
first : I use the network proxy discovery in advance thunderbird settings, to be able to use the Thunderbird mail account assistant to work. I define their my personal imap/smtp account. Then I refine with manuel settings to set SSL option and user authentication. at this step imap/smtp don’t work, since not relayed by my corporate proxy to my ISP.

Second: I installed latest Pyhon2 version and your “ptunnel.py” script file. I rename ptunnel.py into ptunnel.pyw to avoid getting an unnecessary windows.
I have created a “ptunnel.bat” like this :
start “C:\Program Files (x86)\Python2\pythonw.exe” “mypath\ptunnel.pyw” -d -p myproxy:myport 9993:imap.myISPserver:993 4465:smtp.myISPserver:465
and put a shortcut to ptunnel.bat into my “startup” menu.

Third: into Thunderbird email account settings, I replace the smtp host/port by localhost, port 9993 for imap, and localhost, port 4465 for smtp.

Hi,
thanks for link, the main difference is that ptunnel falls back to direct connection if proxy is not available, so it works for me on mobile notebook.
Otherwise proxytunnel is cool and can be used for tunnelling as for instance for ssh protocol.
I.

I need help with this step (3)
3. You have to reconfigure your email account – IMAP server should be localhost:9993 and smtp server localhost:5587.

I am a complete noob – so bear with me .
Do I need some IMAP and STMP servers running locally? Which server should I install and how do I configure it?
Because when I tried to install postfix on my Linux (Debian), it asked me system mail name, what should it be?
thnaks.

No you do not need any servers running locally – this program is tunnelling traffic – so if you start it with:ptunnel.py -d -p your-https-proxy.com:80 9993:imap.gmail.com:993 5587:smtp.gmail.com:587
It listens locally on port 9993 for IMAP and 5587 for SMTP protocols, which are then forwarded though proxy to their final destination – gmail servers.
So now you need to reconfigure your client to communicate with this tunnel – e.g. IMAP is localhost:9993, SMTP is localhost:5587

Hi,
I was looking for the same solution using perl. finally after understanding http proxy and how it works, got solution.

1. Squid Proxy to add 993 port as SSL and restart proxy
2. Make http request to proxy using authentication if any.
3. Make same connection as SSL.
4. You will be able to login imap server.
5. Tested with gmail and aol etc.

If any one need help to implement same solution and having trouble, feel to send mail to pankajmoon@gmail.com
Thank you,
Pankaj

For perl there is script connect-tunnel (http://search.cpan.org/dist/Net-Proxy/script/connect-tunnel).
Key difference here is that ptunnel.py has the direct connection as a fall-back – which works well on notebook, where connection changes between restricted ( with proxy) and open (without proxy).