Access SSH behind NAT

Introduction

I self-host a lot of stuff at home. 2x Raspberry PIs, 1x QNAP, and there are lot of software there. I do want access to my home from outside (example: rogue telegram bot starts spamming about air quality in home because my air quality hardware module got busted – don’t you just hate when that happens). Anyway, I don’t feel very comfortable exposing SSH to public. So, what else to do?

One option was to install OpenVPN, and enable access only from that network. That would work. In fact, that is probably 10x more secure solution that what I am about to present, but well… nobody’s perfect.

Reverse SSH tunn…what?

Maybe I should be ashamed, but I really didn’t know about this until 2 months ago (and I use Linux for decade or more). There is this thing called “reverse SSH tunneling” and this blog post is all about that (BTW, I purposely didn’t just named this blog post like that as I wanted to convey intent better; FWIW I do have public IP, so NAT is not even a problem:D). So, after 2 months of never letting me down, I can finally write about it, as this thing is fu*&^%$ robust.

So, how this works? Behind this cryptic name is actually just simple inversion of control, similar to how any software from 90s exposed your port to internet (anyone remember TeamViewer or similar software) – it just used SEC (somebody’s else computer) to connect to it from inside NAT and proxies (or forwards, if you will) all traffic back to you. But in our case, we don’t just want anyone to be able to peak into our traffic, so we don’t want to use any third-party service (is there actually reverse-SSH-as-a-service offer?:). So, we will build it ourselves. What do we need:

One public machine (A) with SSH exposed over the internet. It can be any cloud VM, DO droplet or anything you have (root) access to

One private machine (B) in a secure network that you want to access

So, how to do it, Here is step by step guide.

Step 1 – make sure regular SSH work

Make sure you can do regular SSH from B to A (ssh usernameOnA@A). If this doesn’t work, no point in proceeding:)

Step 2 – create reverse SSH

So, pick any port (I will use 10022 here), expose that port from machine A to internet (make sure it is not under firewall or expose it in portal of your favorite cloud provider). On machine A make sure you have line GatewayPorts yes in /etc/ssh/sshd_config. Now, from machine B, do:

ssh -fN -R 10022:localhost:22 usernameOnA@A

This will create a tunnel B<===>A and move itself to background (so you should get your prompt immediately). Now, from anywhere, try to connect to machine A, but not on regular port 22, but on port 10022:

ssh usernameOnB@A -p 10022

This is moment when you want to digest this whole thing:) You just connected to newly exposed port 10022 on machine A that will “tunnel” you straight to machine B! If this doesn’t blow your mind, you are soulless person and I hope you ever find love in your life. On the other hand, if this doesn’t work for you – tough luck, try searching online, I am here to expose you to new concepts, I am not here to be your support.

Step 3 – paswordless login

OK, same thing, but without passwords. Generate new key on machine B and add it to authorized keys on machine A (just duckduckgo for “ssh-keygen ssh without password” or something along those lines). Now, you should be able to issue something like this:

You should not get asked for password! You should, however, be asked for password when you ssh to B machine (on port 10022). Now, for last step.

Step 4 – auto-login on boot

To wrap it up, you want this connection to be always alive, always available and available upon boot. So, first you need to install autossh tool, it will make your life easier. After that, you should place these lines somewhere that will executed upon boot, like /etc/rc.local or whatever init system rocks your boat:

There are some new lines here, but I left it as exercise to reader to figure out what do they mean. I also left as exercise to reader another cool thing: do passwordless login directly from any machine to B on port 10022. So, basically, generate key pair on machine C, expose it to authorized_keys on machine B and try to fully login without passwords (C to A to B). Trust me, you do want this, if you are already exposing your SSH to internet!

Conclusion

Self-hosting is sometimes not easy, especially when you have a lot of things. That’s why I like robust things that do not break. This little guy still didn’t let me down and I do use it from time to time – connection is always there and is never broken (and when I need to use, it is probably when I need it the most, so I would be really pissed if it doesn’t work:).

Now, I know what some of you would say – “but, hey duuude, isn’t it just as insecure as regular SSH?”. Well, I am not security expert, but I would say it almost isn’t – you are protected with another layer, but it is so thin that you can also assume that it is not there, so no – this is not your FBI/script-kiddy protection, for that – OpenVPN is better suited and I might talk about it soon.

My basic point is – I did it, this is freaking cool, I use it daily, it just works and you just learned something new…or you didn’t, you snarky smartass:) Anyway, I am geek for writing this, you are geek for reading this (you just got to this point, didn’t ya!) – being geek is awesome:)