Your own public development URL.

Tue, May 16, 2017

How to setup a public URL to access your local machine? and why you would even need that.

There are a lof of use cases where a public URL is required:
* you want to expose a webserver running on your local machine to the internet so that a colleague or a customer may have a look at it.
* you are using a service, like twillio, which allows you to setup webhooks URL: the service will call those URLs to notifiy you of something.
* you want to test an OAuth integration, with Facebook or Twitter, and you want to provide public callback URL.

It is always an option to install your code on a public server and have those requests hit this server. During development, it is much more convenient to have thos requests hit your local machine so that you can debug and/or see logs in real time.

You can also build your own custom redirecting solution using SSH, and more specifically its, SSH Reverse port forwarding feature.

A high level description of this redirecting solution would be:

Your local machine (a) is behind a firewall, either a corporate firewall or, if you are at home, behind your ISP router. In both case, you don’t want to poke a hole in your router to let public internet access your machine.

The idea is then to have a public machine (b) which would act as a gateway between the public internet and your local machine.

pre requisites

In order to implement this solution, you need:

a domain name: thhis domain will be used to get public URL(s) for your machine

a public DNS: as you want to have public URL to access your server, a DNS is required.

a machine on the public internet: this machine will act as a bridge between the public interbet and your local machine

a sshd daemon running on that server: SSH is the swiss army knife for that kind of setup, and you need to be able to fully configure the SSH server

optional: SSL certificate. If you want to protect your connection, you can also setup SSL certificates so that the publci URL can be available only through https.

DNS server and a domain name

The DNS is required to resolve the adress of the URL to access your machine using your domain name.

You have a lot of options here:

you can use the DNS can be the registrar which provided your domain name

tests on your local machine

Before setting up the DNS, you can already check that the SSS setup is working properly.

In order to test that a loc al service can be reached from the public interbet, you need to start some sort of local server.

A smple HTTP server will do the trick:

You start from a directoty containing no sensitive data, a simple python server:

python -m SimpleHTTPServer 8000

You can check from your local browser from http:127.0.0.1:8000 that the server is working, and … not exposing sensitive data.

Not is time to invoke the SSH port forwarding voodoo incantation, from your local machine:

ssh root@a.b.c.d -N -R 8080:localhost:8000

where a.b.c.d is the IP address of your droplet. With DigitalOcean, this IP address will be displayed in your Droplets screen.

You can check that the forwarding is working by pointing your server at: http://a.b.c.d:8080/.

Let’s check this SSH command as it is much simpler than it looks like:

ssh root@a.b.c.d -N -R 8080:localhost:8000

root@a.b.c.d : this command will connect to your remote machine (will use the SSH key used when creating the droplet)

-N: by defaukt, ssh will create a shell on the remote machine. We don’t need that here

-R: with this option you are asking ssh to answer on the remote side (your gateway)

8080:localhost:8000: the way you want to tunnel to answer is that any connection on port 8080 will be tunneled to the the port 8000 on your local machine (where the webserver we started previously is listening on).

DNS setup

The next step is that you want a nicer URL to access your service right?

You need to configure the DNS for your domain yourdomain.com and create a A record for alfa.yourdomain.com , with a TTL of 600, witha value of a.b.c.d which is the IP adress of your gateway.

I may take time for the DNS configuration to propagate, but once it is done, you can then access your local web server through the URL: http:alfa.yourdomain.com:88080.

Better, but can still be improved: you may want to setup multiple subdomains which would allow you to host multiple local services, or have multiple machines using this tunnel (each using a specific subdomain).

To do that, you need a proxy on your gateway. Enter …

HAProxy

HAProxy is a reliable, High Performance TCP/HTTP Load Balancer. This can be used to configue multiple subdomains with different ports.

Then you can use the certot client to configure the subdomains you want to have a certificate for.

The first step is to stop your haproxy server:

lets encrypt need to validate the fact that you pwn those domains so you need to make sure first that the DNS is propery configured for each of those subdomains as letsencrypt server will try to reach those URLs and
access a standalone server started by certbot in the CLI command.

Thats why you must first stop haproxy to avoid any conflict with this standalone server:

$ service haproxy stop

Then, you can configure all your subdomains through the certbot CLI:

$ certbot certonly --standalone -d yourdomain.com -d blissed.yourdomain.com -d pcarion.yourdomain.com -d prod.yourdomain.com -d staging.yourdomain.com -d test01.yourdomain.com -d test02.yourdomain.com
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Obtaining a new certificate
Performing the following challenges:
tls-sni-01 challenge for yourdomain.com
tls-sni-01 challenge for blissed.yourdomain.com
tls-sni-01 challenge for pcarion.yourdomain.com
tls-sni-01 challenge for prod.yourdomain.com
tls-sni-01 challenge for staging.yourdomain.com
tls-sni-01 challenge for test01.yourdomain.com
tls-sni-01 challenge for test02.yourdomain.com
/usr/lib/python2.7/dist-packages/OpenSSL/rand.py:58: UserWarning: implicit cast from 'char *' to a different pointer type: will be forbidden in the future (check that the types are as you expect; use an explicit ffi.cast()if they are correct)result_code= _lib.RAND_bytes(result_buffer, num_bytes)
Waiting for verification...
Cleaning up challenges
Generating key (2048 bits): /etc/letsencrypt/keys/0000_key-certbot.pem
Creating CSR: /etc/letsencrypt/csr/0000_csr-certbot.pem
IMPORTANT NOTES:
- Congratulations! Your certificate and chain have been saved at
/etc/letsencrypt/live/yourdomain.com/fullchain.pem. Your cert will
expire on 2017-08-01. To obtain a new or tweaked version of this
certificate in the future, simply run certbot again. To
non-interactively renew *all* of your certificates, run "certbot renew"
- If you like Certbot, please consider supporting our work by:
Donating to ISRG / Let's Encrypt: https://letsencrypt.org/donate
Donating to EFF: https://eff.org/donate-le

Once this is done, you can configure your haproxy with all those subdomains but also the SSL certificates:

As described in the letsencrypt documentation, there is a step to be done to allow the haproxy setup:

Then, to use it, you simply need to prepare a file at: /usr/local/etc/le-renew-haproxy.ini where you configure the list of subdomains you have:

# This is an example of the kind of things you can do in a configuration file.# All flags used by the client can be configured here. Run Certbot with# "--help" to learn more about the available options.## Note that these options apply automatically to all use of Certbot for# obtaining or renewing certificates, so options specific to a single# certificate on a system with several certificates should not be placed# here.# Use a 4096 bit RSA key instead of 2048
rsa-key-size =4096# Uncomment and update to register with the specified e-mail addressemail= pcarion@gmail.com
domains= yourdomain.com, blissed.yourdomain.com, pcarion.yourdomain.com, prod.yourdomain.com, staging.yourdomain.com, test01.yourdomain.com, test02.yourdomain.com
# Uncomment to use the standalone authenticator on port 443# authenticator = standalone
standalone-supported-challenges = http-01
# Uncomment to use the webroot authenticator. Replace webroot-path with the# path to the public_html / webroot folder being served by your web server.# authenticator = webroot# webroot-path = /usr/share/nginx/html

The last step is to add this /usr/local/sbin/le-renew-haproxy in your crtontab so that it is called every 10 days for instance.

More information

If you want to know moe about all you can do with SSH , you should check that video: