Pragmatic Path Towards Non-technical Users Owning Their Own Data

I don't have a statistic, but I assume that the vast majority of the applications and content that the average user interacts with these days goes through a web browser at some point.

If your app doesn't have a URL, who's going to use it?

The question haunts anyone who wants to build new peer-to-peer applications and networks.

Beyond the ability to type in mystuff.newcoolthing.com and receive instant HTTP-based gratification, users expect a lot from web applications these days:

100% functionality in the app/browser within milliseconds

Gmail, Youtube, Google Docs, Facebook, Twitter, etc

Mobile Friendly

Global

Internationally hosted with Geo-specific DNS records

High Availability

99.9% Uptime

Worst case first impression load time less than 3 seconds.

Green lock icon in URL bar

If your app doesn't fit those criteria, it is likely to be left in the dust. So how can we build peer-to-peer apps in such an environment? Or apps that respect users' privacy?

In recent history, the answer has been that you don't. Besides a few ancient and highly specialized peer-to-peer networks like BitTorrent, almost all apps are cloud based and operate under centralized authority by necessity. While some people (and some shareholders) might consider that total authority and access to all user data to be a great feature of "the cloud", I consider it a bug.

Let's play Web Mad Libs...

A user types ____.____ into their web browser. The web browser sends an HTTP GET request. The implementation asks the system to resolve ____.____, and ultimately receives some public internet IPv4 address --.--.--.-- from the user's ISP's domain name system. The browser talks HTTP with the server at --.--.--.--. That server may proxy and/or route the HTTP connection to ____ (in the ____ network) at the ____ protocol level. When the connection reaches ____, it will be redirected via the HTTP protocol to HTTP2/HTTPS. The process will repeat, this time with the TLS negotiation occurring/terminating at ____. All of the application's code and data is delivered to the browser over the HTTP2/HTTPS connection, the app ____s in the browser, and away we go. Oh, and don't forget: the user's login credentials are always transmitted and managed via email and SMTP.

Even with the rigid restrictions imposed by established infrastructure and user expectations, we might be able to leverage open source tools and cloud service providers to sneak user autonomy, data ownership, and decentralization into this framework.

I always liked the idea of hosting my own stuff -- providing a URL to myself and to the world under my own power, under which I can put whatever I want. I don't even have to pay for it, besides the energy used and $1 a month for a domain name. But the devil is in the details. What happens when your internet service provider has an outage in your area? What happens when someone trips over the modem in your living room? What happens when the power goes out, or when you move? The TLS certificate expires? The server runs out of disk space or powers off unexpectedly?

These days, people don't bother to self-host anything because it's a nightmare to manage, not because it's expensive. There are so many things that can go wrong, and it's so much easier to just fork over all your data to the free cloud services. Or if you are a business, maybe you shell out thousands per month for an Enterprise Cloud Services Account with the expectation that, while your counterparty does in fact own all of your data and systems, they won't touch them.

But with all of the free services, automation tools, and great FLOSS software that is around these days, I think self hosted apps could get a second chance. The trick is to automate the complex, manual parts into a platform that's just a few clicks away from potential users.

Here's my proposed architecture for highly available web applications, owned by their users:

There are a few key features of this architecture that I would like to point out:

If one app server goes down, the service stays up and chances are no data is lost.

As a matter of fact, this is exactly what I plan to move towards for sequentialread.com in the future. I already have an instance of openvpn and haproxy running in TCP mode on AWS, serving the page you are reading right now... All I need next is a good way to set up my apps in clustered mode with replication, and a secure management tool to administer the whole system from one place.

Of course, that would require a lot of work, so it won't be done any time soon. I've been thinking about this stuff for a while, and I'm sure I'll continue to revise my thoughts as I try things out.

Setting up Gogs in HA mode is one of my primary goals to figure out first, so look forward to a post on that in the future.