Saturday, November 18, 2017

Note: due to formatting issues, some bash commands below have incomplete display. Just copy and paste the bash code block somewhere for the complete commands.

Deploy ZeroNet to Heroku as a Public or Private ZeroNet Proxy Service

Introduction

This is a tutorial on deploying ZeroNet to Heroku as a Python web app. You can host a (1) public or (2) private ZeroNet proxy, or only host (3) your personal zsites (ZeroNet site) statically (with all your zsite contents there) or dynamically (as an open gate to the clearnet and fetch your zsite contents from the zero network) on Heroku. An example zsite hosted on Heroku:https://dcentral.herokuapp.com

Limitations

No persistent storage with Heroku dynos.

If you host a (1) public or (2) private ZeroNet proxy, your user data will not be saved on Heroku. For example, if you visit your proxy and subscribed some new sites, and next time you visit the proxy again, all subscriptions will disappear and it is like a fresh new deployment. The reason is data/zsites downloaded on Heroku are on a temporary space and will be erased whenever a session ends (e.g., when your dyno sleeps and restarts, at most every 24 hours).

If you want to host (3) your personal zsites statically, you need to deploy every change of your zsite (e.g., after you published a new post) to Heroku. Otherwise, it is equivalent to host it dynamically because user interactions through the ZeroNet UI will not be saved, and the Heroku copy of your zsite files is always the same as your last deployment, which can be older than the latest copy in the zero network. Although it will update automatically and fetch your latest zsite contents from the zero network whenever you visit your zsite hosted on Heroku, the updates can not be saved for your next visit, which means it has to download even the same updates from the zero network next time — in other words, it is like to host your zsite dynamically.

Heroku does not allow opening ports or use multiple ports. By default, ZeroNet use the port 43110 to serve the web UI, and also want the port 15441 for peer communication, although the second port is optional. However, Heroku only allows you to use one port, so we have to use the only one for ZeroNet web UI. Without a port for peer communication means we have to run ZeroNet in slow mode and creating new sites or publishing new contents may not work well (but you can do that locally).

Step 1. Register a Heroku account.

Step 2. Install Python 2. (Optional)

Python version 2.X (instead of Python 3.X) is needed for testing and using ZeroNet locally. If you only want to host a (2) private ZeroNet proxy, you can skip this step. If you want to host a (1) public ZeroNet proxy without a zsites whitelist so any user can visit any zsite through your proxy, you can also skip this step. But if you want to host a (1) public ZeroNet proxy and only allow users to visit some whitelisted zsites (disallow users to add new sites), or host (3) your personal zsites, you must be able to run ZeroNet locally.

Ubuntu should have Python 2 installed by default, you can check it by:

You now can delete the file “GeoLite2-City.mmdb” in the “ZeroNet/data” folder to save space for deployment. The file “users.json” in the “ZeroNet/data” folder contains a “master_seed” and a list of zsites you just visited/subscribed. You can login using the “master_seed” as the administrator later to add and delete subscribed sites through the ZeroNet UI in multi-user mode. Make sure your “ZeroNet/data” folder contains a complete copy of the default homepage site “1HeLLo4uzjaLetFx6NH3PMwFP3qbRbTf3D” in the folder “ZeroNet/data/1HeLLo4uzjaLetFx6NH3PMwFP3qbRbTf3D”.

Now enable git to include files in “ZeroNet/data” folder:

nano .gitignore

find and change the following two lines

# Data dir

data/*

*.db

to

# Data dir

#data/*

#*.db

then ctrl+o, enter key to save and ctrl+x to exit.

Step 6.2.

Enable multi-user mode for ZeroNet,

mv ./plugins/disabled-Multiuser ./plugins/Multiuser

Alternatively, manually rename the folder “disabled-Multiuser” in “ZeroNet/core/plugins/” to “Multiuser”.

Step 6.3.

Now, add the Procfile for Heroku:
If you want to disallow users to add new sites to your proxy,

depending on whether you want to disallow users to add new sites or not.

Note, ZeroNet use the default local IP 127.0.0.1 and port 43110 to serve the web UI. However, as you will install ZeroNet on a remote machine, you must set the UI IP as “*“, as described in https://zeronet.readthedocs.io/en/latest/faq/#is-it-possible-to-install-zeronet-to-a-remote-machine. Also, Heroku requires all web app to bind a dynamic port (which can be accessed by the environment var $PORT) it assigns you when your app starts, so you need to change the default port for ZeroNet UI to $PORT.

Now go to Step 9.

Step 7. This step is for hosing a (2) private ZeroNet proxy.

Now enable the UiPassword, so a user can only access the UI with the password.

mv ./plugins/disabled-UiPassword ./plugins/UiPassword

Alternatively, manually rename the folder “disabled-UiPassword” in “ZeroNet/core/plugins/” to “UiPassword”.

Open http://127.0.0.1:43110/1DCNTRLnCAGxhZh4GEbkLAJu8AVFAM82ui in your browser to visit your zsite.
If you want to host your zsite, make sure a full updated copy of your latest zsite is downloaded to, e.g., “ZeroNet/Data/1DCNTRLnCAGxhZh4GEbkLAJu8AVFAM82ui”.
Now enable git to include files in “ZeroNet/data” folder:

nano .gitignore

find and change the following two lines

# Data dir

data/*

*.db

to

# Data dir

#data/*

#*.db

then ctrl+o to save and ctrl+x to exit.
Next, enable multi-user mode for ZeroNet,

mv ./plugins/disabled-Multiuser ./plugins/Multiuser

Alternatively, manually rename the folder “disabled-Multiuser” in “ZeroNet/core/plugins/” to “Multiuser”.
Now add the Procfile for Heroku:

#every time you modified something before deploy/update your changes to Heroku

#(A) add the modified files to the local git repository:

git add .

#(B) commit the changes to the repository:

git commit -m "notes_on_changes"

#(C) deploy the code:

git push heroku master

#Ensure that at least one instance of the app is running:

heroku ps:scale web=1

#Now visit the app at the URL generated by its app name

heroku open

#View logs

heroku logs --tail

In you browser, you should be able to see the ZeroNet web UI served by Heroku at your Heroku app URL.

Step 10. If you want to update something.

Firstly, make the changes using your local copy of ZeroNet. For example, you can update “ZeroNet/data/users.json” to whitelist some new zsites by visiting them using your local ZeroNet copy. Or if you’re hosting your own zsite statically, you can post some new contents through your local ZeroNet, so your local copy of “ZeroNet/Data/1DCNTRLnCAGxhZh4GEbkLAJu8AVFAM82ui” is newer and has more contents than the Heroku copy.

Then, deploy the changes,

#(A) add the modified files to the local git repository:

git add .

#(B) commit the changes to the repository:

git commit -m "notes_on_changes"

#(C) deploy the code:

git push heroku master

If you only want to host your zsite dynamically, you don’t need to deploy again after you updated your zsite.