Deploying an Electrum Buildpack to Heroku

Long story short, I had an idea for a project but no real time to make it happen. Rather than let my initial research go to waste, here are the steps I took to get the Electrum Bitcoin client running on Heroku.

Warning

Storing your Bitcoin private keys anywhere that is accessible to the Internet is dangerous! Please take caution when implementing the steps read here. I am not responsible for coins lost as a result of running electrum in the cloud.

I will talk about proper architecture on here in the future but always have as few coins as operationally possible in online 'hot' wallets.

I should also point out that I am just a hobbyist programmer. I managed to get this working but that doesn't mean I did it 'the right way'. Herokuists may cringe while reading through this.

What is a buildpack?

A buildpack is used in the Heroku world to setup your worker agent's local environment (also known as a 'slug'). If no buildpack is installed during deployment, you are basically just left with a barebones OS - incapable of doing all that much.

Most of the time, the buildpack your application needs is automatically detected and ran during deployment. For example, if you pushed a Ruby application, the deployment process would detect the Ruby code and install heroku-buildpack-ruby, giving the slug the ability to run ruby code.

Buildpacks are not solely for adding language support though. This same system can be leveraged for installing other services such as database servers or messaging queue.

In short, a buildpack is a set of shell scripts that bootstrap the slug so that the deployed application actually has all of the prerequisites available when it goes to execute.

Why Electrum and not Bitcoind?

Electrum has a number of advanced features, which are awesome to have, although in this case they were irrelevant to my motives. Electrum was chosen because it doesn't need to store the entire blockchain locally, it can query the details it needs from external services.

The filesystem on a slug is completely volatile and anything stored within a worker may be lost at any time due to redeployments, restarts and scaling changes. To have bitcoind deploy as a worker, would mean 14GB of blockchain needs to somehow show up on the slug every deployment. Alternatively, storing the blockchain external of the worker agent would mean using a service such as Amazon S3 ($$$).

Writing the buildpack

If you don't care about how the buildpack is put together, you can skip this step all together and go straight to including this buildback in your project.

To start my buildpack, I setup a directory with the files needed by Heroku.

This script is the heart of the buildpack and, even so, is fairly straightforward. As you can hopefully see, the main action is the pip call, which actually installs the electrum client.

There are a couple tricks within the above though. Particularly regarding the fact that pip needs to be installed prior to this buildpack or it will completely fail during deployment. Finding where pip was stored was also a bit of a chore but that fun is done now.

Also note the last line, there is no configuration of wallets that take place during the deployment. Your worker will need to handle this part.

buildpack-electrum/bin/release

#!/usr/bin/env bash
# bin/release <build-dir>
# I didn't really feel the need to add anything here.
# This script can be used to start services but in our
# case electrum is just an application that gets called
# later by our worker.

buildpack-electrum/bin/detect

#!/bin/sh
# bin/detect <build-dir>
# Similarly, not much to detect here prior to the buildpacks install
# Without the following line, the buildpack deployment will fail
exit 0

That's pretty much it for the buildpack. Now we just need to push it somewhere that Heroku can access so that it can clone.

Creating your application

Now we have our electrum buildpack, either through creating your own or by using the one I have already deployed to git://github.com/Abstrct/buildpack-electrum.git. We can now build our own application that will use the electrum client.

That last command is one of the most important so far. Since Electrum is actually a python application, we need to install python to the slug before we are able to get Electrum on there. Unfortunately, there can only be one buildpack specified in the BUILDPACK_URL config var.

To resolve this issue, we tell Heroku that our new application needs to the the heroku-buildpack-multi buildpack. This will allow us to define multiple buildpacks in the next step.

Alternatively, we could have forked the heroku-buildpack-python codebase and then added in the elctrum details along with it, but that seems short sighted. Any time changes are made to the python buildpack, we would need to rewrite the electrum buildpack to include the new code.

Since we are using the heroku-buildpack-multi package, we need to create a file listing all the buildpacks we need installed.

Now we should quickly write up our Procfile so that Heroku knows how to run our application after deployment.

hot-electrum/Procfile

worker: python worker.py

And finally, let's quickly write that worker.py file.

hot-electrum/worker.py

import os
quit()

Realistically, we would be writing our python code to setup the wallet, handle lookup requests, craft transactions, etc, etc. But this is where I ran out of time. So let's cut to the important part, confirming that we have electrum running inside Heroku!

Now, I really feel the need to say this again. If you create a wallet on this instance, and you do not back it up, any bitcoin being stored by this wallet should be considered gone.

Maybe I should say this again. If you run heroku run bash and you create a new wallet. This wallet will be destroyed when you exit the terminal. If Bitcoin are stored in this wallet, and there isn't a backup, then the Bitcoin is as good as gone.

As a final thought, I just want to make this a bit more useful in the real world. Let's make a couple quick changes to the worker.py script, so that it creates a wallet locally, based off of heroku environment variables.

First, set the variables. These variables are generally considered to be pretty secure. I can't speak for Heroku really, but the idea is that you should use these so that static passwords/values are not stored directly in your codebase - which sounds like a good plan to me.

I had already created an electrum wallet, using electrum create and this is the seed that it generated. This is a string of words that you should NEVER tell anyone - my display of them is for training purposes only.

This really doesn't do all that much still, just sets up the environment. From here, you would likely want to start working with a queue service so that your web app can send commands back to this worker such as transactions, balance lookups, new address requests, etc.