Ever since I was a wide-eyed little boy, I would look up at the stars and wonder in wonder:
“What if I could lease my very own, beefy, dedicated Hetzner server and have an easy way to deploy all my projects onto that?”
But lo, my dreams were dashed because Docker wouldn’t be invented for another twenty years, and Hetzner did not accept Mastercard at the time.

Decades later, with Docker finally invented and Hetzner accepting all major credit cards, my dream lay all but forgotten, because Docker could not do zero-downtime deploys natively and I hated it.
That was how things remained, until my friend Theodore told me that he tried Dokku and that it worked very well.

I had heard of Dokku (and Fig, Deis, Flynn, Kubernetes, etc etc), but I never paid too much attention, as these PaaSaaSes struck me as too webcale for my simple projects.
All I wanted was a way to skip through all the boilerplate configuration of deploying a Django app, and Ansible wasn’t cutting it, as it was still too much plumbing.

Since Theodore tried it and said it was apparently pretty easy to deploy with, though, I figured I’d give it a shot and see.
It helped that Dokku was explicitly designed to be light and self-contained, whereas Kubernetes is for much larger deployments, so Dokku fit my use case exactly.

Trying Dokku out

I finally managed to deploy Django in a Docker container on production! I’ve
been trying to switch to a full Docker development/production model since Docker
came out, but only recently did the ecosystem mature enough to allow me to
easily use Docker both for development (where it excels, in my opinion) and on
production (where it’s pretty okay and quite useful).

In this post, I will quickly give you all the relevant details and files you
need to go from a newly checked-out repository to a full development environment
in one command, and to deploy that service to production. As a bonus, I’ll show
you how to use Gitlab (which is awesome) to build your
containers and store them in the Gitlab registry.

Let’s begin!

Development

After designing my first PCB, I went on
a designing spree. It turns out that making PCBs(printed circuit boards, basically a piece of plastic
that includes all the connections of your components in it. It helps make your
project smaller and cut down on the amount of wires floating around) is
so enjoyable, I’m PCBing all the things! The next victim for PCBfication is
a circuit I had originally built on an Arduino and subsequently migrated to an
ESP8266.

The circuit is a home sensor and controller. It can sense light, temperature,
humidity and motion, and includes an RF controller (at 433 MHz) and an infrared
LED so you can control your TV and other home devices. In this post, I’ll go
into some detail about the build and how it connects to other sensors and
controllers around the house.

This post is also a test of my new
Expounder concept library.
Throughout the post, various terms will be underlined
like this (with a dashed underline), and
you can click on them if you’re unfamiliar with the underlined term. After
clicking, some text will expand and explain the term.

As you can probably tell from previousposts, I’ve been pretty into hardware lately. I’ve
especially been building things like home sensors and controllers, so I have
a central computer reading motion, temperature, humidity, light and other values
in the house and deciding whether the lights or air conditioning need to be on
or off.

I also want to be able to turn these on and off from my mobile phone, from
anywhere in the world. The problem with that is that I need a way to ensure that
only my phone can turn things on in my house. I wouldn’t want someone to be able
to turn the heating on in my house at full blast when I’m not there and waste
all my electricity bill (or set fire to something).

TLS is a pretty good solution, as it ensures confidentiality between client and
server, but it does nothing for verifying the client or securing communications
against a malicious server. I needed something better, and I couldn’t find
something readily available. So I set out to write it. Thus, string
phone was born.

I very recently discovered the very useful pychromecast library, and, being the owner of a Chromecast, I started wondering what I could do with it. Since there aren’t that many things a Chromecast can do (it pretty much only plays videos over the network, although there are some other things as well), I decided to make my own TV station. Thus, Shufflecast was born.

Earlier today, i.e. a few minutes ago, I was working on my latest guinea pig, TiThess. It’s an events guide for my city, and the latest project I’m trying everything on. I like having a project I can try new things on, as it helps keep my skills sharp.

As I was testing page load times with the excellent Web Page Test, trying to get them down to the absolute minimum, I was getting an F in time-to-first-byte. This is very odd, because the whole site is supposed to be cached, so I was wondering whether the cache is doing something wrong and slowing page generation down.

To make sure, I needed a simple way to show how long page generation took, like the old “page generated in X seconds” footer that was all the rage with PHP sites way back when. Here’s how I did it:

This post needs no introduction, so it doesn’t have one. If you want to write an API and need to know how to make it secure, and have the requests authenticate against a server or a client, look no further! Well, do look a bit further, because I’m going to tell you how to do all these things in this post.

Use cases

As with most other things, your API authentication method will depend on your use case. I will detail a few common ones, along with the best authentication scheme for each one:

You know the deal, you have your fantastic Django application and it’s working great and everything, but you need to make a small change which is too cumbersome to do in the shell, so you figure “duh, I’ll just write a script to do it”. You write your external script in two minutes and then struggle for two hours to figure out how to load the models and the rest of the context so it will work with your app’s settings and all your Django goodness.

You visit StackOverflow and a bunch more sites, and they either tell you to use a management command, which is great advice, except your thing is a one-off and you don’t want to have to check it to git and go through all that hassle to get it deployed just to do this simple thing, or they give you some arcane lines that just don’t work.

Fear not, for I am here. I will give you five simple lines that will make everything work perfectly. Perfectly, I say!

Without further ado (all the previous ado was just so I could fill the paragraph so the side-box doesn’t look weird with short text), I give you the magic commands! Here they are:

One of my pet peeves when it comes to Django is that you can’t call methods that require arguments in templates. While this is fine most of the time, it does mean that you need to have one property or method per call you want to make, which sometimes gets very cumbersome.

I needed a way to define various dynamic permissions that are calculated at runtime (for irrelevant reasons, Django’s permissions framework wasn’t a good fit), and writing properties like can_register, can_add_tags, can_subscribe got tedious. These tended to be defined all over the place, rather than in one central spot, and it was hard to add more checks without cluttering the classes.

I would much prefer to have a single method (let’s call it can()) that accepted a string with the permission I wanted to check, and return True or False, depending. This is easy to do in the views, but templates would never be able to call it with an argument.

However, since Django can do dictionary-style attribute lookups, I could add a dictionary interface over the method, and allow

While writing encbup, I wasn’t very satisfied with having to download the entire huge archive just to restore a file, and still wished that I could use EncFS together with rdiff-backup to have true remote-mountable, encrypted, deduplicated, versioned backups.

Trying obnam again (spoiler: it’s still pretty slow), I noticed that it included a mount command. Looking at it, I discovered fuse-python and fusepy, and realized that writing a FUSE filesystem in Python is pretty much trivial.

The astute observer will have already realized where I’m going with this: I decided to write an encrypted filesystem layer in Python! This layer would be very similar to EncFS, with a few crucial differences:

Backing things up is important, and, luckily, there are many high-quality services geared to everyday people that are very easy to use and cheap. Unfortunately, I am not everyday people, as I am very paranoid and insist that absolutely nobody be able to see my photos of my dog and lawn. It’s a matter of privacy.

To that end, I’ve long been looking for a secure/encrypted backups service, but I haven’t managed to find a single service or tool that fulfils my requirements:

A few days ago, I wrote a post about a peculiar piece of code that a friend of mine had sent me. Since it was interesting bit of code, I thought Hacker News would enjoy it, so I posted it there. To my great pleasure, the post shot up to the first place in a few minutes and continued there for a full day, bringing just over 50,000 visitors to this blog, in total.

I was very happy that people were liking and discussing this post (and the discussion was very interesting in its own right), but I noticed that AppEngine, where this blog is hosted, was struggling to serve it. I had to create new instances because the average latency was about ten seconds(!), even though this blog is pretty much only text and static media, and I use Django’s per-site cache to cache every single page.

I came upon a very interesting and cryptic snippet of code somewhere nameless, and I can’t decide if it is brilliant or completely insane. It is a very obscure way of accomplishing the required task, but it’s around four times faster than the alternatives I’ve tried, so I have to admit that it’s not completely without merit. Still, I cringe a bit at seeing it, since it packs around four unusual Python concepts in almost as many characters.

If you’re an industrious Python programmer, you probably use pip and virtualenv to install your packages in an isolated location, which is great, and good job to you. However, having to install Django, pandas and ipython every time you want to experiment with something is a bit tiresome, since they’re all big files and downloading them takes a while, plus what if you’re on a data-limited connection and bla bla bla.

Here’s a very easy way to both speed up your downloads and avoid having to do them in the first place:

If you haven’t heard yet, let me break the good news to you: Django 1.6 is getting template apps/projects. This means that, when starting a new project, you don’t have to copy your old directories and change stuff around any more, you can just put them into reusable directories (or even repositories) and Django will be able to use them very simply:

django-admin.py startproject --template=/djtemplates/default myapp

I’ve been needing a feature like that for ages, and now I’m giddy with excitement, so, without further ado,

I have suddenly noticed that my computer’s fans have been more noisy than when I first bought it, and now it’s driving me crazy. I wondered if I could cut the fans’ power without increasing the temperatures inside the case too much, or even leave the case open. To test it, I spent five minutes writing a small script to monitor temperatures in Python, using smartmontools and lm-sensors.

If you use Django for your web apps, you will probably have come across a situation where you need to run some custom code on assigning a value to a model field. You’ve probably hacked around this by overriding the save() method or some other arcane hackery, but this is not only unnecessary, but it’s more complicated than it should be.

I’ve recently been looking at syntax highlighters, both for this blog and for Instahero, the new product my company is developing. I used to use a JS-based solution for the blog, but it broke with many inputs, it was fiddly to work with, didn’t parse correctly sometimes, and slowed the site down considerably. I especially disliked the fact that you had to include one JS

If you’ve ever programmed in Python, you have a 96.2% chance of having used the range() function. As you have a 96.1% chance of knowing, said function returns a list containing an arithmetic progression of integers, exactly as the documentation says. For example, range(4) returns [0, 1 2, 3].

If you’ve ever bothered to look at the docstring, you will see that it goes thus:

This is one of a series of tutorials that started with the popular Learn Python in Ten Minutes. The entire series is available as an ebook, which has all the tutorials in one convenient, DRM-free format, and will get any updates

I’ve been hard at work these days, and to relax I decided to develop a small application/side project. As with many applications, I needed a reliable, persistent store, so I went with the most reliable, easy to set up, performant, and generally awesome store I knew: SQLite.

However, since this was a very experimental prototype, I got tired of the frequent schema changes and wished I had something

As I’ve mentioned many times on this blog, I’m a big fan of Google AppEngine. I like the free tier, the effortless scaling, the zero administration, and how I can run Django almost unmodified and deploy in seconds. It has its downsides, but overall I like it very much.

The one part of deployment I didn’t like was dependency handling. Due to the way GAE works, you need to install all packages in t

Often I find that I need to include administration views in my site, such as statistics, management, etc etc. When these are more than just model CRUD views, a good idea is to extend the builtin Django admin site, as it’s the easiest to do.

While looking at the docs, however, I realised they were a bit cryptic and lacked good examples. It didn’t take me long to figure out how to do it, but here’s

Well, I have a computer architecture exam in six hours and can’t be
bothered, so I figured I would realize a lifelong dream of mine, and
make a program that prints “Hello world!” using curve fitting
techniques. Enlisting the help of a good friend with numerous
mathematical papers under his belt (ostensibly because he could not
afford a tighter belt), MATLAB and a longing for procrastination, we
em