When I was replacing peewee with PonyORM, I was evaluating a few options, including moving away from an ORM entirely and simply storing the metadata in indexed tables in memory. This would have also helped to solve a couple of minor annoying design issues (such as improper encapsulation of the actual content state into the application instance), but I ended up not doing this.

A big reason why is that there don’t actually seem to be any useful in-memory indexed table libraries for Python. Or many other languages.

This article was originally written for the Publ blog. I have reproduced a slightly modified version here so that it hopefully finds a wider audience.

Whenever I build a piece of software for the web, almost invariably somebody
asks why I’m not using PHP to do it. While much has
been written on this
subject from a standpoint of what’s wrong with the language (and with which I
agree quite a lot!), that isn’t, to me, the core of the problem with PHP on the
web.

So, I want to talk a bit about some of the more fundamental issues with PHP,
which actually goes back well before PHP even existed and is intractibly linked
with the way PHP applications themselves are installed and run.

The problem

Let’s say you want to make a single-binary application that has embedded resources (images, GLSL shaders, etc.). Let’s say you want to automatically wrap your resources in a storage container to make it easier to deal with stuff. Let’s also say that you might even be using CMake as your build system.

CMake doesn’t provide a way of making a custom build rule, and using extern data is a little unwieldy. So here’s an easy-ish way to do both parts, making use of C++11 language features (and a scary preprocessor hack).

The C++11 bit is also useful on its own, even if you aren’t using CMake, although things will have to be adapted to your build system of choice.

Note: Back when I wrote this the general compiler ecosystem was different, especially on macOS. If you just want a library that does all this stuff for you in a platform-independent manner, check out this resource embedding script. Or you might be interested in a CMake-only approach for the resource generation and using that in conjunction with the rest of this article.

The UNIX select() API should have been deprecated years ago. While unsafe operations like sscanf(), sprintf(), gets(), and so forth all provide compile-time deprecation warnings, select() is also incredibly dangerous and has a more modern, safer replacement (poll()), but yet people continue to use it.

The problem is that it doesn’t scale. In this case, “not scaling” doesn’t just mean it’s bad for performance, “not scaling” means it can destroy your call stack, crash your process, and leave it in a state that is incredibly difficult to debug.