Custom Directory Listing with Nginx and Go

3 minute read
Published: 21 May, 2013

For the last few years, I’ve been maintaining a large repository of files and
folders on my website here using lighttpd’s
default directory index generator. The generator is fine to get the job done,
but offers no extra features. I just recently switched to nginx and its
directory index generator is a bit worse than lighttpd’s (the autoindex
directive). This approach worked fine for a while but I really wanted the
option to have a custom file ordering for certain directories, e.g. to order
by date descending so newer files would automatically float to the top of the
file list. So I wrote a HTTP server in Go to do just that, and a little more!

This project was my first real foray into the Go programming
language (which I have a few choice opinions about but
I’ll express those in another post later). For the most part, the experience
has been pleasant, save for a few language warts. The Go runtime is rock solid
and my HTTP server has not gone down at all. I keep it running with upstart
on my Ubuntu server. If you’re not managing your daemons with upstart, you
definitely should start. It’s far easier than the horrible copy/paste/modify
workflow of those awful init.d scripts.

What I do is have nginx act as a reverse proxy for /ftp/ requests to my Go
HTTP server which is just listening on a localhost port. I intend to change
this over to use local Unix sockets for more security and to save my sanity in
dealing with TCP port numbers and remembering which one goes where.

The main features of this directory listing generator are custom ordering of
files per directory and slightly advanced symlink support.

To specify a custom ordering for a directory, just create a file named
.index-sort in the directory and have its contents be a single line
specifying the sort mode. The available sort modes are documented on the
GitHub project’s README. To override the default sort order, you can
specify the ?sort=mode query string parameter in the request.

The advanced symlink support helps to translate filesystem symlinks into HTTP
302 redirects. This works for both files and directories. If the symlink
target path is within the filesystem jail being served up, the request will be
served, otherwise a 400 Bad Request error will be presented.

For example, if you have a set of versions of some file and a symlink that
always points to the latest version, the directory listing will 302 redirect
from the symlink request to the actual target filename that is the specific
version. In other words, a request to file-latest.kind might redirect to
file-v1.kind. This way, the downloaded filename will represent the symlink
target file-v1.kind and you can be sure which specific file your users have
downloaded, instead of the file being served up as file-latest.kind and you
having no clue which one that represented at the time the user downloaded the
file.

I’m really pleased with this setup and it took me only a few hours to code up
and test. Go does allow one to be productive right off the bat. Best of all,
there’s no funny business about threading, concurrency, or reliability like
you get with other things like Ruby or Python (mostly the concurrency issue
here). There’s just fast, compiled, statically typed code here; just the way I
like it. Of course Go isn’t perfect, but we’ll get into that later.

Feel free to use this process for hosting your own directory listings. I look
forward to the pull requests!