What it is (or, what it will be)

It’s hierarchical key-value storage. No schemas. Tables can contain tables, with no limit.

This implementation is the lowest level: the part that gets, sets, and deletes data from the database.

It’s application-agnostic, at this level — it doesn’t know about all of Frontier’s data types, for instance. A level on top of this will be needed for new-Frontier.

SQLite, my favorite hammer

I’m not actually writing a new database — I’m using SQLite. And that’s because I’ve been using SQLite for 15 years, and I love it and know it well, and I know how incredibly stable it is. I’m not willing to write my own thing, and I’m not willing to use a thing less mature and rock-solid than SQLite.

Every table has an id. Every table (except the root table) has a parent_id that points to its parent table.

And every value has an odb_table_id that points to its parent table.

This way it’s easy to get a table’s children: it takes just two select statements.

(Both tables and values also have a name, since this is key-value storage.)

Tables and values will be cached in memory, so not every call will require a database read.

(Before you suggest I use something other than SQLite, know that I won’t change my mind on this.)

(Also, again: it’s not done yet. Doesn’t even build.)

Why I’m doing this now instead of something else

I’m using schema-less storage for feeds in Evergreen. (Articles and article status, on the other hand, are stored using a schema, in SQLite.)

Currently I’m writing a big binary plist with all the feed data, and it has to be rewritten every time a feed property changes. The writes are coalesced — but still, this isn’t great.

I’m using schema-less storage in part because of syncing systems: I don’t know, and can’t guess, what I’ll need to store. Different systems will have different requirements.

Also: I may add features later that require additional feed properties. I don’t know what those are.

I realized that what I really want for this is a feature from Frontier: hierarchical key-value storage.

Each system will gets its own database on the client. For each, I’ll create an odb table called feeds. Each feed will have its own subtable. The key will be its id (which may or may not be its URL, depending on the syncing system).

And inside each subtable I can put whatever I want, at any time, without having to change any schemas or implementations.

Example:

For the On My Mac account — not synced; reads feeds directly — we keep track of Etag headers in order to support conditional GET. So, for example, I’d want to get, set, and delete feeds.[feedID].etag.ifModifiedSince.

But with most syncing systems we get the feed content from the system itself — not by directly reading the feed. There might be some other data from the service to store: feeds.[feedID].syncToken, for instance.

Assuming there’s a data model, maybe a database, some networking code, that kind of thing, then you can use that exact same code in your Mac app, quite likely without any changes whatsoever.

That leaves the 20% or whatever that’s user interface. AppKit is not the same as UIKit, but it’s recognizable. Same patterns and concepts, and often similar names (UITableView/NSTableView).

Given that you’ve done the hard thing — learning UIKit, Xcode, and Swift and/or Objective-C — taking the next step and learning AppKit seems like a very small thing. You’ve climbed the mountain already, after all.

You might complain that AppKit has some weird stuff. True. Some of it, though, isn’t truly weird — it’s just weird to you if you’ve never dealt with things like a menubar and multiple, live-resizable windows.

People coming from AppKit to UIKit (few people these days; many people 10 years ago) might also complain about safe content area insets (or whatever the thing is these days) and size classes and all manner of strange stuff they like not having to deal with in Mac apps. UIKit’s weird too, to some people.

Ten years ago I thought that all the new iOS developers would translate to lots more Mac developers. That that didn’t happen is a huge surprise to me. Because if you’re an iOS developer you’re practically a Mac developer already.

(And — little-known secret — the economics of Mac apps appear to be more favorable than for iOS apps.)

There’s an unofficial Seattle Xcoders this Thursday at the Cyclops in Belltown. I plan to get there around 6 pm.

We’re always in back, next to the bar but technically in the restaurant section. Anyone is welcome — you don’t have to be a coder! We regularly have designers, testers, support people, product managers, and so on.

Heck, even if you’re a fan, you should come. Should be a beautiful night to hang out with some fine folks.

To publish to this blog, I run a little web server on my Mac that implements the MetaWeblog API, which then renders this blog and rsyncs it to the server. (This way I can write using MarsEdit.)

What I’d rather do: run that little web server on the actual server, and do the static-site generation there. That way I can post from my iPhone and iPad, not just from my Mac.

But… here’s where web deployment gets tricky. I’m on an inexpensive shared host plan at DreamHost. The machine is running an older version of Ruby that’s incompatible with my scripts.

I could use RVM and Bundler, I guess, to use the version of Ruby I want to and to install the gems I need. (It’s just a few, but it’s more than zero.)

That is, if I could figure out how to use this stuff and get it installed on the server. Looks like something I could spend weeks doing (remember that my hobby coding is limited to nights and weekends).

Alternately, I could get an inexpensive VPS from one of the various providers and set things up there. That might be easier — maybe I could skip RVM and Bundler and just install the things I want to use in the old-fashioned way.

But then I have to deal with a bunch of other things myself, including setting up Apache or Nginx. All the things DreamHost does for me automatically I’ll have to handle myself. That doesn’t sound like fun at all.

I totally don’t know what to do. It’s not my plan to become a Ruby deployment expert or to be on the hook for running a server all the time. I’ve done way too much of that kind of thing for one lifetime already, and I’ve mostly been glad to be out of it.

What surprises me is that in 2018 it still requires so much work just to get a CGI script running on a server. It should be easier.

A diatonic harmonica is designed to ease playing in one diatonic scale…

Blues harp subverts the intention of this design with what is “perhaps the most striking example in all music of a thoroughly idiomatic technique that flatly contradicts everything that the instrument was designed for.”

That breaks my heart, because so much of my start on the web came from being able to see and easily make sense of any site I’d visit. I had view source, but each year that goes by, it becomes less and less helpful as a way to investigate other people’s work.

One of the ironies of this is that HTML5 makes it easier than ever to make readable, simple HTML. I especially like two things:

Quotes for attribute values are optional (when there are no spaces), and

There are semantic tags for things where before you had to guess at the author’s intention. We have header, main, nav, article, and similar now.

I realized that this blog — since it doesn’t use cookies or JavaScript, since the layout is as straightforward as can be — would make a good personal test case. How easy-to-read can I make the HTML?

So I adopted the semantic HTML5 tags, simplified a few things, and now the source is as easy to read as any HTML I’ve ever written.

Lesson learned: the discoverable and understandable web is still do-able — it’s there waiting to be discovered. It just needs some commitment from the people who make websites.

The UI design is much higher contrast — Twitter for iOS even acknowledges when you have the system’s Increase Contrast setting enabled, as I do. And, crucially, the official client natively supports alt-text, which allows users to append image descriptions for the blind and low vision before tweeting.

I interviewed Aaron Bendickson — Omni sysadmin, pinball wizard, Very Patient Man Who Loves His People and Isn’t Bothered At All Ever By All My Incessant Questions — for the latest episode of The Omni Show.

I had to spend some time turning 50 years old, which was ridiculously good fun. (One day I hope my 11-year-old nephew and I finish the cover of “Smells Like Teen Spirit” we were working on!)

And… my nine-year-old blogging system needed an update, and I just couldn’t stand it anymore, so I rewrote it. It’s nearly finished now — finished enough that I can post to my blog again, at least.

And then I realized that I had kind of a mess with Evergreen and Frontier frameworks. I was thinking about how I wanted Frontier’s hierarchical key-value database (which I haven’t written yet) in Evergreen — and so, obviously, they should share this framework. And, well, there are a bunch of frameworks they should share.

So I started work on converting over to Git submodules, so that they can share frameworks, and so the frameworks can live in their own separate repositories. Which of course also meant learning how Git submodules work in the first place.

And it turns out that Frontier doesn’t build right now, and needs to be updated for Swift 4. But it needs to build before I can tell if I’ve got frameworks-as-submodules set up there correctly.

Anyway — long story short — there’s finishing the blogging system and then doing a bunch of housekeeping stuff.

In other words: it’s infrastructure week! (And will be for a few more weeks, I expect.)

And then I’ll be back to Evergreen. It should be just one more push of a few months to get it to 1.0.

Here’s the thing, though: the RSS title attribute was always optional. It’s just that RSS readers were written with the expectation that it was mandatory.

If you write an RSS reader with a timeline and detail view, here’s what you could do:

Use the first part of the description in the timeline, after stripping HTML.

Show the post in its entirety in the detail view — but minus a title. No “Untitled,” no synthesized title. Just no title at all.

If you want to see an example, subscribe to this blog in Evergreen. Sure, it’s not 1.0 (or even beta) yet, but it handles title-less posts the way I’ve described above.

It’s the future

Here’s why this is important:

We’re already seeing more and more microblogs, and we’re seeing blogs like this one that have some long posts and some microblog posts. (When you see the word “microblog,” think tweet-like, but with HTML.)

This is an important part of the future of blogging. It’s the movement away from posting to Twitter first — instead, you post to your blog (or microblog) and then, optionally, echo the post to Twitter.

The ability to run the site generator on my server, not just on my Mac.

In other words, I needed to be able to write tweet-like posts with no title — while on the go, on my iPhone or iPad.

I’ve done #1 and part of #2 — now it’s just a matter of figuring out how to deploy the system to my server (which is a shared host on Dreamhost, but where I can run CGI scripts).

The code’s up on GitHub. I don’t really expect other people to use it, but you can, if you want to. I apologize in advance for not having time to write extensive documentation or provide support.

The system’s pretty fast: it rebuilds this almost 20-year-old blog in about three seconds on a five-year-old iMac. The code is, I hope, understandable and hackable, and I welcome you to fork it if it interests you.

* * *

Another part of this: I’ll stop using micro.inessential.com. This blog will be my blog and my microblog. I want just one place that’s me.

I don’t know if I’ll have my posts here automatically echoed to my Twitter account. Maybe. They do already appear on Micro.blog.

PS Titles were always optional for RSS, but most feed readers don’t handle this well. Evergreen was written with this in mind. (It’s not 1.0 yet, though. Working on it!)