May 2014

I got lots of email, including email from people I haven’t heard from since when I was at UserLand. That was cool.

Everybody has a different idea of what a new Frontier would be like.

There were some common themes. The big one: most people wanted the scripting language to be whatever they’re using now: Python, JavaScript, or Ruby instead of UserTalk.

People also assumed that what I was talking about was writing something like the version of Frontier that ran Manila, that could host tens of thousands of blogs on a single server.

Which made me realize that Frontier has several personalities.

It started out, after all, before the web, as a scripting system for Macintosh. Let’s call that Small Frontier.

In the mid-’90s came Medium Frontier, when Frontier gained web scripting tools and a framework for building static websites via scripts and templates.

Then in the late ’90s came Big Frontier, when it became a server, when it hosted those many thousands of Manila sites.

The question appears to be: which personality interests me?

The Real Question

But the real question is something else: what’s my goal? Why do I want to do this?

It can’t just be nostalgia or a sense of unfinished business. There is a reason.

In the ’80s and ’90s, in the Hypercard era, the original mission of the Macintosh was applied to developer tools. The world of GUI applications, which we take for granted now, was still new, and there was a sense that these applications could be a democratizing force.

People could experience the power of scripting without having to learn Unix and the command line. (Which Macs didn’t even have in those days.) Everything should be visible, point-and-clickable. A database is something you browse and edit in a GUI — it’s not some hidden service.

To me this is lovely. Lovely because, through the virtues of good GUI app design, it brings real power to anybody who wants it.

One of the early principles of the Macintosh revolution, the GUI revolution — making software development available to everybody — seems to be lost. But it shouldn’t be.

So Naive

Well, that was a lovely vision for the ’80s, but history has shown that people want to look at Facebook and cat videos. They don’t want to make things. You’re never going to turn regular people into programmers.

That’s fine. The point can’t be to get everybody in on the fun, because it would never happen. A modern Frontier is still a nerd’s tool.

But here’s my point: it is (or would be) more approachable than most of the tools we have now. There would be some people who get their start programming this way. There would be people building things they wouldn’t have built otherwise.

Some people. Not most people. Not 1% of all Mac users. Some small number.

It Can’t Just Be a Prometheus Complex

Is my motivation to bring the power of programming to people I’ve never met just because I’m a nice guy?

Well, sure.

But I have a political goal. The web we lost is my constant companion, my shadow friend sitting right next to me as I do everything I do.

While “rebooting Frontier” doesn’t show up on the list of things to do to rebuild that web, it’s on my personal list.

As the web has become so dominated by centralization and limitation, I want more and more to build my own little Molotov cocktail and toss it in the middle of it all. That’s what my shadow friend wants. That’s what this is.

In less-dramatic terms: my premise is that if I build a good, approachable, easy-to-learn GUI app that lets people experience the power of scripting the web, that will actually help.

After all, Frontier played a role in building that web in the first place.

But Which Personality?

What does that mean in practical terms? Small Frontier? Medium? Big?

Well, the small version didn’t have anything to do with the web. Medium Frontier is where my heart is.

One of the common things in the email I get about this is the idea of modularity. Frontier was always a monolithic GUI app and runtime, with an http server baked-in.

And there’s a good reason for this — that was part of what made it approachable. You didn’t have to install and configure a bunch of different pieces and do all the crazy things developers do. You just launch an app.

Here’s the thing about the modern world: it could still be a single app and be modular, as long as the various pieces can be separated out and rebuilt and repurposed for other things.

And — because computers have gotten so fast, and because of my own mania for performance and scalability — creating Big Frontier is just a side effect of creating Medium Frontier.

Starting at the Bottom

I don’t have a detailed design. I expect to feel my way along. I also expect not to finish — I have the suspicion that this is just something I like to think about, but in practice have trouble finding time to do. Disclaimer disclaimed.

I also expect to use other people’s work as much as possible. When Frontier was created, pretty much everything had to be done from scratch. That’s not true today.

The first step is obvious: the under-the-hood implementation of a hierarchical, schema-less database.

It should be a library that any app can use. It should have a scripting API. (I’m not sure of the mechanics of this yet. Research required.)

It might sound odd at first, but the plan is to build on top of SQLite. (And FMDB.) Reasons: SQLite is incredibly stable. It’s fast. I have a decade of experience with it. And building a hierarchical database using SQLite isn’t that hard.

The API (still in progress) lets you do things like move and rename things. Get all children of a parent. Etc. Deletes are recursive — delete a table and all its descendants are deleted. Moving something is as simple as changing its parentID.

Changes to the database are reflected immediately (there’s no separate save). There’s an in-memory cache. The database is queried and updated in a background serial queue, but because of the cache much of the time you won’t have to wait for the database.

The idea is that anyone using the database has the simplest possible interface: just add, remove, and edit data via the API, and persistence is automatic.

There should probably be some way to make queries, too. (Perhaps using NSPredicate.)

I’ve written some code. Maybe 5%. If I get it close to completion, I’ll put it up on Bitbucket.

The step after that is probably a GUI app that browses the database and lets you make changes.

After that I’m not sure yet. Something something scripting. Don’t know. JavaScript? Seems like a modern choice, and I’ve been getting to know it. JavaScript is probably the closest to UserTalk, and it means I don’t have to write and maintain a scripting language.

Maybe other languages too? I’m a long way from having to think about it.

There’s also Fever — which is a little different because you host it yourself, and has been around a while, but is worth mentioning (and is very cool).

This is all good news. An ecosystem is better than a single 800-pound gorilla.

At the same time, native iOS and Mac apps tend to use several of these systems as syncing providers. Reeder, ReadKit, Unread, and others support multiple accounts from multiple systems.

Some of these readers have also branched out to include queues from read-later services such as Instapaper and bookmarks from services such as Pinboard.

Why This Is Good

In a way, the situation hasn’t changed — in a way it’s the same as it was when Google Reader still covered the map.

The situation with RSS readers is much like the situation with Twitter clients: there’s no data penalty in switching apps. Just as I can go from Tweetbot to Twitterrific and still get the same tweets, I can go from Reeder to Unread and still see all my feeds and the correct read/unread states.

This is great for users. New thing comes out that you like more? No problem in switching. Like Vendor X’s Mac app but don’t like their iOS app? No problem. Use Vendor Y’s iOS app instead.

Again: this was true during the Google Reader era, and it’s still true.

But during the Google Reader era it might have made sense for a native RSS app developer to create their own syncing system, because relying on a single syncing provider is a very bad idea — especially when we knew that that one system was likely to disappear.

These days there’s no way to justify it. There are multiple providers, and any one or more could go out of business (or just start sucking), and it would be okay, since there are other services.

Furthermore: RSS reader users now have many years of experience showing them that they can mix and match and switch native apps at will. There’s no lock-in.

This leaves native RSS app developers free to concentrate on their app without the distraction of writing and maintaining a server.

And that means better apps. Which is cool.

(Any native app developer also writing a syncing system is making a fatal mistake, since users won’t accept the cost of switching. And they’d be wasting time better spent on making the app itself awesome.)

(Note: the same is not true for podcast clients right now. It’s RSS, but it’s a different thing.)

In general, the UI within sync is intended to be as minimal as possible. It looks and feels like stock iOS whenever possible, seen through a Vesper-y lens. Some apps might use this as an opportunity to show off and have a little fun, but Vesper’s personality is about minimalism and subtlety. Signing in or creating an account should not be a memorable process. Our hope is to make it pleasant, but instantly forgotten.

Of course I was anxious about Vesper syncing day one. My job was to write a bunch of iOS and Node.js code that ought to actually work. My job is also to monitor the servers and make sure they’re happy.

Everything was fine through the betas, of course — but things can change when it’s suddenly many thousands of people instead of dozens.

I’ll invent a day one 0-to-10 scale (because as a programmer I have to start with 0):

0 - everything went perfectly
5 - there were near-constant struggles and moments of panic
10 - had to remove the app from the App Store temporarily and turn off the servers

We ended up at 0.1 on that scale. That’s just about as close to perfection as we’re likely to see on a day one. I’m totally happy about that.

At the same time — non-technical stuff now — we got great response. The app is rated highly, with lots of reviews, and we’ve heard from even more happy people on Twitter and email.

As much as I love it when people write about how syncing is fast and unobtrusive, my favorite part is the feedback that mentions our customer support.

Customer support is all Dave.

Dave is the second-best support person I’ve ever worked with, and I’ve been privileged to work with great support people throughout my career. First best is my wife Sheila Simmons, who did support for years for NetNewsWire and MarsEdit. Everybody else is judged by that standard.

Great support takes empathy, imagination, and brains, and Dave brings all three to the job.

What’s great about indie software is pretty simple: it’s a small team of people who aren’t just doing a job — they’re making, by hand, carefully and with love, the best thing they can make for other people. That focus on other people doesn’t stop with the product itself: it extends to every part of the process. It’s why I blog about development. It’s why Dave makes sure that he does the best possible job helping people.

And yesterday we were reminded, again, that people like indie software. Even with all the changes over the years — App Stores, iPhones, iPads, etc. — people still like supporting the village toymaker.

(And if I’ve got you inspired about indie apps, and you’ve already bought Vesper, go buy Acorn. Or Napkin. Or MarsEdit, so you can blog more. Or Hazel. Or Capo. Great apps by great indies.)

If you follow my blog, you have a good idea of the work that went into it. Writing about syncing was one of the most fun parts of this — and I very much appreciate all the feedback. It made for a better product. Thank you.

One of the things I didn’t write about much, but that I’m proud of, is the performance enhancements. Vesper 1.0 was fast already, and now it’s faster (and uses less memory).

This work was all about getting the data layer — which was rewritten for Vesper 2.0 — just right. (It still uses SQLite plus FMDB, as did 1.0.) I ended up writing the data layer of my dreams. This is a thread I’ve been working on for years, from NetNewsWire through TapLynx and Glassboard through Vesper 1.0 to now.

Another thing I’m proud of is how little actual syncing code there is. There isn’t much room for bugs to hide.

And we shipped with no known syncing bugs and no known crashing bugs. (Obviously I’m tempting fate by saying anything about bugs.)

The Hardest Thing I’ll Have to Do

Writing syncing meant writing a bunch of different pieces. On the client: data layer, API layer, sync merging, sync state management, sync account UI. On the server: data model, API endpoints, sync merging, and a separate site for resetting passwords and account verification.

I knew this would take a long time, in part because I had to learn JavaScript and Node.js. I also knew that I wouldn’t be able to see the app actually syncing until it was pretty close to being finished. It had to wait till all the parts were put together.

It required patience.

Anything else I might do — Vesper for Mac, for instance — won’t be as difficult as this was. Even were we to write a hypothetical (completely hypothetical) second app with a separate web service it wouldn’t be as difficult, because now I have all this server programming experience I didn’t have before. (And I’ve already got a data layer I can reuse.)

Vesper Mac Diary

The Mac version is indeed next. Look for posts on that subject to start soon. Though probably not till after WWDC.

As much as I love writing iOS apps — and I do — the Mac is my natural home. I think of myself as a Mac developer first. The last Mac app I wrote was NetNewsWire Lite 4.0 in early 2011. It’s been over three years. Definitely time to come home.

Vesper for Mac is entirely a UI job. The data layer and API and syncing code already builds for Macintosh. Now, of course, UI is no small thing, not at all — but the challenge isn’t UI plus other things. It’s just that.

Built from the ground up to be flexible and extensible, Mobile Services lets you code your app backend in C# or Node.js. You can save app data either on-premises or in Azure SQL database, blob storage, table storage, and MongoDB.

SQLite4 is an alternative, not a replacement, for SQLite3. SQLite3 is not going away. SQLite3 and SQLite4 will be supported in parallel. The SQLite3 legacy will not be abandoned. SQLite3 will continue to be maintained and improved.

My software runs on SQLite. I’m happy to see that there will be a SQLite4, and I’m happy to see that SQLite3, which I currently use, will continue to be improved.

One of the changes I like is that the primary key is the real primary key. In SQLite3 the rowid was the key into the storage engine, while in SQLite4 the primary key will be the key — tables won’t even have a rowid, except in the case where no primary key was specified. This should be a performance gain.

As mentioned in my previous post, back in the ’90s many people were skeptical about using templates and scripts to build websites.

I wasn’t among them, and I didn’t understand them. I’m still trying to figure it out.

One possibility is this: desktop publishing was still a really big deal. The web hadn’t overtaken print. And the way many people made pages was to fire up PageMaker or QuarkXpress and start laying things out, page by page.

There were some people who used templates and automated systems to create publications. But I think those people were viewed as jerks who just pumped out content without regard for design and for what the content actually was. The only artistically credible way to create publications — print and on the web — was to make each page its own special snowflake. (Perhaps with a style guide and design language, maybe even a super-minimal template, but not more than that. No scripting, for sure.)

I don’t know. I’m just guessing. But it’s the best theory I have right now.

I worked at UserLand from 1996 to early 2002. This article is going to make it sound great — for the simple reason that it was great.

(Obviously this is all from my memory and my perspective.)

1996

I was a fledgling indie. I wrote a few little things that worked with WebSTAR, the Mac http server. One of those was a website searching system called Spotlight, which used UserLand Frontier and Filemaker Pro.

I was becoming active in the Frontier community, which was mainly a discussion mailing list. Frontier, previously an app that cost hundreds of dollars, had become free in 1995.

I worked with Dave Winer, UserLand CEO, on the 24 Hours of Democracy project. My part was an app that took submitted text and created a web page. People told their stories about what democracy meant to them. All the pages were linked together so you could step through them.

Dave and I worked well together and the project was a ton of fun. Not long after that I started working as a contractor. Later — I forget when, precisely (1997?) — I became an employee.

I had no formal training in computer science. I just had what my parents had taught me. I don’t think anybody else in the world would have given me a shot. But Dave did.

Background on the Tech

Kids these days don’t know what Frontier was. To describe it briefly is to gloss over what was delightful about it, but I’ll do it anyway.

It was a developer tool — a Mac app — that included an integrated environment. It was all one piece. It was started in 1989 (I think) — before AppleScript, long before Macs became Unix systems.

It was early NoSQL. It was a schema-less database built on tables. Tables could contain other tables — and scalars, scripts, outlines, text-editing windows, and so on. It was all exposed in the user interface: you could browse the database, open tables in separate windows, open scripts, etc.

Scripts were written in an outliner — which sounds weird until you try it. It was very cool.

The scripting language, UserTalk, was in some ways like a simplified C. Easy to pick up.

Database access was simple — it used dot notation. If an object named “baz” was in the “bar” table which was in the “foo” table, you’d access it as foo.bar.baz.

There were plenty of built-in verbs and suites of scripts. It did IAC — which meant you could script your apps as easily (more easily, I’d argue) as you could with AppleScript. (Frontier was, in a way, the long-promised “programmer’s dialect” of AppleScript.)

It was a remarkable app. You could build really cool things very quickly. I loved it. (Still do.)

Beleaguered Apple

It was 1997, I think, when we had a team: me and Dave, Doug Baron (who’d worked at UserLand in early days), and Bob Bierman. We were working on Frontier 5 — which would also ship on Windows.

Frontier had been a Mac-only app for its entire career. But it was clear that, for the product to survive, it needed a Windows version, since the Mac market share was clearly dwindling.

As a Mac guy this pained me. Those were dark days for the platform.

Doug and Bob did the Windows programming. (Later on we added more people to the team: André Radke, Jake Savin, Robert Scoble, and John Robb.)

I don’t recall when it shipped. Maybe 1998. 1999? It shipped on both Windows and Mac. At that point Frontier became a for-pay app again. And it looked like a healthy product with a future.

I claim credit for nothing

But while that was happening, and as we shipped follow-ups, Dave was on a brilliant streak. It remains my pleasure to have been there for it.

None of it was mine — but I was privileged to often be the first to hear about something, to sometimes talk things over before the world knew about them.

The way invention works in computer science (as in many disciplines), you can’t necessarily always say who invented what. (Sometimes you can.) But there’s no question that Dave invented some things and popularized and fleshed-out other things.

Here’s the sequence, as I recall it:

• Templated websites. AutoWeb, a set of Frontier scripts, was the first example I know of for building static sites with templates and scripts. This was followed by Frontier’s website framework. In the mid-’90s people thought websites had to be hand-crafted. It may seem incredible now, but lots of people — some of them influential and very intelligent — were deeply skeptical about this idea.

• Weblogs. Scripting News existed before the term “weblog” was coined. (Coined by Jorn Barger.) It wasn’t the only such weblog — but it helped popularize the form, and UserLand published scripts that made it easy to use Frontier as a weblog authoring and publishing tool.

• RSS. The original format was a pretty simple array of links formatted in XML. Invented at Netscape for their portal. Dave saw the potential in it — and moved the format forward. He wrote some of the first RSS readers, and RSS output was added to our blogging engines.

• Podcasting. Adam Curry wondered about the “last mile” problem — how do we get rich content to people at home on slow connections? (Recall that broadband was much less prevalent in the ’90s.) Dave’s answer: an enclosure element in RSS. The RSS reader running on your local machine would download those enclosures at night, while you slept. I think it was Adam who came up with the name “podcasting.”

• OPML. Frontier was built in many ways around the concept of outlining, and OPML is a document format for outlines. It happens also to be useful as a way of specifying an RSS subscription list, but that wasn’t its original point.

• Hosted blogging for everybody. I was in charge of shipping Manila, which was an online, dynamic website with blogging and CMS features. It shipped shortly after Blogger. (I don’t think Blogger had hosted blogs at first — it worked by FTPing to your own site.) EditThisPage.com let you set up your own blog. There were “Edit This Page” buttons everywhere — the idea being to make blogs and websites simple enough for everybody to write and publish. To sign up — which was free — you created an account and chose the “foo” part of foo.editthispage.com. (Some people and sites you’ve heard of got their start there: Daily Kos, Joel Spolsky, Robert Scoble, Doc Searls, and so on. This site was originally a Manila site: it was created fairly early during Manila development.) This was long before Tumblr and Wordpress — even before Movable Type.

• XML-RPC. It was like JSON before JSON. The idea was that you could serialize just about anything using XML. It had a date type, which I still wish JSON had. Alas. (Says a guy who’s been dealing with a lot of JSON and dates lately.)

• External website editing APIs. Blogger had a simple XML-RPC-based editing API. We extended it, and then wrote a huge API for Manila, and an external client (written in Frontier) so you could edit just about everything. Not just blog posts but pages and templates too. This is an area where the world is still catching up. (MarsEdit still uses the original extended MetaWeblog API.)

• Desktop RSS readers. The first one that I know was Radio UserLand, which was a blogging system that output a static site and that also included an RSS reader. (On the grounds that bloggers also read blogs, and often want to blog about the things they read. It was because of this that NetNewsWire 1.0 also included the blog editor that later became MarsEdit.) Radio UserLand ran as an app on your desktop but had a browser-based interface. I didn’t work on Radio UserLand that much — that was Dave and Jake Savin more than me. But it was Radio UserLand that got me hooked on RSS.

I’m skipping some things (like distributed database updates via XML-RPC). I’m barely going into detail. An excellent book could be written about this period, about how Dave changed the internet.

2002

By early 2002 I was burned out and in bad shape. I felt like I’d been sprinting for years, and the years of adrenaline had taken their toll.

I was 33 years old by then, and I had learned a ton from Dave and my experience at UserLand. I was ready to go indie again.

First thing I did? Write an RSS reader and a blog editor. But this time in Cocoa. I shipped it in early 2003, and it was a smash hit, and I felt great.

But have I done even 5% of what Dave did to change the internet? Nope. It’s only in retrospect that I realize what an amazing streak he was on. The older I get the more I appreciate it.

Question I still have

How much of this creative period was due to Frontier itself and how much to Dave?

I think it works like this: Dave was creative and productive, and he had built for himself an environment where he could move very quickly and try lots of things. Each layer built on top of other layers, and he was able to work at a very high level.

The tech was his invention too: he built the thing he needed to be able to build other things.

Skepticism

When I was at UserLand I was frequently skeptical of what Dave was working on. But I knew I was young and had little experience, and I was smart enough not to let my skepticism get in the way.

And then Dave was proved right, time after time. (You have no idea how much I disliked RSS for a couple years. I thought it was Dave’s pet distraction, and it bugged me that he wasn’t spending more time helping me ship our other software. I hope I didn’t actually say that to him at the time, but I might have.)

And so I look at what Dave’s working on now — Fargo — and I pay attention, because I know his track record.

UserLand After Me

Jake continued to work there. Robert Scoble, Doug Baron, John Robb, and Bob Bierman left. (Whether anyone was laid-off of or whatever I don’t know.) UserLand was sold — I think? — or somehow Dave left and it was passed to other hands, where it didn’t thrive. (I didn’t pay attention.)

Dave made Frontier available via the GPL license. The kernel is a fascinating historical document. Here’s how we used to write Mac apps. It was minimally Carbon-ized (by Timothy Paustian and a little by me, as the last thing I did at UserLand). But it’s still a WaitNextEvent app. Doesn’t even use modern Carbon timers or HIViews. To reboot it requires rebuilding from the ground up, at this point.

I still keep thinking that we lost something when we lost Frontier as a modern, updated app. It was a remarkable playground, in the best sense, and I’d still love to be able to reboot it as a modern Cocoa app (forget Windows) — because I think the internet would benefit by what people who use it come up with.

It’s inherently geeky, since it’s a developer tool. But at the same time it’s more accessible than text editor + command line + Ruby/Python/whatever. It can give more people a taste of what power on the internet is like — the power to create your own things, to re-de-centralize, to not rely on Twitter and Facebook and Apple and Microsoft and Google for everything.

I have a plan, but it’s long and slow-going. I can’t promise anything.

And maybe that’s just me thinking I can bring about a silver age by using the only skill at my disposal: writing code. But maybe that wouldn’t be wrong, either. Most likely it’s not something I’ll ever complete, which I’ll regret.

P.S.

My co-worker at UserLand and friend Jake Savin moved into my neighborhood a few months ago. I see him twice a month at Xcoders meetings. He works at Black Pixel, which now owns NetNewsWire, the RSS reader I wrote just after leaving UserLand.

P.P.S.

My burn-out at UserLand was in part due to running servers. They were actual machines. We didn’t use a data center — instead, Dave and I each had a T1 to our respective homes. So I had almost a dozen machines running, loudly, in my office. And they required constant care and monitoring.

(Of course, it’s not really the same. I have a couple hosted Node.js sites. No noisy old machines in my office. Giant difference.)

P.P.P.S.

The first time I visited Dave’s office I noted his Eddy award statue for Frontier. I remember thinking: “Wow. I’d dearly love to win one of those myself some day. I think I’d just retire happy at that moment.”

This is probably the last of the Vesper Sync Diary posts — there’s not much left to write about the design. But I should go back and note which things I wrote about are no longer true, and why. I’ll go article-by-article.

In this article I considered switching from SQL to table storage for account data. Table storage is much cheaper, and everybody likes NoSQL these days.

I did not make that switch. I stuck with SQL — for the reasons the article talks about, but also because all my experience in the last ten years or so has been with SQL, and it’s worth sticking with something I understand well.

I haven’t had to do much debugging — but it sure is nice to control all parts of the system.

It does make for craziness where I have Xcode open with the client app, BBEdit open with the API server, several Terminal tabs open, and several Safari tabs opens (server log, for instance). And the app running in the simulator and the app running on my iPhone.

It may be a distributed system, but I can see and affect all the parts on my computer. Sure beats getting frustrated with someone else’s server bugs.

And there’s nothing wrong with that code. (Not exactly, anway — and those lines of code are unchanged even now.) See? I can code late night. Right?

But that code was where I was getting the beyond-bounds exception.

How Could That Be?

Some more details.

The sync server provides a list of noteIDs in no particular order. They might be displayed in any order in the timeline.

The indexes-to-remove might be something nice like [3, 6, 30, 65] — but it’s more likely they’d be in some random order like [30, 6, 65, 3].

It doesn’t really matter whether I loop through those indexes in reverse order or not — it’s going to crash when they’re not sorted.

Think about it: if there are 66 notes, and it removes note 3, then there are 65 notes (0-64), then removing note 65 will be beyond the bounds of the array, since the highest index would be 64.

(And when it didn’t crash it would be removing the wrong notes.)

I would have realized this had I been fresher when I wrote that code. But it was late, and I thought I was being smart. And I missed something.

The Fix

It was after 10 pm when I was working on the fix. I should have quit already and left it for the morning — but, sheesh, there’s no coding task I love more than fixing crashing bugs, and when I’ve got one by the throat I’m not going to let go.

Still, though. After 10 pm. I should have quit.

But I didn’t. And you know what I did? I fixed it. But the stupidest way possible.

I sorted the indexes before looping through them and removing notes from the array.

Well, it works, the code functions exactly as it should. It’s fine.

But why didn’t I think of removeObjects​AtIndexes:, which would have saved me all this hassle in the first place?

Because I really do need to stop at 10 pm. I may think I’m adding productive hours to my day — but I’m not. I’m writing bugs, or, at best, not the best code I could be writing. And I pay for it later.

I have to do some non-coding things. Exercise. Eat. Sleep. I have some leeway on the timing, usually.

Here’s what happens sometimes:

I know exactly what to do next. I haven’t looked at the code, but I can see it in my mind, and the task is clear.

But it’s mid-afternoon — time to go run around the neighborhood. Do I stop right then, or do I do that next thing first?

It used to be that I’d do that next thing. I’d do every single next thing until I ran out of next things, or until I had no choice but to stop.

These days: I put on my running shoes, and leave that next thing for when I get back.

The benefit isn’t so much the self-discipline of it (doing things at the same time every day, which I do like) — it’s knowing that when I return I can get back in the flow quickly and easily, since I have something clear to work on next.

This is a coder’s variation on Hemingway’s advice: “The best way is always to stop when you are going good and when you know what will happen next.”

I think a “soft crash” is preceded by a hang or other indication of a problem. “Hard crash” is boom, app gone.

Ah. Got it.

A soft crash must mean that somehow the app is either 1) trying to recover, and doing so badly, or 2) eating the exception, even though it means things are screwed up, and limping along for a little while before finally crashing.

Soft crashes are very, very bad. You always want a hard crash — there’s an unhandled exception and the app is gone, right away, no waiting. No chance to further screw things up.

One thing: I tend not to use try/catch. There are none in Vesper. If I’ve made a mistake, I want to shut down the app, get a crash log — or, better, steps-to-reproduce — and fix the bug.

(One random caveat. I haven’t used NSXMLDocument in years, but its initWithData method can throw an exception, even though the documentation doesn’t mention that. If I still used it, I’d have to use try/catch.)

Some bit of data goes wrong — I do a thing on my day phone, and then check my night phone and it’s wrong.

Were I just debugging the data layer of a single app, I’d have the app running in the simulator and the Xcode debugger, and I’d have its database open in the sqlite3 terminal app, and bugs couldn’t hide for long.

Computer 1 (my main development machine) also has open: Azure portal, sql-cli app (like sqlite3 but for the sync server’s SQL Server database), and the API server code (in BBEdit).

Clearly a recipe for fun.

So I do a thing here and see if it ends up correct over there. And, when it doesn’t, I realize that enough state has changed, usually in all parts of the system, that I can’t simply re-run that change. Instead I have to try a similar change.

Where’s the bug? Did the change happen locally as it was supposed to? Was syncing triggered? Did it send the right info to the server? Did the server process it correctly? Did the server store it in the database correctly? Did the other app ask the server for sync changes? Did the server reply correctly? Did the other app handle those changes correctly and update its database? Did it update its UI with those changes?

I don’t have any silver-bullet method for debugging all this. I have all the access and data I need — but every bug is a matter of detective work. A single apparent bug can have multiple causes with fixes needed in the client and server code.

Example Bug

Here’s one. The system relies on comparing dates, and I had these two dates:

2014-05-13 04:32:39 +0000

…and…

2014-05-13 04:32:39 +0000

…which turned out not to be equal.

Huh?

Well, timeIntervalSince1970 tells me that one date has millisecond precision, and the other doesn’t. The dates really looked like this:

1399955559.325

…and…

1399955559.000

Not equal.

Were this just a single, stand-alone app with no syncing, a bug like this is unlikely: NSDate’s precision would get used everywhere, and that’s that. I wouldn’t even have to think about it.

Where’s the bug?

The client app might be sending dates without milliseconds to the server.

The server might be stripping milliseconds in the code.

The server database might not be storing dates with enough precision.

The server might be returning dates without milliseconds in its responses.

It could be one or more or all of the above.

The code, on both client and server, is simple and well-factored. For example, there’s exactly one place in the client code where an NSDate is formatted as JSON, and it’s a simple check to see if it’s sending milliseconds or not. So these bugs aren’t necessarily terribly difficult to fix — but it’s still more complex than just doing a stand-alone app.

(Looks like the client code is correct: the format string is yyyy-MM-dd'T'​HH:mm:ss.SSSZZZZZ.)

Anyway: the good news is that I control all parts of the system. The bad news — the inescapable thing, the thing that makes syncing difficult — is that it’s a distributed system with a bunch of parts.

When I first started out as a software developer I assumed that the worse the effect of a bug, the harder it would be to fix. Seems obvious, right? I’d bet the average non-programmer assumes this. It’s intuitive.

I learned quickly, though, that there was no relationship at all between the severity of the effects and the difficulty of the fix.

A crashing bug, after all, could be just because you didn’t add a : character when you needed one. That’s not even a one-line fix — it’s a one-character fix. While a crashing bug is highly dramatic, the fix is often very simple.

I had a bug the other day where a table view was empty when I knew it should be full of data. Definitely dramatic. What’s gone wrong? Is the data layer totally fubar? Do I need a complete rewrite?

Nah — I’d forgotten a ! character in one line in the view controller. Whew.

But then there was this other bug, a UI glitch. It was annoying — obviously wrong — but not at all dramatic. Just wrong. Looked like a quick fix — but it wasn’t. It took hours.

After all these years I’ve come around to this thought: the apparent severity of a bug is in inverse proportion to how hard it is to fix.

(Side note. Question. Why do people sometimes say that an app crashed hard? The app stopped running abruptly. It’s not a crash otherwise. What makes a crash a hard crash?)

Counter-examples

Consider the situation where you have an intermittent crashing bug because you haven’t quite figured out mutability and concurrency and your data layer is, in fact, not good. Then you’d have a dramatic bug (crashing) that’s also difficult to fix (maybe rewriting the data layer).

Or consider the situation where you’re relying on something else that’s crashy. I had a Mac app years ago that used to crash mainly because of WebKit and Flash bugs. (Mostly Flash. Ugh.)

There was nothing I could do about these crashes. They weren’t even fixable (by me) at all.

Revised Observation

Perhaps this makes more sense:

Given a programmer who’s good at their job — who knows the language, frameworks, and runtime well — the dramatic nature of a bug is in inverse proportion to the difficulty of the fix as long as the bug is actually in that programmer’s code.

And that sounds pretty good. Except that there are cases where something is off by one pixel and fixing it is super-simple. So some non-dramatic bugs are also easy to fix, which means this revised observation still isn’t true.

But then there are times when something is off by one pixel — or three — and there’s absolutely nothing you can do since the frameworks won’t let you. (Nothing you can do short of going partial or full Brichter, that is.)

So now, as much as I wanted to make a nice and true observation, I’m left with this:

The more dramatic the bug, the easier it is to fix. 90% of the time. Well, 75% of the time. Well, half the time.

I sat in the back of the bus the whole time and I don’t really know all the details. But here are the parts that I know about.

NewsGator began by calling itself “The RSS Company.” It was founded by Greg Reinacker, who’d written an Outlook plugin that was an RSS reader called NewsGator. (NEWS aggreGATOR, right?)

The company got funding, and it then created a browser-based RSS reader and a syncing system. It acquired FeedDemon in early 2005, which gave it two Windows readers and a browser-based reader. The one obvious hole in the line-up — remember that this was before mobile was huge — was a Mac reader. In late 2005 it acquired NetNewsWire and hired me.

NetNewsWire users needed syncing, and many of them used Windows at work, and so they wanted cross-platform syncing. It was a good fit.

So I went to work. A little while in I realized I didn’t have time to do both NetNewsWire and MarsEdit well, so we sold MarsEdit to Daniel Jalkut.

NGES

But if you have a funded company, you probably need to convince your board that there is the potential for more than just selling software at $25 per copy. You need enterprise money.

Enter NewsGator Enterprise Server. I spent a grand total of perhaps 30 minutes looking at it during my time at NewsGator, so I’m no expert. But the idea was to put NewsGator’s RSS platform behind the firewall.

In addition to straight RSS reading it had some kind of controls for making groups of people. An admin could say, for instance, that all developers are subscribed to these ten feeds, plus whatever else they might want.

I think (memory is hazy, and I really didn’t care) that there was also commenting and rating articles. Sharing too. Intranet-y workflow-y things.

At one point I added a feature to NetNewsWire so that you could put a config file in the right place and it would talk to NGES rather than the consumer system. I’m not sure anyone actually used this feature ever, since Macs weren’t big in the kind of places that installed NGES. (Luckily it was a small feature to add — mainly it just said what the endpoint was.)

Nick Bradbury, on the other hand, wasn’t as lucky — FeedDemon was popular among NGES users, so he had to do a bit more work.

Enter Sharepoint

I don’t know when or how this happened, but at some point NewsGator noticed that the commenting and sharing and social bits of NGES were more interesting than RSS reading itself. It created Social Sites, which was a Sharepoint plugin that provided all that social stuff behind the firewall.

As far I could tell — again, from the back of the bus, where I was working on NetNewsWire — Social Sites was very successful. I don’t recall, and probably couldn’t share, some of the details, except that the company did things like buy everybody a Kindle for Christmas because sales had been so good.

But what about RSS?

Somewhere along the way the decision was made to shut down the consumer browser-based app and the syncing platform. (2009? I think.) NetNewsWire and FeedDemon would sync with Google Reader, since that was what customers kept asking for anyway, and it was just about the only game in town.

Fine. NewsGator wasn’t getting out of the RSS business entirely, but its future investment was limited to just the cost of a few people. (Me and Nick Bradbury, essentially.)

The original Outlook plugin, which had been renamed Inbox, was shut down then. (I think. It didn’t make the transition to Google Reader.)

Nick B Departs

The iPhone came out, we switched to Google Reader, and, not long after, Nick Bradbury departed. I was still there — and NewsGator had treated me very well all along, and I loved working on NetNewsWire.

While the enterprise side of the company thrived, the consumer side dwindled. We came up with an idea to make a library for making other apps: TapLynx. The idea was to use what we knew about iPhone and RSS so that other people could create publications-as-apps without having to do any programming.

We worked with some launch partners: Variety and AllThingsDigital. We shipped some apps. (Those apps are not still TapLynx-based.) People purchased TapLynx licenses and created their own apps. Eventually we realized it needed more resources than we could give it — and better marketing — and we sold it to Push IO. (Joe Pezzillo’s and Dan Burcaw’s company.)

One Last Shot

It became clear around 2010 or 2011 that NewsGator wasn’t interested in paying me just to work on NetNewsWire. They had come a long way from “The RSS Company” — which is totally fine. They made a successful product which wasn’t an RSS app, and that’s okay. (Nothing gold can stay.)

In fact, I say good for them. The software business is difficult, and if the pivot works, you don’t stop just because the original idea for the company was different.

It also became clear to me that I was going to leave NewsGator. The problem, though, was that NetNewsWire was still there.

Walker Fenton and I talked about doing a spin-off — with the encouragement of NewsGator. It included three other NewsGator employees (Brian Reischl, Jenny Blumberg, and Nick Harris) who we thought would be great members of the team.

We called it Sepia Labs. (I like those old-time-y names.) We came up with a product idea: Glassboard.

As we were getting started I was working on NetNewsWire 4. I shipped NetNewsWire Lite 4.0 on the Mac App Store, and a couple months later NewsGator sold NetNewsWire to Black Pixel, when I realized I couldn’t give Glassboard the energy it needed if my time was split.

That was a super-hard decision, since NetNewsWire was my baby. But after nine years it was time to turn it over to a bigger team with in-house designers and more programmers.

(I was extremely proud of that last release. It felt great to do one last good thing.)

So it was full speed ahead on Glassboard. Almost.

Nick B Returns

We needed an Android developer. My first priority once Walker and I decided to do Sepia Labs was to bring back Nick Bradbury.

I loved working with Nick in the past, and — even though Nick was an iPhone user and Delphi programmer — I wanted him as our Android developer. Nick was not eager to be on the NewsGator payroll again, but he accepted since we were spinning out into a separate company. And he quickly turned into a great Android developer, as we all knew he would.

But Glassboard didn’t make much money

The trick with something like Glassboard is that it has to be free because it’s worthless without people using it. But you still want to charge for something. We weren’t the first people in that pickle, and there’ll be plenty more people in that pickle.

It is solvable, but the solution is unique to each app.

We made some progress, but we didn’t solve it.

Surprise!

Though Sepia Labs existed as a company, we didn’t actually complete the spin-off.

Then one day in the summer of 2012 the CEO resigned. (He was the only CEO the company had known.) The new CEO really liked the work we’d done on Glassboard, though he didn’t think the app itself was likely to be financially successful.

So we had a choice:

Complete the spin-off. Take with us six month’s of funding. Sink or swim.

Or redo their Social Sites mobile apps for iOS and Android. Make them at least as good as Glassboard.

There was no way that six months would have been enough to become profitable with Glassboard. (Remember that there were six people. Lots of payroll.)

In all my years at NewsGator I’d avoided enterprise work. But suddenly I found myself as a designer of Social Sites for iOS and Android. (Nick B did wireframes and the Android app. I did Photoshop mockups and assets. Nick Harris wrote the iOS app.)

I decided to leave. At first I thought I could manage to do some side work instead, but that thought quickly passed in favor of leaving altogether.

I stayed long enough to finish what I’d committed to — I delivered a design for Social Sites for iOS and Android. And then I went straight to work — February 1, 2013 — at Q Branch, where I plan to be for the rest of my career.

And that’s the tech industry for ya. Everything I brought to NewsGator is now somewhere else. Everything I created while I was there is now somewhere else. Nothing of me remains there — and all I have from the company is a red stapler they gave me when I reached five years of service. (And some money.)

And even NewsGator isn’t NewsGator anymore.

But even though this is a weird long story about change and loss, I have nothing but good feelings for the people there. They’re smart people who made tough decisions — and they treated me exceptionally well. I wish them continued success.

Last things to note: even Google Reader is gone. And AllThingsDigital.

We don’t have a sponsor for next week (Monday May 12). If you’re interested, get in touch and I’ll make you a deal, since it’s late notice.

This blog is read by developers — especially Mac and iOS — and power users. They’re super-smart people: people you’d hire, people who like cool tools and conferences, people who like learning and reading.

What makes RSS truly powerful is that users still have the control. The beauty of the system is it that no one can force you to be tracked and no one can force you to watch ads. There are no security issues I am aware of and no one ever has to know what feeds you subscribe to. This may be the last area of the Internet that you can still say things like this.

There is, however, old wisdom — perhaps from an earlier version of the Mac HIG — that says how to create error messages: they should be of the form “Can’t x because of y.”

They may optionally include additional detail and/or recovery steps. “Can’t x because of y. Something is true. Try a thing.”

(Best case is when there’s a button that lets you try that thing without having to do it manually.)

A similar form is this: “Noun can’t x because y.” (As in “‘Downloaded.app’ can’t be opened because it is from an unidentified developer.”)

One thing error messages never say is sorry. They’re just reporting, and they respect you enough to know you want the facts, clearly expressed, and don’t need to be apologized-to by a machine.

Also: they rarely (if ever) use the words I, me, my, you, and your.

HAL 9000, were it a Mac, would report: “Can’t open the pod bay door because this mission can’t be jeopardized.” Or: “The door ‘Pod Bay’ can’t be opened because it would jeopardize this mission. Try the emergency airlock.”

I think it’s worth keeping this in mind when writing error messages, but also note that there may be some exceptions. I haven’t thought it all the way through, but a couple things come to mind.

Form-field validation errors. I think you want something succinct next to the field rather than a full “Can’t submit this form because blah” error.

iOS and space constraints. There may not be enough room for a classic error message. Of course, this may mean that the design should be revised until there is room — but that may mean a worse trade-off.

What do you get when you take some of the best Apple dev authors, trainers, and speakers and combine them with the most passionate, engaged developers in a region? You get a learning and networking experience that will not soon be forgotten! You get CocoaConf!

After an exciting, sold-out Spring tour, CocoaConf is gearing up for the Fall. We’re coming to five cities across the U.S. and bringing great speakers — including Brent Simmons, Daniel Jalkut, Daniel Steinberg, Danny Greg, Jaimee Newberry, Justin Williams, Chris Adamson, Mattt Thompson, and Marcus Zarra.

For the newest and best Apple developer technology training, join us in one of the following locations:

I find that this helps me understand quickest. My brain filters out the nonatomic part. Strong is the most common thing and thus omitted, which makes it easier to pick out weak, assign, readwrite, and readonly.

I never use the getter= thing, because I find it easier when the getter and setter have the same name. It’s too much to remember when there’s a custom getter name.

PS I don’t mean that I order the properties in any particular order, though. It could look like this:

If you’re calling something a Handle, it should be an actual Mac memory Handle. And we shouldn’t be using Handles anymore.

All of your class names should have a prefix. All of them.

Header files should expose only what’s needed by the outside world. Use class extensions to keep private things private.

If you’re jumping through hoops to create a cancelable block, consider an NSOperation instead. Or performSelector:​withObject:​afterDelay: combined with cancelPrevious​PerformRequests​WithTarget:​selector:​object:. Or an NSTimer.

Don’t over-comment. No need to say a line is creating a UILabel, for instance, when it’s obvious. Comments are for illumination and are sometimes necessary — but, in general, you should write code that needs no illumination.

You shouldn’t use self.whatever accessors in init, except when unavoidable. Use _whatever instead. (Same thing in dealloc and in custom accessors.)

Turn on more errors and warnings. (Even consider treat-warnings-as-errors.) Peter Hosey’s list is good. It makes your code better — and it means that if someone else uses your code they don’t have to go through and fix things.

If your code is meant for iOS 7 or Mac OS 10.9, use @import.

If you register for notifications, and there’s any code path where you might not call removeObserver, fix it.

My point, by the way, is not to make anyone afraid of releasing their code. Don’t be afraid — releasing code is a great thing to do.

I’ve released dumb stuff sometimes. Totally idiotic things. It’s fine. The point is to learn and get better. That’s it.

Since I’ve never used AdSense and don’t plan to, I’m not personally concerned about the alleged AdSense wrongdoings. But then there’s this:

They decided it was a good idea to alter the statistical data
shown for websites. It first began with just altering data reports for Analytics account holders that also
had an AdSense account, but they ran into too many issues and decided it would be simpler just to skew
the report data across the board to remain consistent and implement features globally.

So what this means is that the statistical data for a website using Google Analytics is not even close to
being accurate. The numbers are incredibly deflated.

I do use Google Analytics, and had no reason to doubt the numbers. Now I wonder.

I had some circles to animate. I’ve been living in server-side land and database land for months — but here I am back to the normal world where we jiggle stuff for a living.

(I want that on a T-shirt. “I jiggle stuff for a living.”)

These circles start out small and get bigger, then small again, then bigger.

So I made them at the normal size and did the transform scale thing to make them bigger — and they were blurry. Blurry little jigglers.

So I did what I always do — which is to go into a tailspin of research (and loathing of computers, this industry, and everybody who was born since 1800). (Well, I went to StackOverflow, like everybody else does. I’m not weird.)

Nothing leapt out at me. Is there anything I can do to make them not-blurry when they get bigger?

The answer, finally, was so simple: draw them big first, and do the transform scale thing to make them small. It doesn’t actually matter that they’re shown as small first — they can still be drawn big.

Problem solved. I take back all the cursing. Sorry.

* * *

The truth is that it probably wouldn’t have mattered. On the big simulator the blurriness popped out at me, but on the device it probably wouldn’t have been noticeable. But I would have known, and that matters.