As a small side project, I recently launched a “link unroller” service. This is a very simple service. You give it a URI, and it follows any redirect chain for you. Then it spits out the final URI via a friendly JSON API.

Toki uses an App.net sync mechanism that I’ve been thinking about and working on for a while — I thought that Toki would be a good low-profile app to test out some ideas in real-world use.

Although Toki is a production app, the sync portion is not feature-complete yet. I’m working on ironing out the basic bugs (planned to be released in 1.0.1 and 1.0.2 releases), before moving on to the more advanced features. My “end goal” for the App.net sync mechanism is an open-source multi-platform library that anyone can use in their own app to use the App.net backend to keep their objects synchronized2. From now on (and until I think of a better name), I will call this sync mechanism “App.net Object Sync” or “ADNOS”.

There are two fundamental steps of the sync protocol (as of version 1.0.2, still in development):

Pulling remote changes: ADNOS pulls all new messages from the channel, processing them one by one, creating or updating local records as necessary. If the record is deemed “duplicate”, it will overwrite local changes3. If the record was newly created in the local database, ADNOS will also record the App.net Message ID in the local database.

Pushing local changes: ADNOS makes a new message post for each record that does not have a App.net Message ID recorded in the local database. When the message is successfully created, the message ID is set.

This has some limits, some inherent in the protocol, and some inherent in the App.net backend:

Recreating the database requires a full playback of messages — this takes time, especially on databases with more than 200 objects.

These limits are pretty much unavoidable. I work around the rate-limiting problem by queueing sync up requests, and pausing the queue when X-RateLimit-Remaining gets dangerously low. The queue is re-started after X-RateLimit-Reset elapses. Any app that plans on using ADNOS will want to be aware of these limitations before implementing it.

And, now for the features I’m planning on implementing (in no particular order):

Proper conflict resolution.

Streaming support for pulling new changes.

Updating and deleting previously synced items. This isn’t implemented yet, but should be relatively easy.

I’ll be writing more as I progress. I hope to get some sort of open-source library out there in the near future.

Retina, Please! is that JavaScript snippet on steroids. It is a combination of JavaScript and PHP to allow your Retina users to only download Retina images, while non-Retina users only download the standard-resolution version.