Menu

Monthly Archives: December 2012

Daniel and I took a week off from Core Intuition for the holidays, but I was a guest on the iDeveloper Live podcast last week for an end of year show with Scotty, John Fox, Saul Mora, Brent Simmons, and Guy English. Topics included highlights from the year’s events, what projects we’re working on now, learning from Apple’s successes and failures, predictions for 2013, and some detours into Auto Layout, Core Data, iCloud, and other new APIs.

Even though I’ve been podcasting for years, I think this was the first podcast I’ve been on that was actually broadcast live. It was a fun show. Check out the iDeveloper Live web site for previous episodes and to subscribe.

As 2012 was winding down, I was fascinated with LongPosts.com (built on top of App.net) and so used it to post some thoughts about Twitter archives. The site is gone now, so I’ve moved the text back to my weblog here, where it belongs anyway. The link to the ADN discussion is here. — future me, January 2016

One of the main goals of my web app Watermark is to archive and search tweets and ADN posts, so it was natural for me to implement support for Twitter’s new archive export format. I finished it last night and linked it from the Watermark account page this evening for all customers.

I had heard that Twitter’s export included a CSV version before I saw the actual files, so I started work coding an importer based on that, with the assumption that I could tweak it later. Once I saw a real tweets.zip, I had to throw out most of my initial work. The CSV files have two problems:

They don’t properly escape values using quotes, so a comma inside a tweet makes the files more difficult to parse.

They don’t include some essential Twitter metadata like the reply-to ID.
I switched to using the JSON files and it’s working well. They’re JavaScript but not strictly JSON, so you just skip the first line.

Since the ZIP archive can be fairly big, instead of uploading in a web browser I let the user choose the file via Dropbox. This was a nice opportunity to try out the Dropbox Chooser. Then on the server I extract the files and load the data.

Dave Winer is doing something interesting with archives too. He’s started linking up other people’s archives on S3 — both the HTML view and the .zip file. I have a test Watermark account that I’ve loaded one of these into. It’s interesting to import multiple archives and have them all merged together and searchable.

For so long we’ve waited for access to our old tweets. In the meantime I’ve shipped two products around fixing this limitation, so it’s especially funny that Twitter finally rolls out archives after I’ve stopped posting there. (And of course I love that ADN has allowed access to your full post history from the very beginning.) Not entirely sure where all this is going to lead, but I agree with Dave Winer that new apps should be possible now.

Since introducing the Tweet Marker $1/month subscriber plan earlier this week, I’ve received a few questions about how the Safari extension works, and whether Watermark customers will also receive the new features. Yes, Watermark subscribers automatically have access to the Tweet Marker extension, which can be downloaded here.

I’ve prepared a screencast to show how the extension works. It’s about a minute long, and you can view it right here.

Thanks to everyone who has already subscribed to either Tweet Marker or Watermark.

The original goal for Tweet Marker Plus was to help cover the hosting costs for Tweet Marker. It succeeded for a little while, but it also ended up evolving into a larger independent service: Watermark, with much higher hosting costs for archiving and search, and a bunch of new features like App.net support, Dropbox sync, saved collections, and more. I’m really excited about the future of Watermark.

I also hear from Tweet Marker users who don’t need Watermark. They still want to support Tweet Marker, though, to make sure it continues running and that it’s as fast as possible.

So today I’m introducing a separate, inexpensive subscription option for Tweet Marker. Just $1/month! You can subscribe from the new Tweet Marker home page. And as a bonus you’ll get the first official Safari extension for Tweet Marker, shown in this screenshot: