My father-in-law is obsessed with Twitter poetry, or tweepetry, as he calls it. As someone incapable of succinct expression, I’m a great admirer of the masters of the 140 character quip, foremost among them the inimitable Anderson Miller and the by now overfollowed Shitmydadsays. I’m not sure anyone really knows what the point of Twitter is, but it does lend itself to recording felicitous turns of phrase, so I applaud my pop-in-law’s to up the literary ante.

As a gift last year, I made him a site that aggregates tweepetry using Seaofclouds’s nifty little Twitter-scraping javascript. The concept was super simple: tag any tweet with #tweepetry and it automatically appears on tweepetry.com. I added a couple of snippets to the code to remove the hash tag and done!

It worked great for a couple of weeks. Then the bits of tweepetry that had appeared on the site began disappearing. After three weeks, they were all gone. I did some research into the Twitter Search API and it turns out that only the last 10 days or so are indexed. Anything older than that can no longer be fetched dynamically using search. So tweepetry.com sat empty for six months gathering internet dust.

After three months of database programming, I realized I’d amassed the tools to resolve this problem so I revisited the site, rewriting everything but its visual elements from scratch.

HOW IT WORKS

The site is still dynamic, but the dynamism is all backstage.

There’s now a php script running on my server that polls Twitter every 10 minutes (the maximum number of searches from one IP is limited to 150 per 24 hours). It talks to a MySQL database with two very simple tables, the first to recorded the id, username, time, and text of a particular tweet and the second to keep track of the id of the last tweet. I used two tables to keep my database queries fast. Every time the script runs, it searches Twitter for tweets that include the “#tweepetry” hash tag. If it finds some, it compares the id of the latest with the id stored in the second table. If they’re the same, nothing happens, but if they’re different, each tweet is stored in the database until the id of the last recorded tweet is reached, at which point it is updated to reflect the new additions. Then there’s a simple script that fetches all this information and displays it. That’s it!

This entry was posted
on Wednesday, December 2nd, 2009 at 1:14 pm and is filed under Random. You can follow any responses to this entry through the RSS 2.0 feed.
Both comments and pings are currently closed.

Comments

Comments are closed.

CONTACT

E: alex at chinaalbino dot comP: (six one seven) five one five eight eight three eight