The Problem

I mainly use Twitter as a news reader. I follow some big news sites (like @NZZfeed), technology sites (@TechCrunch, @wired), entrepreneurs (@elonmusk) plus a few friends. The problem is that a few of my followees generate so many tweets that I struggle to keep up. And I regularly miss tweets from my friends, because those tweets are swamped with the large volume of tweets from the big news sites. So I would like to have a Twitter client that shows only interesting tweets.

A tweet can be retweeted – if someone reads an interesting tweet, they might forward this tweet to all their own followers. They retweet the original tweet on their own timeline. Assume you have just read a very interesting, inspiring, shocking, or newsworthy tweet. You decide that all your followers should see that tweet as well – you retweet the tweet to your own timeline. Of course, other people have different interests than I do, and they might consider something interesting which I just find boring. But the whole crowd is probably not so wrong. If enough other people find a tweet interesting, I’m probably also interested in it.

The Idea

The way I want to distinguish between uninteresting and interesting is by the tweet’s retweet count. Twitter allows to query the retweet count for each tweet. For every tweet in my stream, I want to see how many times it was retweeted. Based on the retweet count and the time elapsed since the tweet was originally posted, I want to determine if the tweet is worth reading.

A threshold function should give the criteria, whether the tweet is interesting or not. The first criteria of the function is time:

Time: The first input to the function is the time elapsed since the tweet was posted. Right when the tweet is posted, it has no retweets. For a few hours, the tweet might be in the spot light and get retweeted. The number of retweets follows probably a monotonically increasing curve. The steeper the curve, the more people have retweeted the news, the tweet is probably worth reading.

Based on the time we can define a threshold curve. If the retweet count of the tweet lies above my threshold curve, then the tweet is interesting and I want to see it. If the tweet is below the curve, it didn’t get enough retweets, so it is probably not interesting enough to waste my time on it. This solves the problem that news sites post a lot of articles which I am not particularly interested in. However, the second problem is still not solved: I don’t notice my friend’s tweets. The threshold function actually makes this worse, because my friend’s tweets don’t get retweeted normally.

“Steepness” Parameter: The second parameter of the threshold function controls the slope of the threshold function. For my friends, the threshold function should be very flat (or constant zero), while the news site should be evaluated using a steeper curve.

At the moment, the curve I use is quite simple. I assume the tweet gets retweeted linearly during the first four hours, and once it is 4 hours old it will not be retweeted anymore. The threshold function looks like this:

The tweet does not have enough retweets to make it through the filter.

The Solution

I have implemented this function in my own Twitter client, the “Twitter Summarizer”. You can access the Twitter Summarizer at twtr.rxtx.ch. The client is based on the Fork-A-Twitter-Client, which is implemented purely in JavaScript. I have forked and extended their client and implemented my threshold function. The client runs in your browser, so there is no need to sign up or register at my website. You only need to approve the connection to Twitter, then the client directly connects to Twitter and fetches your tweets.

Using the “Mute Settings” link you can configure the threshold curves yourself. You can define a default steepness parameter which applies to anyone, and you can control the parameter per-user.

Some Limitations

As explained above, the threshold function is pretty basic at the moment. A tweet will get more retweets after the first 4 hours, so my curve might be wrong at times. I am trying right now to find a better curve.

Because my Twitter client uses Twitter @Anywhere for the authentication, you can be sure that all data remains in your browser. There is none of your Twitter data (username, password, tweets, etc.) stored on my server. The disadvantage is that you can’t stay logged in for very long. Next time you access the page, you will have to hit the Connect with twitter button again. Also, your preferences are stored in your browser’s local storage. If you switch to a different browser or device, you will have to configure the client again.

Because of Twitter’s API limitations, we can only query the most recent 800 tweets of your stream. Because some of those tweets are filtered out, your stream might look quite short.

Let me know if you’ve got any problems with the Twitter Summarizer or ideas how to make it better!

In January, I messed up the setup of one of my web sites. I misconfigured the privacy settings in WordPress and locked out all search engines. Of course, my website started to drop out of Google’s top results pretty soon. The results were nearly fatal, with an upcoming event depending all on the website and it’s position in the Google search results. Luckily, I was alerted by one of my colleagues a few days after my mistake and we could get our site back into the Google results within a few more days – just in time for the event.

However, I wanted to build a tool that would alert me next time much sooner, if one of my websites would start to drop out of the Google results. Basically like Google Alerts, frisk monitors the Google results for a given search term. But instead of notifying you whenever there is a new hit for your search query, frisk alerts you as soon as your website is moving downwards the Google results.

In frisk, you will have the possibility to define multiple search agents. You specify the search query, which website you expect in the results and at which rank you expect that site to appear. For each agent, frisk will query Google once a day and let you know if your website is not in the results anymore. And frisk is for free! You will have to provide your Google Search API key and frisk uses that key to make one search per agent per day. Obtaining such a key is free (for up to 100 searches per day) and pretty easy. Simply follow these steps:

]]>http://benediktkoeppel.ch/blog/2012/07/frisk-inverted-google-alerts/feed/0Samsung Galaxy Nexus with JellyBeanhttp://benediktkoeppel.ch/blog/2012/07/samsung-galaxy-nexus-with-jellybean/?utm_source=rss&utm_medium=rss&utm_campaign=samsung-galaxy-nexus-with-jellybean
http://benediktkoeppel.ch/blog/2012/07/samsung-galaxy-nexus-with-jellybean/#commentsWed, 11 Jul 2012 06:12:36 +0000http://benediktkoeppel.ch/?p=1006On Monday, Google has officially released the source code for Android JellyBean 4.1.1 in AOSP, the Android Open Source website. From there, everyone can download the sources and build his own version of Android. I’ll explain in the following paragraphs how I built Android JellyBean for my Samsung Galaxy Nexus phone.

Getting started

My Samsung Galaxy Nexus is a GSM/HSPA+ version, officially called Meguro, not rooted and was shipped with yakjuwx OS installed.. You can find out which hardware version and software version you have by installing Android System Info. In the System tab, you will find the hardware version. If it is not Meguro, then don’t follow this guide! If your phone is shipped with the yakju OS installed (not yakjuxw) then you can skip part of the instructions.

A word of caution: following this guide might brick your phone, or you might lose data.

Java SE 6

For the Android SDK and to compile Android itself, you need Java SE Development Kit 6. Download it from the Oracle website (I took jdk-6u32-linux-x64.bin). Install it with the following commands. In the last two commands, you will have to enter the selection number of /usr/lib/jvm/jdk1.6.0_32/bin/java and /usr/lib/jvm/jdk1.6.0_32/bin/javac:

This will install all the needed tools in /usr/local/android-sdk/, setup the environment and bring you to the official Android SDK updater in the end. Make sure that you install the following tools:

Android SDK Tools

Android SDK Platform-Tools

everything from Android 4.1 (API 16)

Once all tools are installed, launch the Dalvik Debug Monitor Server by typing /usr/local/android-sdk/tools/ddms in a terminal.

This should give you a running Android environment on Ubuntu. Open the settings on your phone, and open the developer settings to enable USB debugging. Then plug your phone into your PC. To verify that the Android toolkit is setup properly and can find your phone, run adb devices and verify that you see your device:

Backup

First of all, take a backup of your phone. You can do this with adb without rooting your phone. The adb backup tool has many options, you can get a list of supported parameters by running /usr/local/android-sdk/platform-tools/adb help. You want to save all shared data and all installed APKs:

Now unlock your phone and confirm that you want to run the backup. If you have turned the device encryption on, then you’ll now have to enter your encryption password. This will create a new file called ics-4.0.4-backup-*.bkp in your home directory. My Nexus had about 9 GB free space (of total 13.3GB). It took me two hours to dump the backup, and the backup file had a total size of 5.9GB. (While you wait for the backup to complete, you can already start compiling…)

In the Dalvik Debug Monitor Server (/usr/local/android-sdk/tools/ddms) you can inspect the log file for the backup. In the lower pane of the DDMS, filter for “backup” and set the log level to verbose. Once adb backup has finished, make sure that you don’t see any backup-related errors. In my first attempt to take the backup, adb aborted after about 45 minutes with a stack trace in the log file. The 2nd attempt worked fine.

Switch to yakju

Google officially releases the yakju version of Android for the Samsung Google Nexus phones. However, Samsung has shipped my phone with yakjuwx installed. This basically means that my phone does not receive or accept OTA upgrades from Google directly, but only from Samsung. If your phone is already running yakju, you can fast forward to to Compiling Android JellyBean.

Configuring USB Access is not required. Linux und ich’s installer already created the required Android rules. Also, ccache is not required but saves you time if you want to compile Android for multiple platforms (or the emulator too).

The repo sync command will take a long time to download all the required sources into your ~/ANDROID directory. For me it took about 45 minutes.

Now don’t forget to download the proprietary drivers. From the Google driver page, download the drivers for Maguro, JRO03C (Android 4.1.1). Move those files into your ~/ANDROID directory and unpack them there:

Now you have all the proprietary drivers in ~/ANDROID/vendor. When we compile the source code, make will find the drivers there.

Building the System

Let’s build JellyBean!

cd ~/ANDROID/source build/envsetup.sh

You have to select the target. We are building for the Galaxy Nexus, so the build name is full_maguro. The build type defines how much debugging options you will have, I chose userdebug.

lunch full_maguro-userdebug

Then build the code. On a quad core use -j 16 or -j 32, on a dual core -j 4 or -j 8 for the fastest results.

make -j 32

On my Core i5 M520 @ 2.40GHz it took 3.5 hours to compile the complete system. Make sure that make did not end with any errors. If you don’t see any errors in the last 20 to 30 lines you should be fine.

To flash your custom built Android to your phone, set the Nexus again into fastbood mode (reboot with the two volume buttons hold down, or run adb reboot fastboot). Make sure it is connected correctly to your PC: