Facebook recently released a feature that allows you to download a static version of your Facebook profile which includes all your videos, photos, your wall and a few bits of periphery information. If you’re planning on quitting Facebook, this is great because it means you don’t lose anything and it’s also a nice offline backup of your Facebook info.

What this doesn’t get you, however is all the photos you’re tagged in. For this you need to go through the Facebook API. Here’s how to grab all those photos in the best possible quality in a bit of a hacky way. No idea if it violates Facebook’s terms of service – who the hell knows what they are any more?

First of all you’ll need a system which has wget, grep, sed and awk. You can get these for Windows and this should all work, but I’m going to assume you’re logged into something with a bash prompt.

Find your Facebook ID. If like most people you have an alias for your homepage, click on your profile picture and you’ll see “id=xxxxxxxx”. That’s your Facebook ID

Run the call. Click “Call Method” – you will then see a bunch of code on the right and you will have a URL at the top starting with https://api.facebook.com/method/photos.get?subj_id=. Click on it to open that page in a new window.

Download the data. Either copy the file from the browser window (ctrl+a, ctrl+c, paste into a file) or run wget -O photos.json "<your URL>" to save your data in a file called photos.json

Grab the images. You can now run cat photos.json | sed 's/,/\n/g' | grep src_big | grep http | sed 's/\\//g' | awk -F\" '{print $4}' | wget -i -. This will pluck out the URL of each photo you’re tagged in and download them to the current folder.

You’ll then see wget going crazy downloading all those photos and when it’s done you’ll have a copy of every photo you’ve been tagged in. After that you’re free to stick them into an album, edit them into your existing offline Facebook profile – whatever you fancy.

Root your phone if you haven’t already and install the latest stable CyanogenMod, which includes bash. If this step sounds scary, stop reading now 🙂

Install ConnectBot:

ConnectBot is a fantastic SSH client, but it also has a local terminal emulator. Open it up and select “local” from the drop down list and give it any nickname.

Open it up to see the local shell works. It will, it’s not bash though and it won’t have tab completion. Hit the menu button and disconnect.

Long-press the connection and select Edit host.

Find the “Post-login automation” option.

enter:

su
bash
export PS1="\w\$ "
cd /

This step automatically switches you into root mode (you will be prompted by the superuser manager the first time you do this), starts bash, sets your prompt to the working directory and then changes directory to root.

Hit “OK” and then the back button to return to the connections screen. Select the connection to test if it’s all working.

For quick access, you can add a shortcut to the home screen. Long press on the home screen, select “Shortcuts”, “ConnectBot” then the name of your connection.

Tada! A one-click root terminal in bash with tab-completion (courtesy of ConnectBot’s keyboard shortcuts). Much better than the Terminal app every other blog tells you to install 😉

I have created a simple TwitterClient class (available under SVN) based on my previous set of basic auth and curl-using functions from a couple of years back. I have left it with my app key in their on purpose, so tweets from it will appear to come from “Strawp’s PHP lib”. Register your own app if you want and use those instead, of course. On the first use of a method you will be prompted to authenticate the class against your twitter login. The secret keys will then be stored in your home folder.

Do you ever get people you follow on Twitter who, for some reason have suddenly become really spammy? Normally they’re great, but maybe one evening they’ve come back drunk and are vomiting tweets all over their feed, or maybe they’re live-tweeting some inane reality TV show for the opening night or are taking part in an incredibly uninteresting meme hashtag.

Whatever the reason, you don’t want to ditch them entirely, just until they’ve stopped blabbing on about stuff you don’t care about.

So I made a command line script to get around this. If you want to temporarily unfollow someone, write:

unfollow mrspammy for 1 week

or for several users:

unfollow mrspammy,mrsspammy,mrspammystennispartner for 4 days

Due to the genius of of PHP’s strtodate(), the words after “for” can be pretty much any time period phrase that works with the “+” modifier from the current date. You can also specify exact dates:

unfollow mrspammy until next tuesday

or

unfollow mrspammy until 12 july

This then looks at users that were previously unfollowed, checks if you want to re-follow them by now and does so if required. Run it without specifying any unfollows to just get this behaviour. This means you can also put unfollow in cron (or whatever) to run periodically and it will automatically check if users need re-following by now.

This is written for Ubuntu. Any Linux will be OK, Mac too probably if you have the paths set up properly. You might even get it working under Windows.

I marked that last one up in JavE, which is fine for ASCII art but a bit laborious if you want to build a diagram. On the ditaa home page there’s a link to a screencast of the perl app, Asciio (App::Asciio on CPAN) which is almost perfect. With a couple of tweaks to the /setup/stencils/asciio file, changing all the corners to the ‘+’ character and then another tweak to setup/actions/file.pl to automatically create flat txt and native asciio versions of the diagram, then calling ditaa in the background on the txt file to build a neat PNG version.

I now have a Visio-lite system that can create data flow diagrams suitable for both source code headers and also pasting into more formal documents. So… yeah. How was your week.

I’ve been using a modified original XBox to stream media from around the house since about 2003, however that same box is getting a little long in the tooth – high res videos with a lot of visual noise chug and stutter and even at the best of times the XBox isn’t doing justice to the higher resolution plasma we now have.

So what are our other options? We’ve got a Wii and two XBox 360s. If someone ends up releasing a game on the PS3 we might even get one of those as well.

Wii

This is doable but completely pointless compared to the original xbox. I’d need to transcode everything in to the lower-quality sorenson flash video codec (whatever format it was in, this will be lower quality) and it won’t output at an HD resolution.

PS3

I’m told this works very well as a media playing client and gets along much better than the MS products with other machines on the network (as you’d expect). I don’t have one though.

XBox 360

There are basically two options for streaming to an XBox 360: Setting up Windows Media Center on your Windows-based machine or creating a UPnP (Universal Plug ‘n’ Play) based media server on the network. This is Linux (Ubuntu 8.10) we’re talking about so option one is right out. There are a few UPnP servers for Linux around though. Here’s a breakdown:

This is a very slick service which is easy to set up, automatically scans your media and presents it in an organised manner. It supports video, pictures and audio with a variety of formats and has the ability to transcode between formats on the fly using VLC. Very cool. Works with everything except a 360.
Next!

This is pretty cut-down. The video playback works fine but there is no rescan option at the time of writing meaning you have to restart the service for new videos to appear. Not good if you want to watch something the minute it arrives on the server. This can be partly alleviated by creating a cron job to restart the server daily, but it’s not ideal. The audio library on uShare is also completely useless. It seems to attempt to organise everything, completely fails and only lets you see a handful of tracks in amongst folders which don’t even contain audio. It would have been better if it had just let you walk through the folders like in video mode.

Steer clear of this if you don’t like the idea of not only an old fashioned manual configure, make, make install style setup but having to google error messages in order to even get it visible from an XBox. It’s almost even worth just using uShare and x360MediaServe both at the same time so that at least you have reasonably hassle-free video and audio.

If you do happen to have enough patience and you’ve managed to craft the config files into something workable it turns out that this is actually one of the better least rubbish solutions for streaming to 360. The video works and although you have to hammer on the vfolder.cfg file (which doesn’t get built for you) in order to get something workable going it does kind of do the job in a very basic and raw way. Given a bit more time and a decent setup script this might turn out to be quite decent server, but then given a rescan option and proper music support in uShare I wouldn’t even be trying this out.

Twonky sticks out in this list because although there is a native linux server available it’s locked to free trial only before you have to part with your readies. This proprietary nature of software releases is so foreign to Linux it left me reeling in confusion and I gave up on it before even getting it working. If none of that bothers you then it’s probably the best out of the entire list. I’m a man of principles though. Or a tight-arse.

Samba

Inexplicably notable by its absence is Samba, SMB, straight-forward common Windows file sharing. Every OS on earth understands it and I’d been sharing media using it for years already so why doesn’t the 360 support it? No DRM control? Is it because the protocol has been tainted by FOSS under the evil hand of the EU courts? Whatever the reason, it’s pretty typical Microsoft.

At the time of writing I have FUPPES serving both video and music although I can’t map the 360 media player’s “Artist” and “Album” drill down options to it. After a lot of fiddling I got most video working with a drill-down directory list. I haven’t given up on FUPPES yet but the truth is that despite the lack of rescan and decent music support, uShare is still the better option for video streaming at the moment. Install via apt, edit a couple of lines in the config file and then that’s the whole thing set up. Sure you have to restart it to recache the media list but it’s still less effort than FUPPES is.

Conclusion

If you want to stream music, use x360MediaServe. If you want to stream video, use uShare. If you have a ton of patience or you’re a masochist, use FUPPES. If you want to stream both video and music and you have better things to do, use MediaTomb and get a PS3, or you could try both x360MediaServe and uShare at the same time. Bit messy though, right?

Oh hey – could this be the PS3’s first worthwhile exclusive? 😉

Update:Experiment1106 on Twitter pointed out PS3MediaServer which is apparently the best solution for streaming to PS3s (and claims basic 360 support).

I just got a G1 (A.K.A. The GooglePhone) the other week so I’m trying to pick up Java again so I can write some apps for it. I’m bloody terrible at Java and I’ve had flu for the last week but nevertheless I’ve managed to hack together a class which can download a TV show off iPlayer given the programme ID (PID).

It’s not totally finished yet – I’ve got a warning about not releasing connections there that I need to clear up (answers in the comments please!). You can take a look at the code on my SVN repo: http://svn.strawp.net/java/BeebhackDownloader/

Once this is all stable I’ll be having a go at making a GUI and getting it onto my phone.

As I’m sure you all noticed last week, the latest fun leaked data from a polical party came from the BNP who somehow had their entire membership list leaked onto the internet. For people like myself this presented two fun opportunities:

To do a little bit of geographical and statistical analysis on some odd data

To laugh at a bunch of hate-filled racists

There were quite a few nuggets of statistical analysis in the first couple of days: A proximity checker to see which of your neighbours members, the obligatory google maps mashups (since, sensibly taken down), a sort of heat map and the Grauniad did an excellent map broken into electoral wards. They were all pretty good, but they still rather suffered from the problem that you see “hot spots” in areas which are naturally population nodes. There was no accounting for population density.

Anyway, in a spare moment I took a copy of the database, cleaned up the postcode information a bit, ran it through a geocoder to get lat and long data and then ran that through a lookup for population density and then grouped the data by postcode area. What I now have is a count of each person in a postcode area, divided by the population density – this should then give a population normalised rank of how hate-filled post code areas are. Anyway, here’s the Top 40, Top Of The Pops style:

Creating pipes is a bit of a mental leap from traditional programming.

I noticed Yahoo Pipes pretty soon after it launched nearly two years ago but other than aggregating RSS feeds I couldn’t really get into it. The visual psuedo-flow-diagram programming didn’t gel with me at all and I just found myself thinking I could have more easily achieved the same results in less time in PHP.

Well, after chatting with the Yahoo Developers booth at FOWA the other week and this week failing to find a decent UK Xbox 360 release calendar in iCal format I decided to have another crack using the messy data on Gamestation’s website as a source. The result: Gamestation XBox 360 releases. I’ve also created a conversion script in PHP to make that data iCal compatible and therefore importable into pretty much any calendaring system you’d care to mention.

If you’re used to writing screen scraping software in a real programming language there are a few mental barriers you have to clear in order to get anywhere:

There is no regexp “match” function. You have to use substitute if you want to extract a term and make sure you “.*” at either end of it to remove unwanted text.

You have to program loops one after the other instead of nesting logic all inside one big loop. Makes it a very odd thing to read.

Multiline regexps are a headache to write because you have a single-line regexp input. I got around this by making a multi-line string single line with “s/[\r\n]/” before doing proper regexp operations

It’s really quite buggy. Under Firefox in windows drag and drop operations would stop working after a while and under Firefox in Ubuntu after a while the whole edit area would turn into a bit of a mess. Save frequently.

Anyway, it’s pretty good once you get your head around it and it outsources some of the heavy lifting involved in screen-scraping to Yahoo instead of your own site. I’d like to see the ability to export in iCal format as standard in future updates.