Last weekend (29 Jun-1 Jul 2012) I attended Mark Pearson‘s Hackathon London. This was the second I have been to – the previous one was back in March.

The basic format of the weekend is arrive on Friday evening and hear pitches of ideas from attendees. From this, teams are formed and they work on the idea from Friday evening to Sunday afternoon. The weekend ends with presentations and demos from the teams.

I went along with Nathan and Mike. We work together on a daily basis and they both have been to the Python and Android hack days held in my flat.

Friday Evening Pitches

The event starts with everyone mingling/networking. This is the bit of the weekend I like the least, but it was good to catch up with people who I’ve met at previous events. I’m much more a fan of “active networking” which is the main purpose of the weekend – i.e. lets work together on this project, then i’ll know if you’re actually as good as you say you are.

We had an introduction from Mark and a few presentations from sponsors of the event. After that anyone who attended could stand up and pitch an idea. Most of the ideas presented were very good. I ended up joining the Rustlup team – the core idea is to build a business around a better way of finding recipes by using the current contents of your fridge. This was pitched by Troy Kennedy.

Seemed like an interesting idea and from a technical perspective there were lots of potential to get something built over the weekend. We ended up with a team of 9 people to work on the project: 2 designers / front end (Jason and Christian), 4 backend programmers (Me, Mike, Nathan and Graham), 3 business (Troy, Amir, Tunmise).

Over free Dominos pizza, we spent a few hours thinking about what the product would look like and the user interactions. The technical work was split into the following:

Front End Design

Recipe web scraper

Database backend (Django)

Facebook login

Recipe search tool

A few of us left on the Friday to go home, whilst the others cracked on with the project. I went back to stay with my parents in Essex (yes, to the very same town that TOWIE is filmed in) – there were plenty of drunk orange people on the train I tried to ignore as I worked away on my laptop.

Saturday Morning

Arriving back on Saturday morning, some major progress had been made overnight on the design and my arrival woke up Troy and Christian who were asleep slumped over their laptops.

So first thing was to get the server and environment up and running. I have a spare dedicated server that I setup to serve the website and a github repo for our source control.

Scraper

The core of the project relied on being able to get hold of recipes for the content – which we decided to scrape off the web. We discovered that Google indexes these and has encouraged owners to markup their sites using semantic web “Rich Snippets”. This made the task significantly easier and over the entire weekend we managed to get about 10,000 recipies. There were some problems with the data and we had to create an algorithm to extract the clean ingredient name – e.g. “1 (15 oz (450g) ) container mascarpone cheese, at room temperature” becomes just “mascarpone”. Mike and Graham did a great job writing the Python code to parse and get this data.

Lunch

As a team we took a trip out for lunch. Was a lovely sunny day and we headed towards a place we went last time – Thames Take Away, a Chinese on Tanner Street. Being disappointed that it was closed, we went over towards More London Place and the shops over there. Was a nice to break to get out of the office and relax as a team in the sunshine.

Recipe Search

For the core of the idea, there needed to be a search that takes the input ingredients and find recipes with the same ingredients. In the spirit of a hackathon, we decide that we should use Redis to do this task just because we knew it would be fast over lots of recipes – and it’s a piece of cool tech that none of us had used before. Mike did most of this work but unfortunately we ran out of the time to get it properly working and integrated with the site. It works reasonably well for multiple ingredients but “lemon hazelnut tiramisu” seems to appear as the top result far too often.

Bar Code Scanning

Throughout the weekend, we were continually discussing the best way of the getting the user’s ingredient list into the site. Using the site to manually type in items is a little difficult – especially on mobile devices. So we spent some time investigating using a mobile to scan the products.

Late Night Coding

First off we looked at using the bar code. But following a trip to a local Sainsburys, we discovered that for own brand items, the bar code has a custom (presumably internal) number on it. We also looked at the Sainsbury’s online shopping site, but none of the numbers consistently matched up. After the weekend, I did some more research and found that we could have done it with Tesco own brand as the bar code on their products can be found on their online shopping site.

We also tried using a mobile phone to take a picture and then OCR the image to extract the name of the ingredient. But the online OCR tools didn’t recognise the text well enough. Even if it did work, we’d have a big task to work out actually what the key words were on the packet.

Android App with Pusher

Some time in the early hours of Sunday, Christian and I decided it would be a good idea to investigate using voice recognition to help with the input.

I found a tutorial showing how to use the voice recognition software built into Android, so I created a quick app to do this. From testing it detects words remarkably well, although some food words are problematic – it missed most non-English words and it sometimes detected “pasta” as “pastor”. But we knew it worked good enough for a controlled demo.

After getting the recognised words, we need a way to get this to the website. Since Pusher sponsor the hackathon, we decided to use their real-time messaging service to hook the app to the web browser. We adding a Javascript library to the main page that then was sent data from the Android app. This then used JSON calls to lookup ingredients in the database. Luckily there were libraries in place for all of what we needed – although the documentation was a little incomplete for Android because it was a 3rd party library. When we got stuck, Phil from Pusher was on hand to help us out – even at 3.30 in the morning!

So we ended up with an Android app where you could say ingredients (limited to one word each) which would then be input into the site. Saying “rustl” (recognised as “russell” by the software) would submit the page and give the search results.

Final Integration

Sunday Morning

After getting some sleep I had the task of integrating all the code we had together. This was frantic to get all the design changes, the Redis search code and all the crawled recipes onto the main site. Mike was running around like crazy using all our machines to processing the 10k scrapped webpages and getting this into the database.

But we eventually got it done for the 4pm deadline and ready for the demo.

Demo

The weekend ended with everyone congregating around the presentation area and each team giving a 3 minute presentation and 2 minutes of questions. In the end there were about 25 presentations (and we were about number 22).

Rustlup home page

Troy from our team give a short overview of the business side of Rustlup and then Christian and I gave the demo of our website. We showed off the search results and then the voice recognition demo. Of all the teams that presented, we were one of the few teams that showed off a working live website and app.

Prizes

Presentations

After a short break and packing up our stuff, we returned back for prize giving. Rustlup was a double winner!!! 2nd prize (Mentoring and £40 Amazon voucher) and the Pusher prize (which gives us a 6 month startup plan worth $294).

Conclusion

Having attended the previous Hackathon London, I was impressed with the improvements they had made in the running of the event. The Friday evening pitching and the Sunday presentation was much slicker and organised.

As for the Rusltup team, I was impressed with how much we managed to get done in the time, how everyone helped each other out and how much drive, energy & focus there was to get the task done. Plus we had loads of fun joking around and playing with exciting bits of technology.

Forming up a team in such a short space of time that worked so well together, is a testament to the quality of developers that these kind of events attract. I have been in other dev teams that had been together for months that never worked that efficiently.

Many thanks to Mark and his team for organising another brilliant weekend. I’m looking forward to going to the next one – when ever it is.

Want to get your Python pep257 docstring errors in a Jenkins job? Then you can use the Jenkins Warnings plugin with a custom parser. These can be added under “Manage Jenkins” > “Configure System” and in the “Compiler W...