Trying to find useful things to do with emerging technologies in open education and data journalism

Wrangling Data With “Free” Tools – LASI13 Workshop Round-Up

I’m fortunate enough to be visiting the LASI13, the Learning Analytics Summer Institute, in Stanford this week, and got to lead a workshop session yesterday on tools for tinkering and playing with data.

The presentation I prepped can be found on Slideshare – LASI13 datawrangling tools – though as ever I didn’t get through all the slides, and, as ever again, went slightly off-piste at various points. (The session was a 2hr 15 session, split 1h30 and 45 mins; I reckon the whole slidedeck would be a 4hr session; as it was, we got as far as grabbing data out of Facebook and into OpenRefine, with a v brief tease about starting to analyse the data in Gephi.)

I mentioned several tutorial posts and resource pages in the session – here a few links to some of them:

if search limits for use in Google searches are new to you, I like site: for searching sites or domains (eg site:open.ac.uk or site:edu); filetype: for searching by document type (eg filetype:xls or filetype:pdf); for limiting by document titles, intitle: and for limiting by terms that appear in a url, inurl:

Google charts playground – Code Playground (go to the Visualization API). If writing the code is too hard, (which I think it is;-), appropriate R and the googleVis library (though I don’t think it supports dashboard construction… yet…?). If you want to generate HTML/Javascript for other charting libraries, including libraries built around d3js, try the rCharts library.

If you want to follow through on OpenRefine/LODRefine, there’s lots more to know and I’m still working through tutorials to cover some of that. Grabbing Facebook friends likes using OpenRefine varies slightly from the w/s recipe; I’ll post a new version over the next week or two. For now, if you want to parse the JSON data, the magic phrase is forEach(value.parseJson()['data'],v,[v.category,v.name,v.id].join('::')).join('||'). You then need to Edit cells – split multi valued cells (by ||) then Edit column – split into several columns (using :: as the separator). I’ve posted a whole host of OpenRefine tutorials using the OpenRefine category on the this blog.

If anyone still here at LASI would like to chat further about data related skills development, let’s grab a table… similarly if you’d like to learn more about the School of Data data expedition model as a possible learning or training exercise. Finally, if you’re looking for folk to run data skills workshops, let’s talk;-)

PS if I’ve missed any links you think should be here, let me know and I’ll add them…

… and if you want a cRunch account so you can run R magics on a server with a bit of memory and a bit of processing power ;) – just fill in this application form: http://crunch.kmi.open.ac.uk/contact.php