This project aims to bring some of the affordances of consumer social networks to teaching and learning, and will deliver applications within CamTools, our Sakai-based VLE. This is an informal blog by the project team at CARET, University of Cambridge.

Monday, 16 November 2009

We are starting to think in depth about our Assembly, for the JISC Institutional Innovation programme. We'd like to have a topic of "Personas and User Testing" as we think user testing would be of interest to others at this project phase.

Tentative dates might be 8th, 14th or 15th December, for an assembly roughly 11am-4pm at Cambridge. The programme will include:

Personas and their place in user research and testing (presentation by Cambridge)

10 minute micro-briefing "Post-It Notes and their Special Place in research"

Our experiences of user testing as part of a user-centric design process (presentation by Cambridge)

Best practices in user testing (group brainstorming, to be documented online for comment)

Hopefully some presentations from others about their user testing experiences :)

If anyone would like to join us in this Assembly, please email us: laura@caret.cam.ac.uk and asd38@caret.cam.ac.uk and let us know if you might be able to bring a presentation, and which dates would work for you.

Tuesday, 1 September 2009

As promised here is the 100-pages document which has finally been finished (in fact, it's a bit longer than 100 pages).We even made a 20-page version of it in case you don’t want to go that much into detail.

These documents should give you a detailed overview of our project, telling you all how we went about doing things like research and design.Our colleague Clay Fenlason of Georgia Tech even asked for this 20-page document to use as a use case example during his teaching.

Friday, 24 July 2009

As Laura James said, the project was presented at the Sakai Conference in Boston. Preparing a 45 minute presentation is surprisingly harder as it seems. I was glad we had so much time to present it, but even 45 minutes still didn't seem to be enough to present everything...There's just too much to say about this interesting project!

When doing the presentation, people seemed to react really enthusiastic about it. Some people were familiar with bits from the used methodology like personas or scenarios, but were surprised by the colourful walls covered in post-its notes and the fact lots of methods were now used all together. Some people seemed to recognize themselves in the personas! Cool! Others made some interesting comments, like: "At this moment, those 3 personas seem to be the right set of presentation for those people, but people behaviours change, so it might be possible that there are new behaviours and also new personas coming up within X years. How would we adapt the system to those differences?" Definitely worth thinking about I would say!

Some people were so interested in our personas (or simply in the methodology of creating personas), so we promised to share them.Are you also desperately longing to have a look at our personas or scenarios? Then don't hesitate to have a look!> Personas, scenarios and the methodology of creating a persona

Everyone knows documenting can take a lot of your time. I've been busy for some time now and still I have the feeling we've not covered everything. Sure, these documents are mainly to make sure we're not forgetting anything because it's so easy to forget about something. This means they're actually still more drafts than final documents.

Beside some general writing ups, Tjhien, Oszkar and I thought about writing the whole methodology up within google.docs which didn't seemed to be ideal afterwards but is still helpful if you need to write things up in a collaborative way. We tried to cover all the details in this draft document and we ended up with...more than 100 pages! Really, you don't WANT to read this yet. It's still a draft.

We learned a lot though, even just by documenting. For example: It's really hard to explain certain methods through the medium 'text'. So we tried to start making simple videos of some methods we used.

They're not yet finished, but these are some examples to give you an idea:

How we did task-goal analysis (Research phase): During the research phase, we wrote down the things participants mentioned during the interviews. We captured this in an unusual but extremely helpful way, being: writing down every entry on colour-coded post it notes. Like this it would be easier to sort this information afterwards. Just have a look how we did that.

How we did user testing (Design phase): We made some designs of concepts and of course, we wanted to test them with real people to see how they would react on them. Are we still doing the right thing? Therefore, we used a double mirrored room with in one room the participant and a facilitator and in the other room the people who would observe and analyze the data. During the first iteration, participants were looking at paper prototypes of the designs (which you can see in this video) and in the second iteration, they looked at wire frames visible on a real screen.

Friday, 26 June 2009

We last posted when in the throes of our design phase, with iterations of prototyping and user testing - very intense. We managed one round of conceptual design, paper prototype testing with users, a design combination and refinement, wireframe testing with users, and then final refinement.

We are now very proud to have a set of wireframes for one overall design concept which we have created through a full user-centric design process!

Since then we've been working on capturing our work and recording the details of what we have done, how others might do similar things, what results we've found and what we learnt, for future dissemination. The incredible density of activity through our research and user-centric design processes has lead to a huge amount of information, and crystallising that into forms which are useful to ourselves and others has been time consuming - but worthwhile.

We are also looking ahead to the next phases of our project; potentially another round of design and testing, and then integration of the system into new Sakai. Work on the backend engine of new Sakai ("K2") progresses apace, and we're also building some basic networking features into our user interface for Sakai3 as the first hint of what academic networking might become. This does not yet include the full power and excitement of the concepts this project is generating...

Meanwhile, the JISC Academic Networking project will be presenting in two sessions at the forthcoming Sakai conference in Boston, July 8-10th 2009. Look out for John Norman and Anne-Sophie de Baets there!

Sunday, 17 May 2009

Creating extreme designs using personas and requirementsWe focused on the concepts and requirements which we thought would be most useful and used those to brainstorm and think of ‘extreme’ designs.This means we were using these extreme ideas to turn them into metaphors which were much easier to use during the design.E.g. The metaphore or extreme idea of ‘ballroom dancing’ explained the idea that people sometimes spend time together during a project (so dancing together for a while), after a while they split up and work with someone else (splitting up during a dance and start dancing with someone else in the ballroom).

Extreme ideas turned into metaphors and clustered in groups which seemed to have commonalities

User testing session 1 using paper prototypesAt the end of the research phase we ended up with 3 personas. For each of those personas we created a set of frames (representing the pages within a system) so we ended up with 3 main concepts.These frames are paper prototypes. This makes sure people will give us more honest feedback (and won’t say things like ‘Hmm, I don’t like the colour of that button’ because they’ll see it’s drawn and not finished at all). We wouldn't be focusing on that kind of detailed feedback yet anyway.We recruited a set of people again, representing all the different people within the university (subject, age, role, etc). Whilst showing them the paper prototypes during this user testing session, we also used a set of questions so we were sure we kept focus on the right things. The location was in a usability lab, having a one-way mirror where the observers took notes on the other side of the mirror. This feedback would than be used to refine the concept.

Green concept: focusing on Isobel who’s really sociable, and outgoing. Therefore the frames of this concept focus on the extreme idea of ‘events-going out’ as this is very important to Isobel.

Green concept paper prototypes- homepage -

Blue concept: Focusing on Peter who sometimes feels a bit lost. He doesn’t always know what to do first because he’s not always talking to the right people either. Therefore the frames of this concept show the idea of the ‘boardgame’: representing a set of steps you need to fulfil in order to reach your goal.E.g. In order to fulfil this course, you need to go to this talk, finish this paper etc.

Blue concept paper prototypes- homepage-

Red concept: Focusing on Kate who’s senior and just wants to save time in any possible way. This set of frames shows the extreme idea of ‘the switchboard’ which makes is possible for Kate just to see the things she’s interested in, not overwhelming her with things which just waste her time.

Red concept paper prototypes- homepage-

Refine concept using feedback from user testing session 1We did some brainstorming and used all the feedback from user testing session 1 to merge all the screens and thoughts from the first set of screens into 1 set of screens, representing all the refinements and usable ideas.

Example of a page which got all the feedback merged together

Oszkar redesigning

User testing session 2 using clickable wireframesThis phase was very similar to the first user testing session, which means it was again in the same usability room, with an observer who noted down the feedback. The difference this time was that we used clickable wireframes instead of paper prototypes. This was 1 set of frames instead of 3 because these were merged together. Further, we used again a questionnaire which guided us through the set of tasks.

clickable wire frames- homepage -

Refine concept using feedback from user testing session 2We used the feedback to again refine the last version of the frames.

Documents In the text above, you can find the documents we used during the design phase which might give you a better understanding of the information above.In case you missed out on them, you can also find them below.

User testing session 1 using paper prototypes> General information about the concepts - This gives a general description what's meant with each of the concepts> Green concept - This is the full set of paper prototype frames used within this concept - mainly focusing on Isobel> Blue concept - This is the full set of paper prototype frames used within this concept - mainly focusing on Peter> Red concept - This is the full set of paper prototype frames used within this concept - mainly focusing on Kate> User testing guide during session 1 - This is the set of questions which we used as a guidance during the user testing session.

User testing session 2 using clickable wire frames> Clickable wire frames - This is the full set of wire frames used within this merged concept and which we also showed to participants during user testing session 2- it contains all the merged feedback and refinements we got from user testing session 1> User testing guide during session 2 - This is the set of questions which we used as a guidance during the user testing session.> Sitemap - This is a visualisation of all the different main pages within the system. A sitemap helps you during designing as wel as to focus on the right tasks during the user testing session.

Refine concept using feedback from user testing session 2> Final wire frames - This is the full set of wire frames which were the result of the feedback we got from user testing session 2

Tuesday, 5 May 2009

The blog has been somewhat quiet of late, as the Academic Networking team have been working mostly offsite through an intense 6 week user-centric design phase. Five weeks down, one (this one) to go!

Oszkar Nagy and Tjhien Liao have been working out of Flow Interactive's London offices, learning from the Flow team as they work. We have gone many phases already, which I can outline very roughly here:

figuring out requirements from the user research

initial ideation of many many small design concepts

placing those concepts on axes of "user benefit" and "technical difficulty" (this was a tricky one!)

working up over a dozen concept ideas into rich descriptions (a phase which generated such intriguing concept names as "Ballroom dancing" and "The Spy")

selecting 3 concepts and refining them into extreme examples reflecting the ideas we had

user testing some paper prototypes of the concepts

refinement of the concepts based on user feedback, plus some work to bring them in from the extremes to something more mainstream

a second round of user testing

It's been a real whirl for everyone!

We are now into a final concept refinement round, and Anne-Sophie de Baets is doing a splendid job of documenting all our work so far.

We're looking forward to some reflective time at the end of our 6-week sprint, when we'll be figuring out where to go next (including what to implement in Sakai and when) and also taking the time to wrap up our documentation effort and start to prepare user research results for publication.

Friday, 20 March 2009

Get ready for our design phase! The next months we’ll be doing user centred design (UCD) which basically means we’ll incorporate all the data we gathered from our research. We’ll do that by looking at the personas, requirements etc.

Oszkar and Tjhien will be spending 6 weeks at Flow Interactive where they’ll have the possibility to work together with people from Flow themselves.Probably they’ll make various versions of designs which will be user tested from time to time.

Thursday, 19 March 2009

On Friday 13th March until Wednesday 18th March, we had an intensive period of discussions and workshops with the people from Flow Interactive.These days would be a wrap up of the research phase and a preparation to the design phase, focusing on tweaking the personas so they’re ready for usage during the design phase, writing scenarios based on the personas, and defining the user and business requirements. Defining requirements, based on the personas and therefore also on what real users want, is necessary for the designers to be able to know what approaches should go in the future system in order to answer the users’ needs.

Define user requirements from persona goalsEach persona had a set of goals, focusing on life goals, end goals, experience goals, motivations, and challenges.We went through each of these goals and rephrased them into requirements.

E.g. From Kate’s persona: ‘Discover what other people are working on.’Becomes a requirement like this: ‘The user needs to be notified about relevant people or content they wouldn’t otherwise know about.’ K

Writing user scenariosA user scenario is a good way to feel you one with the persona and to see how this user could eventually use the system. It’s actually a story describing how the user (that particular persona) might use the system and how it’s an answer to his needs. In the end we came up with detailed user scenarios for every persona.

Concept generationBased on the requirements, we started brainstorming on how these requirements could turn into concepts within the system. This was just a kind of warm up for the design phase, a pool of ideas for future design. Each of those concepts got written down on post-its.

Harriet jotting down concept ideas

Prioritise concepts for design activity We made a big grid: ‘user value’ against ‘effort required’. We discussed where every post-it note should fit on the grid.For the further design, we will pin our minds on the post-its in the area ‘high user value’ vs ‘low effort required’.

Monday, 2 March 2009

Pretty songs always come to an end, and so does this interesting research phase. Lets summarise what we’ve done during the last months…

Gathering information through diaries and interviewsWe wanted to gather information from people who could be possible users of our ‘Academic Networking’ system. Therefore we thought of recruiting users within 3 types of roles – being undergraduates, postgraduates and academics. For each of those groups, we gathered 8 people.Some people would think “Hmmm…how can only 24 people possibly be a representation of your whole group of stakeholders ?!” We think that is possible, because we chose them quite carefully, selecting people within the spread of different subjects, gender, roles, age, stage within their career, etc.First of all we recruited undergraduates and made that selection of 8 people. We gave them the task to keep a diary, writing down every time they had a conversation with someone about their work, studies, or research. When they did that, we had an interview with each of them to talk through their diaries and give us some more insight on other things as well. Whilst doing that, we used a questionnaire as well which guided us through the various topics we needed to focus on.Next, we did exactly the same thing with postgraduates and academics; recruiting, selecting right people, giving them a diary, and having interviews.

recruiting people through an online survey

diary entries

questions used during interview

Task goal analysis and sorting During the interviews, Harriet was hiding in one of the rooms which got a camera and audio connection so she could follow the interviews without any problems. [See Harriet doing that here] The reason why she was spying on us, was because she was doing a task-goal analysis of all the things participants were saying. This analysis was also colour-coded, which meant we ended up with walls like this after every interview:

task goal analysis

When we finished all the interviews, we started analysing all the entries. Clustering all the data into affinity patterns would give us a better understanding of all the data and would allow us to see the overlaying commonalities between the users as well, whether they were a student or academic.After doing that clustering, we ended up with a room like this:

affinity sorting

Behavioural axes exerciseNow we had a good understanding of all the users and their underlying ideas and commonalities, we started putting people on grids which contained behavioural axes poles (E.g. High network awareness vs. low network awareness). It’s important we had at least 10- but not more than 15 – behavioural axes. After putting each of our participants on every pole, we clustered them into group. We started seeing patterns of people reappearing together over and over again.E.g. Sarah and Peter would cluster together on pole 1, and on pole 2, 3, 4 etc.We ended up with 3 big groups of people clustering together. These would later become ‘personas’ in a way.

Creating personasThe groups we just discovered on the behavioural axes, where the starting points of our persona creation. (If you want to find out more how to create personas, then have a look at our newer posts as we’ll post something more about this in the future.)After discussing a lot, we ended up with these 3 main personas and 1 ‘negative’ persona as we called it. This means that persona is someone we wouldn’t focus on immediately. Below you'll see the 3 main personas.

Persona 1 (primary persona) - Isobel: Very outgoing and sociable. If she would have a problem or question, she wouldn’t mind asking someone. She also thinks every conversation is valuable (so not only professors are worth having discussions with).Her network of people is very valuable to her. She’s therefore actively building and maintaining her network as everyone can be useful sometimes.She really likes to go to events where she can meet people and learn in an active way.

Persona 1 (Primary persona) - Isobel

Persona 2 (secondary persona) - Peter: Someone who rather tries to find solutions about a problem himself by looking in books or going through some other sources before actually asking someone. He just doesn’t like bothering other people. Because he doesn’t like to ask other people that much, he’s not always aware of events or other important things which are going on and because of that, he’s sometimes missing out on information.

Persona 2 (secondary persona)- Peter

Persona 3 (secondary persona) – Kate: Someone senior in her role (can be a senior student or academic), who feels she already learned a lot and now it’s time to give something back to other people. Because she’s so experienced, she perfectly know who to contact in what situation. She’s maintaining her network rather than building it up. Because she’s senior, she has a lot of responsibilities and is very busy. Sometimes she might miss out on some interesting events because she’s too busy.

Persona 3 (secondary persona) - Kate

DocumentsIn the text above, you can find the documents we used during the research phase, as well as some videos which might give you a better understanding of the information above.In case you missed out on them, you can also find them below.

> Diary - This is the diary we asked participants to fill in.> Interview questions - This is a script of the questions we used during the interview> Personas - This is a digitised version of the 3 personas we ended up with

> Task-goal analysis - A video showing what's going on in the task-goal-analysis-room during the interview. Task-goal analysis in action!> Affinity sorting - A video showing you how to find themes (clusters) within the bunch of post-its.> Analysis phase in general & profiles - A short video showing you when creating profiles and some discussion while doing analysis> Behavioural axes - A short video showing you another use case example which helps you to understand what behavioural axes are

> Interview and task-goal analysis - This video shows you how to do an interview, using diaries, but also showing the task-goal analysis which is going on in another room; a clear runthrough of the different aspects during this complicated process.

Monday, 23 February 2009

Spotted on the BBC News site: UK museums, including the British Museum and Victoria and Albert Museum, are setting up a new joint website which will let users build communities around museum content. It's the National Museums Online Learning project and you can read about it here.

The value of social networking features to people who are both enjoying the collections and learning at the same time is something we hope we may be able to bring out in our project, too.

Friday, 13 February 2009

Just a quick update for those following our research into the uses of social networking in an academic context.

Having interviewed 8 undergraduates and 8 post-graduate students in Phases I and II of our research, we're now onto Phase III, interviewing a diverse collection of academics associated with the University, from people at the start of their careers to a leading Professor, from someone returning part-time after a career break to people holding both departmental and college posts.

We're eagerly anticipating the analysis phase that will be coming up after this, as we try to synthesise the concerns, goals and motivations across the different groups of University members.

Wednesday, 4 February 2009

Over in the Newnham area of Cambridge, local company CamVine is looking at novel ways of displaying internet content on screens. This is a lot more fun than it might sound - check their daily twitter updates, which will highlight new ways of using Web2 and social, user-generated content in a shared environment. One for every day in February! (Is publishing an idea a day in this way a useful method for getting interest and feedback in your project? Should we try it for Academic Networking? Might it also work for research projects? Would a more private, group-oriented microblog such as Yammer or Wiggio be more appropriate for academics? Let us know!)

In a recent blog post, founder Quentin Stafford-Fraser talks about watching social network status feeds on displays, perhaps dotted around one's home, workplace or university. I can imagine having my research group's feeds in my departmental tearoom, where they could spark conversation and ideas (or give me something to read whilst I wait for the coffee machine to gurgle to a finish).

Netskills are looking for information on how researchers, academics, and the people who support their work use emerging tools, such as Web2.0 things like twitter, flickr and more. This is part of work funded by the JISC Users and Innovation (U&I) programme.

Friday, 30 January 2009

Twitter - a 'micro-blogging' and social networking service - is making its way into libraries across the US, whether to update patrons with important news (unexpected closures), to provoke curiosity and questions or to communicate internally. This article provides an interesting overview:

Thursday, 29 January 2009

The Academic Networking team at CARET, together with our colleague Clay Fenlason of Georgia Tech, have been thinking about Academia.edu, the academic networking site.

It's an thought-provoking model of networking, inviting you to navigate a visualization of two parallel networks, one of research interests and one of academic institutions. Research topics and publishing outputs become the organizing principals.

Here are our thoughts in a little more detail:

Good points

Research is a global endeavour, not an institutional one, and Academia.edu reflects this. One of our interviewees had a profile on Academia.edu. She explained this was because at graduate level, you are unlikely to be working in the same field as people in your department. (This is partly because the graduates and academics are hugely aware of subtle nuances and differencesbetween areas . You need to be aware of and keeping intouch with the research and publications of people who are working in your field - and they are most likely to be at other Universities. The ultimate nightmare for a PhD student is that you devote 3 years to researching 'railway companies in the Russian Revolution' and that 2.5 years through the research, anotheracademic publishes a book on 'railway companies in the Russian Revolution', at which point you have to start again. Thus, Academia.edu offers another opportunity to keep abreast of research in your field internationally, and to make others aware of yourresearch.The same interviewee said that in looking at other people's key words (howthey'd chosen to define their research), she had been prompted to think about her research differently.

It's useful to be able to search people by their research field, particularlynow that interdisciplinary research is becoming increasingly important, and it helps you to find connections - i.e. who else is working on Mongolian nomads, whether they're in the Geography dept or the dept of Anthropology. It's a very obvious thing to do, but not something that you can do in our current university website.

Our interviewee said that she particularly appreciated the option to allowpeople to contact you via the site, without you needing to give out your email address.I loved the way that when I put in the URL of my research paper, it turned itinto Flash paper so that I could read it directly online, rather than by downloading it. It's asmall point technically, but it certainly created 'user delight' for me.

I enjoyed being able to see other people's research papers so easily.

I like the news feed, showing what others have been up to, although it's notreally targeted enough to be of much use

DownsidesI couldn't actually find anyone with my research interest ('Who else is working on Restoration Comedy at the moment?'). I couldn't find it by searching, wasn't sure how it would be classed (dramatic literature? literary criticism?) and couldn't find it by browsing. Or at least I don't think I could find it by browsing - 41 people were under 'dramatic literature' and I didn'twant to look at each of them individually to see if they happened to work on Restoration Comedy. The search facility is clearly vital for this type of tool to be useful.

It isn't regulated in any way, so there's no guarantee that anyone on there isin fact who they say they are, that their papers are genuine, etc. For example, John created the Department of Tube Mechanics at Oxford, and added himself to it. No-one seems to have noticed.

It is extra work for academics to fill in and update their profiles - not onlydo they have to put together their official Departmental profile, at Cambridge their college profile, and quite possibly a research group profile or a specialist disciplinary network , but now they have to keep an Academia.edu profile.

The Flash interface drives me absolutely mad! From people's individualprofiles, I'm constantly being returned to the overview of all Universities when I wanted to return to what I was last looking at, i.e. all members of particular department / centre. It doesn't seem well thought out from this point of view.

Friday, 16 January 2009

The Arcadia project blog mentions some of the conclusions of a new Pew report on how US adults use social networks. It seems they network mostly for personal reasons, that they are aware of privacy issues, and when networking professionally, use a variety of profiles on a range of sites.

Friday, 9 January 2009

Those looking for a brief introduction to what user-centred design might be, and how it might benefit businesses, might want to check out this interview with one of the UX consultants at Flow Interactive, the consultancy that's guiding us through the research and design phase of this project.

I was chatting to a mathematician colleague about our academic networking research in a popular Cambridge cafe yesterday, when a student sharing our table broke into the conversation. "I totally recognise what you're saying," she said, and began to tell us her experience of finding books and the inadequacies of reading lists: "I go to the UL and when I see one of my lecturers in the hallway or the tea room, I ask them to recommend me the three best books on the subject."

It was great not only to have a student spontaneously recognise our interpretations of what our research had brought out, but to be interested and excited that this research is taking place!

Wednesday, 7 January 2009

One thing that the team's been keen on all along is that our social networking tool shouldn't aim to replace face-to-face interaction at the University. We want to promote 'idea moments', what a team from another University called 'Rolos'*, and we think that getting people physically together is a great way for these moments to happen. As far as we're concerned, if our tool introduces people to each other who go on to meet up occasionally, it's done its job - they don't have to regularly communicate via the tool as well.

So I was excited to hear about www.meetup.com - a site which helps facilitate face to face communication using the web. The whole point is that you meet real people, living near you, to talk about shared interests. Yes, there are message boards and photo albums for those groups, but the emphasis throughout is on meeting face to face. I think it's a brave concept. I'll have to report back on the quality of discussion at a meeting, though!

Tuesday, 6 January 2009

The quiet time between Christmas and New Year has been a great chance for me to reflect on the key questions at the heart of this research project (it's been wonderful having such a peaceful office!). I've been taking a step back and considering what we really what to know in order to make a difference to our University members. After all, we are the Centre for Applied Research, and making sure that our research leads to valuable change is what it's all about. The data gathering phase of our research has always been seen in three stages, and with one stage under our belts and moving into detailed planning for the next stages, it's a good time to reflect on how the data and preliminary analysis from stage 1 (looking at undergraduates) should influence the research questions for stages 2 (looking at graduate students) and 3 (looking at academics). Returning to and rethinking research questions seems to me a delicate balance - the University landscape has moved on in the last 3 months, with other related projects being launched or piloted, and so it's well worth us asking again what we want to achieve. At the same time, our original research question, 'How can the social networks of students and academics help them in their learning, teaching and research?' remains valid, and it's important to keep a steady eye on that goal. Ultimately, however, we need to break that high-level question down into a more detailed research questions for each stage of the data gathering phase, and it seems reasonable to be willing to reconsider these detailed research questions over the course of the project.