Summary of reactions: >60 Twitter tweets total including all links, grade Eximia Cum Laude Approbatur, honorary mention in the thesis competition (link in Finnish) of ACM SIGCHI Finland. The work continues!

Update (Nov. 18 2011): This thesis won an honorary mention in the thesiscompetition (links in Finnish) of the Finnish chapter of SIGCHI ! Yay! Their statement of the thesis: “The jury thought this as a new type of thesis work, which successfully captures the phases and challenges in a multi-phased process of redesigning a Moodle community application. Open source communities have been little investigated from the HCI point of view, and the author successfully opens interesting new viewpoints with the thesis. The constructive Pro Gradu thesis has also resulted a tangible contribution.”

Free/Libre Open Source Software (FLOSS) development has become an important way of producing software in the modern society. In principle, the source code produced as OSS is openly designed, developed and distributed, and developers take part in the process voluntarily. The resulting code is freely or with little cost available to end-users. Often the software developers and users are from all over the globe, with the OSS community applying virtual forums for questions and user feedback and support.

Taking part in OSS projects often poses challenges and obstacles to the usability practitioner whose main interest is to design the user interface so that it better fits the user needs. This is the topic Olli Savolainen deals with in his thesis. He reports on his personal motivation and continuous interest in improving the quality and, in particular, usability of Moodle Quiz. He also refers to his efforts and perseverance in gaining acceptance in the community before the changes he suggested after several iterations finally got accepted into the code base of Moodle 2.0. The description of the project is given on two levels. While reporting on the actual user centered design work done in the various phases of the project, another, more personal account of the challenges encountered on the way and reactions to them is unfolded. This kind of reflection is very valuable for understanding the norms, values and ways of working in FLOSS communities. These are important for gaining acceptance and recognition as an active FLOSS participant.

The thesis is a well balanced and reflective document of things learned and practiced in the Quiz UI project as well as thinking about them in the larger framework of OSS development projects as described in literature. The background literature cited is extensive, ranging from books and journal & conference papers to blog and discussion forum entries and documentation. Furthermore, it is well utilized throughout the thesis.

The vocabulary in the thesis is versatile and the language in general grammatically correct, though professional proof reading and language checking might still improve it. A minor drawback in the thesis is the structure that promotes the feeling of repetition, since some issues are first introduced in Chapter 2, but discussed in more detail in Chapters 7 and 8 with many cross-references between the sections. However, this is only a mark of thoroughness and consistency in reporting.

Olli Savolainen has been involved with Moodle and the Quiz UI for more than three years, and his skills and expertise are apparent in the thesis. The main findings are based on personal work experience, and they smooth the usability practitioners’ path into OSS communities. The thesis work is relevant to future OSS development practitioners. It unites the fields of software engineering and usability engineering, bridging the gap still observed in computer science education.

The work carried out by Olli Savolainen clearly fulfills the standards set for a thesis in Interactive Technology. We propose that the thesis is accepted with the grade eximia cum laude approbatur.

At the department of Computer Sciences, September 9, 2010
Saila Ovaska
Eleni Berki

May 11th, 2010

(Update July 5th: Working on the follow-up article is taking longer than I expected. Bear with me, it is on its way! :)

Inspired by Tomaz’ blog post, I did an informal interview with a business and marketing teacher I know. There are two separate points I want to make about the interview, so this article starts a series of two articles.

I wanted to go thinking on a very general level of what are the tools that can be used for helping individuals learn on a given theme. I will here call the place to do such learning, a course.

The questions I presented:

What constitutes a course?
What are the defining factors; what do you do on a course, how, and why? In other words, we playfully tried to generate a definition of a course.

What kinds of tools can be used in order to facilitate learning of individuals on a course?
Then I asked the interviewee to list the tools that can be used for learning in each aspect of the course’s definition. “Tools” are defined very widely here, as anything that can facilitate learning on the theme. They may sometimes have natural hierarchy, but here I want to perceive them such that each we can each still see the relations differently.

The definition here is of necessity more narrow than that discussed by Tomaz – I believe that restriction helps when thinking about the design of a platform for courses.

April 9th, 2010

Robert Martin spoke charismatically about test driven development in last year’s RailsConf. This totally saved my day today.

Why? Because the guy promotes the idea of having tests and running them all the time to prevent your code from becoming an enormous, unholy mess. Because when you have tests, you are not afraid of making changes. (In fact, you are effectively improving the user experience of programming1.) You can play all you want, because you know exactly when anything in your code breaks as a result of you changing the code.

Guess what? It applies to usability, too. Three points:

What is a test? Essentially, it embodies what *should* happen. If, after having changed your code, that something doesn’t happen, you know you are in trouble.

Likewise, When you test usability, you expect the person looking at your UI to do something. When they don’t, you know you have two options: go back to the original, or make it better.

Debugging. When you test code, you may spot that oh, there is an error: the test didn’t pass. Ideally, debugging is so built into the process that you don’t really think of it as separate: since you have tests for every little part of the program, the bug may be pretty easy to spot.

When you usability test, when you notice an issue (and try to keep your calm so the test participant does not notice your frustration) and if your test participant is talking out loud like you have told them to, you learn the reason there and then, and the solution is often more or less obvious.

Lastly, tests are not something that lead to great design. Your code may still suck, but at least you have the courage to improve it since you can test whether your new fix breaks anything else. The important thing is that the tests exist so you can rely on them.

The same thing with usability. A test does not do anything for the design in itself. If you have failed to understand the user’s goals in the first place, a usability test will only show you how the user gets confused while doing the wrong thing in the first place, or while doing it in an unrealistic setting2.

However, a test does describe what the UI was designed to do. When you have comprehensive usability tests for a UI, you can use those test tasks against any new version of the UI, and see if it still serves the purpose it was originally created for, and how well.

In ways, usability testing is just like unit testing: When you have tests defined and you regularly run them alongside development, you know your stuff is good.

If you don’t run them, you don’t know. More likely than not, what you create just does not hold together.

February 13th, 2010

After the last iMoot session I had, I was chatting with Silvia Calvet about usability and its social nature.1

During the iMoot conference I also got a couple of precious chances to hear about different community members’ usability efforts within their organisations. Turns out there are a bunch of people already doing usability related to Moodle, mostly inside their organizations. Even more people are interested, but the environment to discuss usability in the context of Moodle does not exist.

We need an environment where community members can make their usability efforts visible to the rest of the community:

A place where people are encouraged to brag about what they have done for usability in their organisation, and to share what has been learned (usability test tasks, results, …).

A site that could propose you directions to take with usability: give instructions interactively for usability testing, for instance.

A corner in the community to chat in about usability, where you could share your frustrations with Moodle, and with doing usability work, and with usability issues in general.

Another aspect of this effort would be to visualize actual concrete usability data about Moodle.

Have a hierarchy on the site for the high level to low level goals, and for red routes of the Moodle UI (of course, these need to be defined first).

Allow people to link user interfaces (cvs/git), tracker issues and usability tests (containing test tasks and results) into these goals, although keeping the user goals as the starting point for everything.

The slogan? How about… We are all about the goals of the learners!

The magic I want to make happen: make usability visible socially since it is, at its foundation, a phenomenon of social artifacts. Engineers are creating artifacts to users, when they would be better off with other kinds of artifacts. If we can make this disrepancy obvious in the community’s social sphere, there would perhaps no longer be a need to try to convince software engineers of the concrete need for the work. As it is currently, it seems many perceive usability as something too abstract and distant for them to actually do something about it.

Before any of this though, I believe the first milestone is to do usability testing to determine the current level of usability. This is to set measurable goals for Moodle usability, and to prioritize the things a given part of Moodle is primarily intended for. The fun thing about the above vision is that it is easy to start small: first start filling in data for one activity module while it is usability tested, and then build on the vision I am proposing here, as we go. Even if we end up just creating a site for documenting Moodle’s current level of usability (as a side effect of doing usability testing), it is still worth it.

Silvia is someone who has since summer encouraged me in work with Moodle in a great deal. She is working for CVA, a Moodle partner, herself and we also met in EuroIA09 in Copenhagen to discuss where to take Moodle’s usability efforts. [↩]

February 7th, 2010

Preparing for presenting in iMoot, an online conference about online learning and Moodle, was an intensive process for me, but it paid off – both in terms of learning while designing it, and because many of the presentations were really inspiring.

August 17th, 2009

Summary of the Summer of Code

For me, this summer has been full of small and bigger openings. I have gotten overwhelmed, figured out why, and organized my thoughts about Moodle development and usability time and time again. I have found a lot of new, relevant questions and understand much more about the challenge that is Moodle usability, and more broadly, Moodle user experience.

The main outcome of the project, Moodle User Interface Guidelines, got off to a good start, and I believe it will serve as a useful reference in the development of current and future user interfaces.

At the end of the project, I did some brief usability testing (contrary to what I believed in the previous post) which shed light on the Login/Forgotten password form and on the File picker for the rich text editor (see below). I will write more about these very guerilla-style usability tests later. The code produced is in the linked tracker items of MDL-19586.

notet

tärkeät parannusta kaipaavat alueet

I have interacted with the community a lot in the forums, on Jabber channels and in the tracker (see below). Still, the actual guidelines never got as much attention from the community as I planned, so this work continues. Time did not seem ripe for many of the guidelines I planned for. And honestly, I did not spend as much time writing some guidelines I perhaps could have. In the end, I decided to concentrate on what was important at a given time. Sometimes it was a specific UI that needed a gentle touch, or an UI element, and sometimes broader issues were discussed.

Tim Hunt, who has acted as my mentor just like last summer, has been available to comment on anything I needed to discuss. This summer he has given me some great advice especially about interacting with the community. Some of it seemed, ehm, too good since it made me feel pretty humble.

Opening my eyes

The issue seemed to be that there werenot many elements in Moodle that could directly be made into guidelines. In the last post, I outlined various ways that a guideline can get born. With Moodle at the moment, way too many of those were just things there was a need with but nothing tangible existed. As there was nothing implemented that could be written about, often the first step was to talk about a given subject in the forum, to get people to think about whether this thing X, that could later become a real guideline, should be taken into account (keeping user data safe, wizards).

On the other hand, I also felt the profile for usability in Moodle needed to improve. So I offered design services to some current development efforts taking place (course backup, help tooltips, paint tool, file picker/uploader for TinyMCE). My hope was that I could show developers what kind of a contribution a usability practitioner could give into the software development, and make developers start to expect someone to be available for the kinds of questions UI/UX design raises.

With the current understanding of the status of Moodle’s usability and how things are developed in the community, the next logical step seems to be comprehensive usability testing of the overall UI model of Moodle. Only after this we will understand just what exactly is in need of the most work. This requires first determining what use each part of Moodle is intended for – some sort of use cases and usage scenarios – in order to create test tasks. I know this sort of understanding exists in the community: the applications are created with the users’ needs in mind, to a degree.

But the knowledge needs to be extracted and documented. If I have to squeeze it out of certain Mr. Dougiamas’ head… well, just kidding. We need to listen to everybody in the community, but we (I?) also need to understand better just where the understanding about users comes from, and how it is being used. It seems to me that at the moment, much of this is never explicitly discussed in the community.

Based on testing and user research, creating a strong UI style/language for Moodle will later also help further develop the UI guidelines, since there will then be something substantial to document.

I have also gathered a list of Major Usability Issues in Moodle that can be addressed in future projects. This is just a start, but it seems to me that bigger usability challenges need to be tracked, documented and discussed, than what can be done in tracker tickets about individual issues. I am not sure about the tracker’s suitability for this.

Help tooltips: I originally published a javascript prototype for these, but did not publicize them more. A month later Nicolas Connault had implemented something similar (as part of MDL-19756), so I gave him feedback about his work: MDL-20000 display bug, MDL-20095 tooltip delay. There is a bit of a conflict between the benefits of these two implementations of quick help, so hopefully the true gains from each will be considered (well, discussed to start with) and a balance found.

Designed and implemented an experimental prototype for the course editing mode UI for the course front page

The efforts I participated in

Also implemented a patch to unclutter the login form, while making it more consistent with most other web applications. Usability testing seems to hint preliminarily that users find the “Forgotten password” screen more easily after this seemingly trivial change.

August 4th, 2009

The flu that started somewhere around July 14th got several diagnoses and finally seems to have gone away pretty much completely. There are two weeks left of GSoC. This week, I will go through the notes I have gathered during UI inspection. I will create some further UI guidelines and bug reports of all the individual issues I have found. Next week, I will implement some of the changes I am suggesting as patches to Moodle 2.0. Before catching my flu I was thinking about the exact process of determining an interaction style for an application such as Moodle. This starts from the assumption UI elements (and interaction style)s of the application have been discovered and documented, like I have done this summer.
For each UI element discovered, there are three options: If the element is determined good, keep it and document it as a guideline. In some cases there are two elements for the same purpose and the better one of these can be used and the other one replaced. Otherwise, the element in question may have to be changed, or replaced with a new element. The issues involved must be carefully considered.

It may be discovered here that issues are so serious or such big changes are required, that they require a project of their own to be solved (consisting of further research, design, and testing).

In other cases, it may be possible to make enhancements or replace the element in question with a proven solution, which means only shorter period research, discussion and finally usability testing is required.

In the current project, I intended to do even usability testing with the elements that I discover. However, I have ended up mainly doing research of what there is and determining what should be the next step in developing each element. To understand why usability testing was not the Thing to Do™ at this point, stay tuned!