Creative Commons

Updates from February, 2010

February 13th, 2010

After the last iMoot session I had, I was chatting with Silvia Calvet about usability and its social nature.1

During the iMoot conference I also got a couple of precious chances to hear about different community members’ usability efforts within their organisations. Turns out there are a bunch of people already doing usability related to Moodle, mostly inside their organizations. Even more people are interested, but the environment to discuss usability in the context of Moodle does not exist.

We need an environment where community members can make their usability efforts visible to the rest of the community:

A place where people are encouraged to brag about what they have done for usability in their organisation, and to share what has been learned (usability test tasks, results, …).

A site that could propose you directions to take with usability: give instructions interactively for usability testing, for instance.

A corner in the community to chat in about usability, where you could share your frustrations with Moodle, and with doing usability work, and with usability issues in general.

Another aspect of this effort would be to visualize actual concrete usability data about Moodle.

Have a hierarchy on the site for the high level to low level goals, and for red routes of the Moodle UI (of course, these need to be defined first).

Allow people to link user interfaces (cvs/git), tracker issues and usability tests (containing test tasks and results) into these goals, although keeping the user goals as the starting point for everything.

The slogan? How about… We are all about the goals of the learners!

The magic I want to make happen: make usability visible socially since it is, at its foundation, a phenomenon of social artifacts. Engineers are creating artifacts to users, when they would be better off with other kinds of artifacts. If we can make this disrepancy obvious in the community’s social sphere, there would perhaps no longer be a need to try to convince software engineers of the concrete need for the work. As it is currently, it seems many perceive usability as something too abstract and distant for them to actually do something about it.

Before any of this though, I believe the first milestone is to do usability testing to determine the current level of usability. This is to set measurable goals for Moodle usability, and to prioritize the things a given part of Moodle is primarily intended for. The fun thing about the above vision is that it is easy to start small: first start filling in data for one activity module while it is usability tested, and then build on the vision I am proposing here, as we go. Even if we end up just creating a site for documenting Moodle’s current level of usability (as a side effect of doing usability testing), it is still worth it.

Silvia is someone who has since summer encouraged me in work with Moodle in a great deal. She is working for CVA, a Moodle partner, herself and we also met in EuroIA09 in Copenhagen to discuss where to take Moodle’s usability efforts. [↩]

August 19th, 2009

Note: The below test results require understanding about the UIs in question. Follow the links in the beginning to have an idea what the UIs in question are like.

I had eight test subjects during three days usability tests last week. The tests took about 15 minutes each, but gave plenty of data. The main conclusions:

Uploading images to the rich text editor in Moodle 2.0 has too many steps and they are well hidden. All 8 of the users really struggled in one or more of the steps (different users in different ones). If it weren’t for a test situation where users typically try more (since they assume the task is possible to do and there is social pressure), even more of them would have likely failed the task.

The old ‘forgotten password’ form failed or caused struggles in 4 out of 5 tests. The new form caused no confusion in any of the three tests. (I intended to have an equal number of tests for both. This was a mistake on my part.)

This is discussed in the tracker item MDL-16597, along with proposed solutions (Updated September 2nd).

File Upload in the Rich Text Editor

In the foreseen Moodle 2.0, it is possible to upload images whenever there is a rich text field. Except for the [Choose…] button in the Insert/Edit Image dialog, this diagram/mockup is a very close image of the functionality in Moodle 2.0 HEAD I tested, although there were only three sections int

Getting from the text editor to the dialog where you can select which image to upload takes five clicks. Users got lost in all four steps except the last one (pressing the “Browse…” button), most of them in several steps.

1. Get to the add image dialog (4 users found toolbar button, 1 found the item in the right click menu for opening the dialog)

Failure: 2 subjects (needed my help to proceed)

Struggles: 3 subjects (searched how to do it for several minutes, trying clipboard and drag&drop etc. but continued on their own)

Passed this step without struggles: 3 subjects

2. Click the ‘browse’ button (URL is a technical abbreviation and got many users lost; some complained that they do not know what it means; one user complained that URLs have nothing to do with uploading an image from the local computer and was confused due to that. Some users wrote the image name in the description/title field or both; they did not understand the difference between the fields)

Failure: 2 (1 needed my help to proceed; 1 gave up at this point)

Struggle: 2

Passed this step: 4

3. Click “Upload a file” (after this clicking “Browse…” and finding desktop where the file was, was quick)

Failures: 0

Struggles: 4 (In general, users assumed ‘Local files’ to mean the local computer and first thought they should look there. Thus they were not looking for upload anymore – apparently they assumed they were uploading already.)

Passed this step: 3

4. Press “Browse…”

Passed this step: 7

Notes:

One or two of the users did not recognize the rectangle under the file upload field as a button so got stuck for a moment there.

One user did not understand they still had to click the Insert button in the first dialog to actually get the image in the document.

Users ignored “Current files” section, some of them wondered what that is

Forgotten password form

The old form was confirmed misleading since it made 80% of the subjects fill both of the fields. Some of the subjects still did not understand what to do after the form gave the error message to fill one field or the other – apparently the visual (erroneous) message given by the form was a stronger clue to them. What the original forum thread reported was clearly right. I am really surprised such an issue has not been spotted before.

When it was too late, I noticed the WordPress people have designed it even better, though than I did. How bitter: I did not believe it when someone told me having a single field might be a better solution. Now that I see it I think that might have worked better. (Especially due to the formslib bug we have that makes it impossible to have two forms on the same page and thus probably makes the current solution not accessible). Nonetheless, since we now have promising usability test results for my design and none for that of WordPress, I still recommend keeping my design (patch) for now.

Other notes

TinyMCE and other parts of the file uploading were in English, although the browser and rest of Moodle were in Finnish since all the test subjects were Finnish. This may have slowed users down but all the test subjects demonstrated during the tests that they understood the English they encountered.

All test subjects were my friends and in theory this might have introduced a bias. However, as they were unfamiliar with the UI in question and I did not help them during their test taking, the issues they encountered seemed genuine. This can be disputed though and I welcome discussion about this style of low-fidelity testing.

The videos taken (screen image and recorded voice in Finnish) are available on request.

I hope to explain how I did this round of testing and which were the parts where I made it easier for myself. There will hopefully be a blog posting or a video, to show the community how little work this can be.

The test preparatory talks and setting were roughly the same as in last summer’s Quiz UI tests, though less formal.

The test tasks

August 27th, 2008

Last Saturday, I published the results of the usability testing in Kuopio in the newfound Quiz Usability portal. It is quite an extensive bunch of documents so there have not been too many comments about it yet. We also got into discussions with Tim Hunt last week and he requested further comments from the community about including it in Moodle 2.0.

August 15th, 2008

I went to Kuopio on Tuesday, and during Wednesday and Thursday ran usability tests with 7 participants in total. I am in the process of making conclusions from the results, and will publish the results and an executive summary in the Docs as soon as possible.

After making the changes to the UI suggested by the testing, the official Kesäkoodi portion of the project is starting to be done and successful.

Then, there are still some finishing touches the application needs (CSS for IE and other browsers, further tooltips, question bank integration controls, further feedback for different editing operations), but those will be completed during and after September. Also, much integration work – moving CSS to Moodle 2.0 themes, discussion about the general library changes I have made – will need to take place as soon as Tim Hunt, the Quiz coordinator gets settled in to Moodle HQ (Perth, Australia) where he has recently moved.

The community feedback has been good, and Moodle 2.0 launch is planned for the end of the year (last time I heard). I will probably be participating in Openmind 2008 to present the results of this project via video conferencing from France.

June 25th, 2008

Just finished the last OOo Impress prototype test, I have now done eight of them in total with different subjects from four different educational institutions. I will publish the results on Friday (and boy, is there a lot to talk about or what).

Tomorrow I will try to code together the first bits of actual code, in order to put them into the tracker, in order to gain Moodle CVS access, in order to start seriously developing :).

June 2nd, 2008

Last Thursday and Friday, I have been to Helsinki University of Technology and University of Turku and done five wireframe/prototype tests. The results were, as always when it comes to usability tests, at the same time surprising and not at all surprising. I am indeed grateful for finding teachers and support staff from these schools to take part in interviews and testing.

The biggest surprise was that actually, adding random questions is not such a hard task – at least not if users first read a bit of introductory text (3 paragraphs) explaining the three core concepts (quiz/exam, question bank, random question). That is, in the test the setting was somewhat artificial since users were told to first familiarize themselves with the conceptual model of the application (actual usage of the application was not explained though). Clarification: Novice users did without trouble create single questions to the quiz before having read the text, though.

So, the tests gave a preliminary promise that if users understand the underlying concepts, with a reasonable UI no further documentation is needed for a simple task such as creating an exam. This was the real challenge I thought we were facing with the UI, but it seems we are getting there, though slowly. As usual, we discovered several issues, too, but more about them as soon as I get the results into a reasonable format.

It is June 2nd, the first official day of Summer Code. As I have already done a great part of the actual usability work and there still is a substantial amount of usability research to do before implementing the UI as PHP, I will publish a new project plan today. (Update: I did.) I am also planning to do my masters thesis about this project, but I have not gotten very far with that, yet.

Currently, I am still processing the interview data I gathered in the six interviews we did in May.

Based on that work, I will publish an as-easy-to-read-as-possible usability document in Docs, based on which discussion about the actual usage scenarios can continue in the Moodle community.

After that, I will put the current prototype and the findings of the actual usability tests online, too.