Posts tagged with 'currentwork'

Every three months, I conduct benchmark usability testing. I’m calling these tests ‘benchmark testing’ because the aim of these sessions is to measure our progress towards achieving a great user experience with Ubuntu. Last testing took place in October 2011. I am now preparing for testing 12.04 to take place a couple of weeks from now.

When I publish the results of usability testing, I get many questions about my process. So I have thought that the best way to explain how I approach usability is to take you along the preparation and execution of my benchmark testing. Over the next month, I will take you, step by step through my process, from recruiting participants, to writing a test protocol to conducting and analysing usability sessions and writing up results. This will afford you the possibility of ‘accompanying me’, so to speak, and of conducting usability in parallel, if you are so inclined.

For this post, I walk through the first stage of any testing: recruiting participants.

Recruiting

This is a crucial part of any successful and meaningful testing. Some argue that just anyone you can get hold of will do. This attitude, in my view, puts the software before the people who will use it, and carries the implicit assumption that software, by its very nature, is usable. But the simple fact, which we actually all realise, is that it isn’t. Take music players, for instance. The challenge for this type of software is to fit into the lives of people who want to listen to music. It doesn’t have to work well for those who don’t listen to music but who are, for instance, heavily into photo editing. In a word, testing your software with your grandmother or your partner might not provide all the feedback you need to create a user-friendly product if they are not engaged in the activities your software is meant to facilitate.

So, the basic idea is: in preparing the testing, recruit the right people. The type of participants you work with will determine the quality and reliability of the results you get.

There are some basic rules for writing a screener questionnaire.

Rule 1: Recruit according to your testing goals

Is your goal to test, for instance, adoption: that is, are you going to assess how new users respond to your software the first time they encounter it and how delighted they are by it? Alternatively, is your goal to test learning: do you want to assess how easily a novice can figure out how to use your software and how they progress over time? Or are you really interested in expert usage: do you want to assess how performative your software is in a specific context of use involving expert tasks? There are, of course, other scenarios as well. The point here is that you need to be clear about your goal before you begin.

With Unity, we have 2 basic goals: 1) adoption: we want to know how easy to use and attractive Unity is to someone who has not encountered it before; and 2) expert usage: we want to know how performative Unity is with highly competent users who are fairly familiar with it.

Given these very different goals, I will need to conduct 2 different user testing sessions with different recruiting screeners or questionnaires, and different protocols.

In this blog, I concentrate on my first project, to test for adoption.

Rule 2: Know your software

You need to review your software carefully: you need to (1) identify the main purpose of the software and the activities or tasks that it is meant to facilitate; and (2) identify where you think potential usability weaknesses are.

When I prepare a usability test, and before I even think about recruiting participants, I spend a significant amount of time trying out the software, and even more time discussing with the designers and developers their own concerns. From this evaluation of the usefulness and usability of the software, I’m able to sketch a profile of participants. Bear in mind that, given my goals as set out above, the participants will need to be able to use the software right away even if they’ve never used Ubuntu, since I am not testing for learning.

Given what Unity aims to allow users to do, we need to confirm (or not) in the testing that Unity users can easily get set up for and can conduct at least the following activities:

writing, saving, printing documents

finding, opening applications

listening to music

watching a movie

managing and editing photos

customising their computer: organising icons and short-cuts and changing setting

browsing the internet

communicating

Additionally, the OS should make it easy for users to:

multi task

navigate and use special features like alt-tab

be aware of what’s going on with their computer

create short-cuts

understand icons, notifications and generally the visual language

In this instance, I want as well to test the new features we have designed since 11.10

Given my goals, my recruitment screener should be written in a way that will provide me with participants who engage in these activities on a regular basis.

Rule 3: Make sure you have an appropriate number of participants, with an appropriate range of expertise, with appropriately different experiences

I’ve often heard it said that all you need is a handful of participants – for example, 5 will do. While this may be true for very specific testing, when your participants come from a homogeneous group (for example, cardiologists, for testing a piece of cardiology software), it is not true generally. Much more often, software is meant to be used by a variety of people who have differing goals, and differing relevant experience and contexts of use.

You need to take these into account for 2 purposes: 1) to be able to test the usefulness and appropriateness of the software for different users; and 2) to be able to assess the reasons and origins of any usability problem that you find – these can be explained by comparing differences between users. A usability problem will have a different design solution if it is created by a user’s lack of expertise than if it is created by a shortcoming of the software that stumped all user groups. It will also help rate the severity of the discovered problems.

Some of the factors a competent recruiting will take into account are:

Different levels of expertise: for example, in the case of software for photo-editing, you probably need to assess the ease of use for people who have been editing their photos for more than 5 years, and for those who have been editing for less than 1 year. Expertise can be reflected in the length of time they have been engaged in the activity and also in the complexity of their activities. You may want to recruit people who do basic editing, like eliminating red-eye; and then, to compare their use of your software to the use by people who do special effects, montages, presentations and the like. This way, you get feedback on a wide range of the software’s features and functionalities.

Different kinds of uses: potential users will have different needs and different potential uses for the software. For example, if the software is healthcare related, it may well be used by doctors, nurses, radiologists – and sometimes even patients. It is useful, when considering recruiting, to include participants from these various professions and other walks of life, so that you will be able to determine how well your software serves the range of needs, processes and work conditions represented by the likely (range of) users.

Different operating systems: you may want to select participants who use, at least, Windows, Mac and Ubuntu. Users who are new to Ubuntu have acquired habits and expectations from using another OS. These habits and expectations become with time equated with ease of use for these users because of their familiarity. Recruiting participants with different habits and expectations will help you to understand the impact of these expectations as well as receptivity to innovation.

Recruiting your participants with precision will allow you to understand the usability of your software in a complex and holistic way and will dictate more innovative and effective design solutions.

Keep in mind, however, that the more diverse the kinds of persons who you envisage will be primary users for the software are, the larger the number of participants you will need. You should recruit at the very least 5 similar participants per group – for instance, in the healthcare example, at least 5 doctors, 5 nurses, and 5 patients.

A few more things to consider explicitly putting into your questionnaire/screener, particularly if you are writing it for a recruiting firm:

It is advisable to have a mix of male and female participants;

Participants from different age groups often have different experiences with technologies, and so you should include a good mix of ages;

The perceived level of comfort with a computer can also help the moderator understand the participant’s context of use. A question about how participants assess themselves as computer users can very often be helpful;

You should always add a general open question to your screener to judge the degree of facility with which the potential participant expresses ideas and points of view. The moderator is dependent on the participant to express, in a quite short amount of time, the immediate experience of using the software. Consequently, being able to understand the participant quickly and precisely is vital to obtaining rich and reliable data. The individual who makes the recruitment needs to be able to evaluate the communication proficiency of the potential participant.

Rule 4: Observe the basics of writing the recruitment screener

The most reliable way to obtain the desired participants is to get them to describe their behaviours rather than relying on their judgment when they respond to the screening questionnaire. For example, if you want a participant who has a good experience in photography, instead of formulating your questions as:

Question: Do you have extensive experience in photography?

Choice of answers:

Yes
No

You should formulate your question in a way to make sure the person has some level of familiarity with photography:

Question: During the last 6 months I have taken:
Choice of answers:
between 20 and 50 photos a month [Recruit]
Less than 20 photos a month [Reject]

By matching potential participants to actual behaviours, you can make a reasonable guess, for example, here, that someone who has been taking 50 photos every months in the last 6 months is indeed competent in photography, whereas when you rely on the person’s own assessment that they have extensive experience, you can’t know for sure that they are using the same criteria as you do to evaluate themselves.

Your screener should be created from a succession of questions representing a reasonable measure of familiarity and competence with the tasks you will test in your software.

That said, your screener should not be too long, as the recruitment agency personnel will probably spend no more than 10 minutes to qualify candidates they are speaking with on the phone. At the same time though, you need to ensure that you cover questions about all the key tasks that you will ask participants to perform during the test.

Summing up

Let me sum up the basics I’ve just covered by showing you the requirements I have in my screener for testing the ease of use of Unity by the general public user, not necessarily familiar with Ubuntu. They include that:

there should be a mix of males and females;

there should be a variety of ages;

participants should not have participated in more than 5 market research efforts (because people who regularly participate in market research might not be as candid as others would be);

there should be a mix of Windows, Mac and Ubuntu users;

participants should:

have broadband at home (being an indicator of interest in and use of computer during personal time);

spend 10 hours or more per week on computer for personal reasons (which shows engagement with activities on computer);

be comfortable with the computer, or be a techy user;

use 2 monitors on a daily basis (I want to test our new multi-monitor design) to carry out a variety of activities online (part of the designs I want to test relate to managing documents, photos, music, and so forth and I want my participants to be familiar with these activities already);

use alt-tab to navigate between applications and documents (another feature I intend to test for usability);

have a general interest in technologies (I want to make sure that their attitude towards new technologies is positive, so they are open naturally to our design);

express ideas and thoughts clearly.

In closing let me add that testing with friends and relatives is very difficult at many levels. First, you can’t ask all the questions you need to: there are many ‘common understandings’ that prevent the moderator from asking ‘basic/evident/challenging’ questions that might need to be asked to participants. Second, participants might not be sincere or candid about their experience: someone who knows you and understands your commitment to the software might not express what they think, and they may not identify problems they are experiencing and thus, they might minimise the impact of a usability issue or even take the blame for it. Third, of course, they might not fit as precisely as they should the recruitment screener.

Feel free to use this screener to recruit participants if you would like to conduct testing sessions along with the ones I will be doing at Canonical.

In a couple of days, I will write a blog post about writing the protocol for this round of testing – which is the next step you’ll need to take while you’re waiting for participants to be recruited.

If you’ve ever had to create Ubuntu or Canonical related design materials, chances are you had a look at the Brand Guidelines, which, until now, have only existed in the form of bulky PDFs. Those days are over, as we happily introduce the brand new Ubuntu Brand Guidelines site, where you can read the guidelines and download the assets necessary to create your projects.

Ubuntu Brand Guidelines homepage

You can learn more about the Ubuntu brand values and the brand assets, such as our logos, colour palette and pictograms, and how to use them. You can also consult some of our Web-specific guidelines, look at examples of design work that has been done, and download assets like the logos and pictograms.

Brand assets section on the Brand Guidelines site

This is the first iteration of the site: lots of content is being prepared and will be added later on, and we will also work on some refinements to the asset download process, as well as adding many more useful downloads, such as templates and photography.

Among the more frequently requested assets are HTML and CSS snippets and templates that can simply be copied and pasted on internal and external projects, so the designer or developer can be certain everything looks as it should. This is in the works, but it’s something that takes a little bit more time to get just right, so please bear with us.

For now, we’d be delighted to get your feedback on this first version: have you found anything particularly useful on the site? What would you like to see there that you think it’s missing? How do you think it can be improved?

For the multi-monitor design project, we have been making use of prototypes to develop and test some of the finer interactions of the system. One such crucial element is the reveal of the Launcher, particularly as we are exploring having a Launcher on each display. The motivation for making the Launcher available from any display is to allow the user to launch and switch applications, without having to travel onto another display to do so.

So, here is a prototype for the Launcher reveal, which we would like to share and get some feedback on.

It is worth pointing out that this prototype concentrates on the detailed interactions for the Launcher reveal only. This is not the more fully featured multi-monitor prototype, mentioned in a previous post (first shown at the October 2011 USD), which will be shared a little further down the line.

Launcher Reveal Considerations

With a Launcher available on each display, we have chosen to hold the cursor briefly at the edge of any display which does not sit on the left-most edge of the extended desktop, allowing the user to push against the edge to reveal the Launcher. Important considerations here are:

The Launcher reveal must be reliable and easy to achieve when required

The Launcher reveal must not be too sensitive: there is already an issue with false positives for the reveal, when targeting items near the left edge of the screen (eg. the browser Back button)

The user should be able to pass quickly and easily onto the display to the left – they will not always be looking for a Launcher reveal.

Related to the previous point, if a user overshoots onto a display to the right (when targeting items such as scrollbars on the far right of a left-located display), it should be quick and easy for them to correct their position back onto the other display.

Running the Prototype

The prototype is a C++ Qt application. Download the archive and unpack it. You may be able to launch the application by just double-clicking on it. If this doesn’t work, you will need to launch the executable from the terminal. The steps are as follows:

Unpack the downloaded archive to a suitable location. In this example, we unpack onto the Desktop.

Open the LauncherReveal folder on the Desktop and try double-clicking the LauncherReveal executable to launch it.

If this doesn’t work, launch the Terminal application.

Type ‘cd Desktop/RevealLauncher’ to change to the directory which contains the executable (replace ‘Desktop’ with the directory you have unpacked the archive contents into if necessary)

Type ‘sudo ./LauncherReveal’ to launch the executable. To grant permission for the system to run the executable, you will then be prompted to enter your login password.

We have used quite a low-level language and framework for the prototype because it needs to create windows across multiple displays and to manipulate the position of the mouse cursor.

You can run the prototype on a PC with a single display, and try out the Launcher reveal. However, the prototype really becomes interesting when you run it with more than one display attached, and check out the Launcher reveal across all displays.

This prototype concentrates on the Launcher reveal only, so there are lots of things (windows, top bar) which do nothing. The prototype will only work properly for multiple displays with the same height, organised in a row. Being a prototype, this is essentially throw away code, which lives just to explore a very specific set of interactions, for a limited configuration of hardware.

In order to reveal the Launcher, you push the cursor into the left edge of a display for a fraction of a second (100ms by default). This works on any display (not just the far-left display), as we hold the cursor at the edge of a display when it crosses from either the left or right. Push a little more (a further 150ms by default) and the cursor will break through onto the next display.

The images of the browser window are used to test for false positives and overshooting problems when trying to target the Back button on the far left, and the vertical scrollbar on the far right.

Tuning the Parameters

You will notice a panel with lots of parameters to tweak. We have chosen defaults which work well for the small sample of people and hardware we have tested the prototype with so far – informal ad-hoc testing at this stage. Here is an explanation of each parameter and the trade-offs they represent:

Launcher reveal: push for 100 ms
The length of time the user must push against the left edge of the display to reveal the Launcher. For lower values, you get a more responsive-feeling Launcher, but you also get more unwanted reveals when targeting items on the left of the display (eg. the browser back button). Try tuning this value up to 200ms for less unwanted reveals, but you’ll need to push a little harder for those reveals that you want.

Pass display edge: push for 150 ms
Once the Launcher has revealed, we don’t want the cursor to break straight onto the next display, so we continue to hold the cursor for a little more time on a left-most edge. This gives the user the opportunity to stop pushing and move to target a Launcher icon. Lower values make it easier to move from one screen to another, but more likely that you will break though onto the next display when you wish to target something in the Launcher.

Event sampling period: 50 ms
The event sampling period is the size of the time-slices which are used to determine when the user has stopped pushing. A time-slice which collects no mouse events will result in an ‘end-of-push’ condition, cancelling a Launcher reveal or movement across a display edge. Lower values will increase the chances of an unwanted ‘end-of-push’ condition (for gentler pushes or older hardware). This period must be large enough to divide the previous time values into multiple time-slices, otherwise they just degrade into timer delays.

Cursor travels freely after crossing display edges for 1000ms
In order to allow the user to quickly correct an overshoot onto another display (when targeting items to the left or right extremities of a display), we temporarily drop the hold-at-edge behaviour once the cursor crosses an edge. Lower values give the user less time to make any corrections, but making this value too large results in missed edge-holds and Launcher reveals.

Cursor travels freely at velocities over 1000000 pixels/sec
In the prototype, we have prioritised easy, predictable Launcher reveals over travelling very quickly across the extended desktop. The user can still travel across the desktop fairly quickly, although they will be detained for a fraction of a second at any edges in the way. If you want to try out an alternative prioritisation (quick travel across the extended desktop, requiring a slower, more careful and deliberate targeting of the edge for a Launcher reveal), then drop this value down from the very high default value (which effectively disables this feature), to something in the region of 2000-5000 pixels/sec.

Hold cursor at right edge of displays (true)
We found that symmetric behaviour, with respect to holding the cursor on display edges, feels more natural, and also makes it easier to target items near the right edge of a display. However, this feature can be disabled, in order to evaluate whether less intervention on cursor movement might be preferable.

Show Launcher proximity shadow (true)
A shadow will appear at the left edge of a display to improve the discoverability and feedback for the Launcher. It grows out as the cursor approaches the edge, and then grows further still when the user pushes against the edge, providing feedback that the push is being recognised and the Launcher is about to reveal.

How It Works

You can check out how the prototype works by looking at the source code. If you know some C++ and Qt, you should hopefully be able to make some sense of it.

Fundamental to the interactions here, is the ability to determine when the user starts to push against an edge, and when they stop. We measure the duration of the push to see if we should reveal the Launcher or let the cursor through onto the next display. It is straightforward to determine when the push starts: we identify the first mouse move event which crosses a display edge. But how to determine when the push ends is more difficult. Jason Smith, on the DX team, came up with the neat solution of splitting the entire duration of the push into smaller time slices. For each time slice, we count the number of mouse move events coming in. As soon as a time slice expires which has collected no mouse events, then we have the end of our ‘push’.

Over the past few months we have been working on improving the multi-monitor experience in Ubuntu. We took the opportunity at UDS in November to get some feedback on a prototype, which shows how we are planning to develop the multi-monitor experience over the next few cycles:

We invested in a six monitor rig and the prototype to test a number of different display configurations and to ensure that our design ideas scale well. However, our main focus for Precise is to ensure that we deliver a reliable and supportive experience for the core use cases, such as connecting to a second display or projector, disconnecting displays and using a closed laptop with an external display.

So here is the Phase 1 specification, scoped for the next couple of cycles, incorporating the feedback we got from the prototype and sessions at UDS:

Work continues now on the prototype, which will be used to conduct usability testing on the launcher, spread, window management and workspace interactions for multiple monitor setups. We will be publishing the prototype on this site (the Ubuntu prototype application, along with the Qt C++ source code) in the near future, so keep tuned for more Multiple Monitor news.

Last night I watched “Press Pause Play” which in it’s own words is a film about fear, hope and digital culture. If you’ve not heard the talk surrounding the movie one of the trailers is below and we’ll catch up once you’ve … well … caught up

Good huh? The interesting thing about Press Pause Play is that it speaks to people who are out in the world creating interesting things and discusses what the brave new world of powerful computers, amazing tools for creation and sharing of content and ideas instantly means for the creative arts. For better or worse anyone can be a film maker now or a photographer or web designer or musician. Install Ubuntu from a USB key, plug that computer into an internet connection and “Ta-Dahhh!” you’ve got instant access to tools which allow you to create amazing things. Or functional things. Or mundane things. Or robots … seriously … people are making robots and they’re using Ubuntu to do it.

The film is extremely good and I’d urge anyone interested how people make stuff today, music, art, film all that “stuff” to watch it. I also think there’s a lot in there for people passionate about free and open source software. The way that we create software, these tools, this approach, it’s helping people who’ve never met to collaborate and produce all sorts of things.

I found it an inspiring watch and best of all it’s available for free from www.presspauseplay.com just download, grab a cup of tea and enjoy. I had jelly babies too but don’t eat too many, you’ll be sick.

Recently we hired an external consultant to compare the usability of 2 email clients: Thunderbird and Evolution. I have taken some highlights from the report to compose this blog.

Setting of the usability session

The sessions took place in early June at the Canonical Office in London. Thirty participants were recruited. All of them used at least 2 email clients.
Methodology

One email account was set up in preparation for the sessions; all users were asked to use this account’s details to set up the email package. Days prior to the testing, messages were sent to this account and it was also subscribed to a mailing list, in order to ensure a realistic influx of emails to this Inbox.

Half of the participants interacted with Thunderbird and the other half with Evolution; Thunderbird 5.0 build 1 and Evolution 3.1.2 were used on a desktop machine using Ubuntu 11.10.

During each 60 minute session, participants were asked to:

set up the relevant email package for an existing account;

create a signature;

compose an email and change font colour;

manage emails in folders;

locate a specific email containing an attachment (sent prior to the session);

respond to an email containing an attachment and send the latter back to the sender;

Both

Both met expectations in terms of generally intuitive access to contextual menus

Both provided intuitive access to search facility

Where Participants Struggled

Thunderbird

Confusion over search use (severe)

Users were confused by the existence of two search fields, often opting for the All messages search box as they intuitively saw this as highest in the hierarchy. This choice often resulted in disappointment when users did not expect to be taken away from the folder they were searching in; in addition, they found the search results confusing and inefficient, reporting that they expected the item they were searching for to be more easily visible.

Participants were further frustrated by the fact that if they had misspelled an entry or their search returned no results, they would not be aware of this until taken to the search results tab, which they saw as a frustrating waste.

Difficulty locating and managing folders (severe)

The majority of participants successfully created folders, either by right-clicking in the folder area or using the New functionality in the top menu. However, most of these users were unable to easily locate the folder created or move once they had located it. This was due to them not realising they were creating subfolders; once subfolders had been created, unless that folder already had folders within it and it was expanded, users did not notice the expand icon next to the folder and bypassed it. Finally, once they found the created folder, users attempted to relocate them to the desired place; majority of users failed in doing this successfully.

Difficulty personalising message text (mild)

More than half of users struggled finding the function to change font colour; the majority looked for this in the message text toolbar, bypassing the colour functionality because they expected the icon to look differently. Users eventually found this with the use of tooltips, but not after looking through all toolbar and top menu options first. Participants voiced the issue for this to be the icon being in black and therefore too subtle; they mentioned preferring a more colourful icon or one resembling a palette.

Unclear server options (mild)

Participants reported liking the apparent ease of setting up but most were confused by the server options provided in the second, and final, step. About half reported that they would navigate away from the window and research more into their options, with the rest either ignoring this message and go with the IMAP option already selected or choosing the POP option which caused them some issues finding emails later on. The majority of users reported preferring helpful information and guidance on the options provided in the set-up screen, in order to avoid navigating away or uncertainty.

Difficulty finding and personalising signature (mild)

The majority of participants were unsure where to find the signature functionality, with the majority expecting it to be either in the main toolbar, message toolbar or Message menu section. Most participants were unable to find this feature on their own without looking up help or reporting that they would ask a friend for help.

Evolution

Longwinded, unexpected set-up (severe)

Despite appreciating the guiding steps outlined in the mail configuration assistant, the majority of participants reported feeling that this process was unexpectedly too long and found the options provided very technical and confusing. For the majority of users, this culminated just at the second step, where they thought they were being asked to retrieve backed-up files, rather than being offered option to set up this feature. Some users failed at this point, reporting that they were confused by this and would revert to using the current email set-up they had.

Locating account email (severe)

The majority of participants had difficulty initially locating account email due to the email folders displaying an Inbox and an account-specific email section. Most participants did not notice that the account email area was collapsed and were confused about the ‘Inbox’ shown at the top of the folder list not showing any messages. Users attempted to view the account Inbox by selecting Send/Receive and then clicking through all folders available. Eventually users noticed the email account folder with the expand icon next to it and accessed the account folders that way. This experience caused great alarm in these users, particularly as it was at the beginning of interaction with the system; as a result, many reported loss of trust in the package and considering ending its use.

Unintuitive message search (severe)

As discussed, search was intuitively used by participants to quickly find a required message in a large Inbox. Many participants failed finding the required search results because they carried out a search unaware that they had selected an irrelevant folder. This resulted in no results being returned and users being confused because they had expected to be able to search all folders.

Once email opened, difficulty getting back to Inbox (severe)

Half of participants naturally double-clicked to read email in more detail; however, in Evolution, this resulted in email opening up over the main Inbox window, hiding the email list. Participants were confused by this and struggled to get back to the message list; majority reported looking for a button or link to Inbox and were extremely weary of closing the window down (either via the buttons or menu items) because they were nervous about potentially closing down the entire email application.

Inability to personalise email text (severe)

Almost all participants were unable to personalise message text in Evolution; they expected access to font colour to be available along with the other font toolbar options and entirely bypassed the HTML option. One participant selected HTML and still missed the font colour option. Participants were very disappointed by the lack of this feature and looked at all toolbar and top menu options for access to this.

Despite long set-up, confusion over lack of password request (mild)

In addition to finding the Evolution email set-up longwinded, participants were confused why this had not asked them for account password details. The majority saw this as a frustrating time waster, particularly as they were asked for this separately, once their email had been set up.

Difficulty locating and managing folders (mild)

As with Thunderbird, the majority of participants successfully created folders, but here mainly by using the Folder functionality in the top menu. All participants expected to be able to create a folder by right-clicking in the folder area and only a few right-clicked on a folder to look for this functionality. Despite being able to create the folders using the top menu, users were disappointed with the lack of quicker access to this feature in the folder area (either by right-clicking or with the use of a button) or a button in one of the top toolbars.

Usability Issues Common to Both

Difficulty finding and personalising signature (mild)

As with Thunderbird, the majority of participants were unsure where to find the signature functionality, with the majority expecting it to be either in the main toolbar, message toolbar or Message menu section. When users found the Signature option in the message toolbar, they were very frustrated that this did not provide a shortcut to signature creation. Most participants were unable to find this feature on their own without looking up help or reporting that they would ask a friend for help.

When they were taken to the signature feature, users were again frustrated at the fact they could not find a font editing facility, despite the interface looking like it should allow for this.

Conclusions

As discussed, users gave both positive and negative feedback on their interactions with Thunderbird and Evolution, with Thunderbird consistently being perceived by users as easier to use and fit for purpose than Evolution.

Thunderbird was widely liked for the perceived straightforward set-up and facilitated access to contact save, search and open windows features. In addition, users commented on the familiar language used in the application.

However, participants encountered a few severe issues which tarred their image of the system. These consisted of extreme, at time show stopping, difficulty with:

successfully understanding and choosing the relevant search field to use;

locating and managing the preferred location of folders.

Finally, these users encountered some lack of clarity over server options in set-up and frustration at the inability to easily format email text.

Participants who interacted with Evolution liked the guiding steps in the mail configuration assistance, the intuitive contextual mention options to add contacts and the ability to easily access alternatives to button shortcuts in the menu.

Users reported multiple severe issues around the Evolution set-up, locating account email, message search use, formatting email text and navigating back to the Inbox. All of these issues were so major that users encountering them reported lack of trust in the Evolution package and a reluctance to continue its use.

One major fact to keep in mind is that, especially as the majority of participants were new to Ubuntu, they saw the email application they used as a representative of the operating system. This is particularly pertinent to the email system that is a system default and it should be ensured that, before either one of these products is chosen for this purpose, the severe issues reported here are addressed.

When we introduced the new overlay scrollbars we knew it was a bold decision and we were expecting some critics because of the use cases we didn’t support.

As hoped, we had a lot of very useful feedback. Most of the people very liked this innovation and understood our need to be consistent to our design principles. But because we were hoping for the minimal impact, it was important for us to understand when this wasn’t the case.

Since then we kept working on the scrollbars to implement some of the other things we had initially in mind and also to resolve, when possible, the issues raised. We are pretty happy with some of the solutions, for instance the Fitts’s law optimization for maximized windows, but we still have indeed work to do.

The most common features which we couldn’t cover yet are:

instant scroll through middle click

reveal the thumb when coming from outside the window

As you can easily imagine, these features are quite challenging by the nature of the current design and, in general, we would like to think any new feature through. We want to make clear that we value your feedback and these requests are far from being forgotten.

Despite our efforts to materialize our vision, there are many toolkits that need to implement them to provide a consistent experience across Unity. We would be more than happy to assist anyone interested in this work stream, in particular: XUL, Qt and the web browser Chromium. If you’re willing to help or you have any question related to the implementation and the technology used, don’t hesitate contacting our software engineer Andrea Cimitan

We all know Ubuntu is great, but we want even more people to know just how great. With this in mind, we thought we’d give the visitors of ubuntu.com the tools to spread the word about Ubuntu.

As of today, you can see Tweet and Like buttons on some of the key pages of the website, such as Ubuntu for you, the Features pages, or the Download page.

This is the first step towards something bigger. In the pipeline are the introduction of more ways of sharing the Ubuntu message with friends, family and (why not?) your entire social network. For now, we’ve focused on the two most popular services.

Sharing your favourite pages of the Ubuntu website on Twitter is a breeze. Before tweeting, you can customise your message too.

We’d love to get your feedback, hear your suggestions, and know your ideas on how you can tell the world just how lovely Ubuntu is. As an Ubuntu lover and active member of the community, what tools do you think would help you and be most valuable in sharing your experience of Ubuntu?

Finally, if you love Ubuntu, help us spread the word: visit ubuntu.com and share those links with as many people as you can!

Every 6 months the Ubuntu journey starts anew. Those of us entering yet another cycle assume that this all makes sense to the outside world but I like to post up dates on the wall in the office and write a blog post to give those new to the project, and some not so new, a handy reminder of the major milestones in each cycle.

Each release that we create has a cycle with certain key milestones in it. These milestones are broadly agreed before the previous release is even out the door and are almost always an exact copy of what came in the previous release. The schedule for our next release 11.10 Oneiric Ocelot can be found by following this handy link to the Ubuntu wiki.

Anyone wanting to contribute to Ubuntu needs to be aware of these dates, designers and developers alike, as dates like the feature and UI freezes are your deadlines for getting new goodness into the release. Miss this deadline and you’ve missed the October release. Like a train the Ubuntu release rolls out of town whether you’re on it or not, however, unlike those dreadful windows trains there’s another one along real soon – in another 6 months in fact!

For more information on the time based approach to projects head over to the wiki pages on the subject and hit us up in the comments if you have any questions!

First and last impressions of Unity were that it was quite user-friendly, and pleasing in its design and ease of learning. The majority of participants left the session with very positive feelings and were looking forward to Unity’s release so they could download it. In short, participants in this testing session were considerably more positive about Unity than participants who tested the previous version in October.

This improvement, no doubt, is due to the significant changes we have made since the last testing, often in response to problems uncovered during that testing. Many of the serious issues discovered then have been resolved. Most significant, as it stands now, there are no longer any “show-stoppers”.

However, there are still a few interactions that were at odds with the product’s general ease of use.

Some important points to keep in mind

First, it appears that those of our participants who were Mac users seemed to have had more facility with the Unity interface than Windows users, especially those using anything previous to Windows 7. Generally, Windows users tended to rely on right click, and they sought menus from which they could find and launch applications as well as move and delete. They did not immediately take advantage of Unity’s visual assets. Accordingly, Windows users will need to be encouraged to manipulate icons and to develop a more physical relationship with Unity than the more text-heavy relationship they have with Windows.

Second, Unity’s concept of ‘Home’ (Nautilus file manager) is different for that of our users, even Mac users, and they did not immediately understand it. They had a tendency to go to the ‘Home’ icon, not only to find information about their computer, but for any programme or application they were looking for. Essentially, many navigated from one application to another using ‘Home’. For example, almost every participant first looked into ‘Home’ to find computer settings and to change their wallpaper.

Third, most participants were not able to figure out how to reveal the Launcher from the upper left corner. They immediately devised work-arounds, like closing windows or moving a window away from the left edge of the screen. They expected to be able to reveal the Launcher by approaching any point along the left side with their pointer. As the Launcher is one of the most important features of Unity, it should be either always visible or at least very easy to bring out.

Fourth, the Dash is hard to discover. The icon is too small and understated compared to the icons in the Launcher. By its size and placement, it is easily associated with the window management buttons. Participants who discovered the Dash found it very useful, but were more inclined to use Files and Applications Lenses at the bottom of the Launcher. This was, I’m convinced, partly due to the fact that there were no data, pictures, music or documents on the computer that they would want to access through the Dash at the time of the testing, whereas the Applications Lens, in early use, is more adapted to general exploration. The Dash needs to be more visible — it needs to be accorded its rightful place as a major feature of the interface.

Notwithstanding these small problems, it is fair to say that this test showed that we have made significant progress since the October testing.

Some Major Issues that Have Been Resolved since October

Visibility of icons at the bottom of the Launcher

During the April testing, participants experienced difficulty seeing the bottom of the Launcher when it was accordioned and then, when the Launcher expanded, it hid the bottom icons. At the time of the testing, it was very difficult to reveal these bottom icons even by scrolling down. Recent updates have resolved this problem by making the Launcher automatically scroll down when users move the pointer down along it. This way, the icons that were previously hidden are effortlessly revealed.

A related issue that has also been resolved is that, during testing, participants wanted to make the Launcher visible by touching any part of the left side border – whereas, in fact, the only way to reveal the Launcher was by reaching with the pointer to the upper left corner. With the updated version, users can now reveal the Launcher from any point on the left side of the screen.

Changing the order of icons in Launcher

During the October testing, when the interaction to move an icon in the Launcher was to select it and bring it outside of the Launcher before giving it a new position, many participants failed to do it. The new interaction supports users’ natural way of moving an icon: participants were able to move icons in the Launcher by selecting them and moving them vertically up and down. It should also be noticed that the feedback provided when users select an icon they intend to move helps them understand that they have initiated an action. Knowing that the icon has effectively been selected afforded them more freedom to move the icon around and to find a way to make it work.

Adding icons to the Launcher

Participants were able, even during the October testing, to drag and drop the icon of an application from the Applications Lens into the Launcher. However, their first attempt, especially for Windows users, was to right click on the icon they intended to move and expect to be offered an option to attach to the Launcher in a drop down menu; and second, to look at the top of the Launcher for a ‘Launcher menu’.

Identifying running applications

Most participants were able to see immediately which applications were running by means of the white arrows beside the icon in the Launcher. However, they were not sure if they had made the right inference. In short, although participants were unsure about the meaning of the white arrows and bars, they were able to figure them out which indicates that this is a feature easy to learn.

Changing the wallpaper

Most participants easily changed the wallpaper by right clicking on the desktop.

Deleting a document

Most participants easily deleted a document.

Detailed Summary of Benchmarking – Comparison of the October and April Test Results

The points above are the highlights of the findings. Let us now examine individually the differences in performance as revealed in the testing of last October and the one just completed in April.

Performance

October report: “The level of performance in this regard significantly impaired the flow of use and the user experience.”April testing: Unity was quick and responsive.Outcome: This is fixed.

Multi-tasking: Having many items opened and accessing them

October report: “Thus, while working on a task, participants expected that Unity would provide them with a representation or visibility of what was available to them and how to easily access what they needed at any given point.”April testing: No problems were observed with overlapping open applications and documents. Participants could easily move individual windows and reveal items placed underneath.Outcome: This is fixed.

General navigation

October report: “Overall, participants found the navigation to be cumbersome.”April testing: Participants used Nautilus to find applications and documents as well as system settings. This is not, however, the most efficient way to do this.
The Files and Applications Lenses icons need to be more prominent in the Launcher. However, participants found it easy to go from one document or window to another and to make them all visible to them.Outcome: In a recent update, the icon ‘Home’ (Nautilus) has been renamed ‘File Manager’ and the icon has been modified to downplay the home relationship. This should help users recognise its role and lead them to look for an alternative place for system settings and other programmes and applications.

Minimising a window

October report: “When participants minimised a document, the document seemed to have disappeared when they expected it to be shown at the bottom of the screen.”

April testing: A few participants still expected to see a trace of their minimised document at the bottom of their screen.Outcome: Since the usability sessions this interaction has been updated to show the window minimised into its Launcher icon even when the Launcher is hidden. This should help users to locate their minimised documents and windows.

Awareness of running applications

October report: “Participants did not always see the white arrows that indicate a programme is running or documents are opened. Consequently, they were not aware of what was available to them.”April testing: Almost all our participants were able to tell which applications were running by looking at the white arrows. However, some were not sure at first and needed to ‘try it out’. So they opened and closed windows and applications to check on the behaviour of the icon in the Launcher.Outcome: The white arrows seem to be working well once they have been discovered. Although they are not obvious, users can figure them out. This is easily learnable.

Displaying documents side by side

October report: “No participant could find a way to resize his/her openoffice documents in such a way that they could be placed side by side while working on both at the same time.”April testing: Except for one, all participants were able to display two documents side by side. However, as noted above, they were not able to discover the semi-maximised state.Outcome: The original problem is fixed. In the new design of Unity, participants have a way to display their documents side by side and to work on them simultaneously. The semi-maximised state is not readily discoverable. Unfortunately, users are not yet taking full advantage of what Unity offers.

Overview of computer

October report: “Many participants wished they could have an overview of what resides in various parts of their computer, as is facilitated by Windows’ ‘my computer’.”April Testing: This is still a problem. Participants in the April sessions were still looking for a place where they could do systems setting and have an overview of their computer.Outcome: None of the participants discovered the ‘system settings’ option in the top right indicators menu. Users need an icon either in the Launcher or in the indicator area, or a folder in Nautilus.

Delete a document

October report: “Participants could not delete existing documents from their files and folders. “April testing: Everyone was able to delete a document that was no longer wanted.Outcome: This is fixed. One remaining problem is that many participants cannot see the Rubbish Bin at the bottom of the Launcher. They used other ways to delete, like pressing the delete key.

Copy and paste

October report: “Copy and paste from one document to another didn’t always work for participants.”April testing: Everyone was able to copy and paste from one document to another.Outcome: This is fixed.

Lack of feedback

October report: “Unity is often slow, and as a result participants tended to be confused about what was going on.”April testing: Overall, and as noted earlier, the performance of Unity was much better and the system responded more readily to users’ commands. Some issues remain with feedback, however, for example, with the Rubbish Bin. Participants wanted to be alerted, either with sound or a message that their document had been moved to the Bin.Outcome: Confirmative feedback is necessary whenever users complete an action, like deleting a file.

Bug #750311(“Launcher – When a item is deleted by dragging to Trash, the trash should pulse once before the Launcher disappears”)

Nautilus search

October report: “When searching, participants didn’t know what the field and scope were that were covered by the search engine they were using.”
April testing: Many participants searched for applications successfully. However, there are still problems with search. Participants made inappropriate searches, for instance in Nautilus, searching for Sudoku (search that pertained to Applications Lens) and they did not get the results they expected.Outcome: This is partially fixed. Some issues with search are related to participants’ understanding of the structure of Unity. There should be some guidance hinting at the limitation of the search and thus, the kind of results that can be expected from the various search boxes in the various parts of Unity.

Adding an icon to Launcher

October report: “Many participants were not able to add a short-cut of an application to the Launcher.”April testing: Most participants were able, this time, to add an icon to the Launcher. Windows users, however, had more difficulties than the others did; they tended to look for options in various menus or right clicking on the icon.Outcome: This is partially fixed. The interaction is quite intuitive but, some users (particularly those using earlier versions of Windows) will require more guidance.

Reordering icons in Launcher

October report: “Most participants failed to reorganise the order of icons in the Launcher.“April testing: A few participants experienced some difficulty reordering icons in the Launcher because they did not have sufficient feedback to understand when the icon had actually been selected so that they could proceed vertically. Consequently, they tried to move the icon too quickly after clicking on it and the icon did not respond.Outcome: This has been fixed in the latest update by providing feedback on selection – the interaction shows the icon as if it was detached from the Launcher – and by allowing users to move the icons vertically within the Launcher.

Finding the Dash

October report: “The majority of participants who found the Dash found it by accident. They were not sure what it was, and didn’t know how they had gotten there if they accidentally had.”April testing: Participants still cannot readily find the Dash.Outcome: The Dash needs to be made more visible and promoted as a major feature of Unity, on a par at least with the icons of the Launcher.

Bug #764771(“The BFB is visually lost and his position does not communicate its importance”)

Ubuntu Software Centre

The same features of the Software Centre were not tested this time because everyone agrees on its need for redesign and its existing usability problems. Nevertheless, some issues emerged in the course of testing other interactions.April testing: The Software Centre is still not recognized and, during testing, was mistaken for ‘systems control’.Outcome: The Software Centre needs to have a different look and feel and general presentation. Needs redesign.

Changing the wallpaper

October report: “Many participants did not succeed in changing their wallpaper because the default screen of appearance was open in full screen by default.”
April testing: Almost all participants were able to change the wallpaper by right clicking on the desktop. Furthermore, one participant who was able to find ‘appearance’ had no problem changing the wallpaper because now, the screen opens in a way to provide visibility of the background. The October usability problem was thus, fixed. However, a new problem emerged.

In the April test, the target feature was, in fact, the ease of use of the Applications Lens by means of changing the wallpaper. Most participants were not able to change the wallpaper by finding ‘Appearance’ in the Applications Lens. They were looking for ‘system settings’ to do that operation.Outcome: The initial problem with the appearance screen covering up the immediate change of wallpaper and so, hiding the change from users, has been resolved. Now, by default, the appearance screen does not open full screen. In the April test, however, users could not find their ’system settings’, where they expected to make these changes. Furthermore, many participants did not think of system settings as an application and, thus, were not confident to find it in the Applications Lens. Unity needs to provide obvious access to ‘system setting’ and make a distinction in the Application Lens between applications and other programmes.

Visibility of Files and Folders and Applications Lenses and Rubbish Bin

October report: “Participants thought that the grey icons at the bottom of the Launcher were inactive.”April testing: These icons still have issues of visibility, especially when they are folded at the bottom. For example, most participants did not find the Rubbish Bin. Another usability problem that arose from interacting with the Launcher is that some participants found it difficult to interact with the bottom part of the Launcher. They found that it was ‘a long way to go’ to the Rubbish Bin or the Lenses when the Launcher was populated with many icons.Outcome: These icons still need more visibility. Changing the colour, and perhaps even changing their position in the Launcher, might help.

Some Usability Issues that Have Arisen from Some of Our New Design

Top Menu Bar

The top menu bar is actually a new design. There was some confusion about the role of the top menu bar: Participants wondered if it pertained to ‘the computer’ or to the application they had open at the time. When participants had many windows opened, they did not understand that the bar corresponded to the selected window.

System Settings

During testing, I encouraged participants to change their wallpaper in another way than by right clicking on the desktop to see if they could find ‘appearance’ in the Applications Lens. Finding system settings programmes in the Applications Lens is not intuitive. Most participants did not succeed in changing the wallpaper by going into the Applications Lens. They were looking for a ’system settings’ icon in the Launcher or somewhere in the ‘Home’ at the top of the Launcher. Those who went into the Applications Lens, did not expect to see ‘system settings’ in that area because they did not think of system settings as applications and accordingly they did not explore. No one discovered the ‘system settings’ option in the drop down menu under the ‘turn off’ icon in the indicators menu bar.

Notification of message

This is also a new feature since the October testing. The majority of participants did not see the notification that they had received a message. The change in colour of the icon was not noticed. However, some noticed the change in the icon in the Launcher, in this case the Xchat, and they induced, by looking at the number that appeared on the icon, that they had received a message. However, when the Launcher is invisible, participants were not aware that they had a message.

This said, a couple of participants saw the notification and the change in colour of the envelop in the notification area. They had a strong positive impression of the feature. It seems that in this case, it might be a question of making the change in the notification area more prominent.

Semi-maximised state

Again, semi-maximised state is another new feature. Semi-maximised state is not readily discoverable. Only one participant discovered it. This participant was a Windows 7 user and said that there is the same feature in Windows 7. Two other participants interpreted the blue preview shadow as signalling that they were about to make a mistake or to do something not allowed by the system. The preview shadow was interpreted as a warning. Users need both guidance and reassurance here.

We are doing better with the user’s experience and our users are closer to adoption

Overall, participants left with a strong positive impression of Unity after having tried it for 60 minutes. Some of their closing comments:

“I like the layout and the screen (…) I want to customise it myself quite easily. It would be good to have a tutorial. (…) I like minimise and the fact that you can move things around. I like the casual font, aesthetically, it looks nice and it is easy to use. Nothing is really difficult. The important things are there and easy to use. It is nice.” [P1]
“The reason it was annoying today is because it is a new package. I like the design and layout. Design is important to me. It is quite clear. (…) “I would like more time to play around with it. It’s Ubuntu, I haven’t used it. This is new, the way I learn is by playing with it. (..) It’s good to use something that is a bit more independent. I like the idea that we can do things rather than being locked down in something more siloed like Windows or Mac. I would like to get it.” [P2]
“I prefer this set up to the start menu. I like the icons. We are a generation to see things with icons. I think there is a lot of significant gesture, like saving documents and I would not have any problem doing these activities. I really like the dragging format. I like to be able to order what I want. I think it is much easier than Windows. With Windows you have to go down menus. (…) I don’t think it’s complicated but it would take some time [to get use to it]. I’ve been working it out in an hour. It’s very user friendly. Even within the hour, I’ve learned a lot about how to do different things.” [P3]
“I really quite like it. I think it’s intuitive with the exception of the favourites, making an application a favourite. I would not be baffled to use it without a manual. I like the look of the desktop. It is modern. It looks like a Mac more than Windows. It’s quick.” [P5]
[About the Software Centre] “I didn’t anticipate to have access that easily to new apps. Also, I like the rating on the side. It’s quite helpful, I can see what I can trust. That’s quite nice.” [P5]
“It’s OK. Quite intuitive but I was going from what I know from Windows. I use the right click a lot, it’s nice to have it on the side. Generally this looks pretty good. It’s a bit more intuitive, for me, though, the right click is vital. It always brings up a good menu.” [P9]
“I think it’s very pretty, very pleasing as it were.” [P11]
“It’s quick and responsive. It’s very responsive, different from what I use, it would take a day or two to get acquainted. I wouldn’t be discouraged. I would rather spend time than pay money.” [P12]

In the summary of their experience post usability testing, participants also highlighted their main difficulties. It is meaningful that, at the end of the session, the following first came to mind:

“I don’t like the dragging in Launcher up and down. I mean I didn’t realise at first this is what I needed to do. It’s difficult to get to the Bin. It’s not easy to get to the top from the Bin, it is hard to drag things down a long way. I don’t like the dropping down.” [P1]
“My frustrations: I would like to know how to change the settings, I expect a button to change wallpaper clicking on a button right at the top. (…) The menu at the top bugged me.” [P2]
“I didn’t like when I have things minimised. There are many things I can’t do without maximising the screen.” [P3]
“It is hard to delete a file in this way. (…) You don’t find the menu bar and you don’t know what’s open.” [P4]
“I don’t know how to make the Launcher visible [when a window is opened]. I’m struggling a bit. This window [Dash] has a tendency to disappear.” [P5]
“I hated the Files and Folders, I didn’t know what it would do when I click on it, if it will open or just let me select it. I wasn’t able to select a document.” [P8]
[About the wallpaper] “I couldn’t find it. I wouldn’t have thought of it as an application for some reason.” [P10]
“I suppose my main thing is what I expected to have in terms of applications and control panel. I couldn’t find it. If I could have found this at the beginning life would have been a lot simple. I feel like I feel with Apple, I feel a bit stupid because I can’t do the things I normally do with my PC. I like things in words a lot, I like the drop down menu. This is interesting because this is generally shown with an icon.” [P11]
“I’m frustrated that I can’t find something like ‘my computer’. I want to find information about ‘my computer’ and what the hardware is, the driver versions, and I want to know if there are updates on Explorer. Here you need to go into ‘control panel’ to see if there are any updates. I still can’t figure it out.” [P12]

This time, I had the pleasure of working with Andreas Nilsson, who came to London to observe the sessions. It was very useful to get his feedback and to work collaboratively with him on the analysis and implications of the findings. In addition to these benefits of our work together, there is an added one: since he observed participants struggling with certain aspects of the interface, he will no doubt be a very effective user experience advocate with his team.

Andreas, thanks for your time!

The Test

Twelve participants were recruited from the general public – one turned out to be a no-show. They represented a mix of gender and age. Special consideration was given to heavy email users. Of the 11 participants, 5 were exclusively Windows users, 3 were exclusively Mac users, and 3 used both Windows and Mac.

In preparation for the sessions, we set up 2 test email accounts. A few days prior to the sessions, I sent messages to these accounts and also subscribed them to mailing lists. When participants signed up, they had already received a sizable quantity of emails, allowing us to ask them to manage the messages the way they generally do in their own email boxes, to find specific messages, to create filters, and more.

Thunderbird was tested on Maverick and Unity.

Between sessions, Thunderbird was removed and all hidden files were deleted, so the next participant got to start from scratch.

The Methodology

Over the 60 minutes of each session, I went through as many features of Thunderbird as possible with each participant. Participants were asked to:

Install Thunderbird from the Software Centre

Create an account

Sign up

Create filters

Set up alerts

Manage emails in folders

Create a signature

Change the colour of the font

Create a contact list

Search for a specific email discussing a form (which I had sent prior to the session)

Respond to an email that contained an attachment: in particular, open the attachment, modify it and send it back to the original sender

What participants liked

There were many aspects of Thunderbird that participants enjoyed, and many tasks at which they succeeded.

Participants commented positively on the tab system, which makes the navigation between messages easy and immediate, and which provides visibility on multitasking. The tagging of messages also got positive evaluation. Many participants commented on the simplicity and usefulness of the contacts. Filtering was perceived as effective, although, as we will see below, the majority of participants experienced some challenges here.

Participants found the activities which they carry out most often – opening, reading, responding to and deleting emails – easy and straightforward.

Where the trouble is

Critical issues

Participants encountered few critical usability issues – by ‘critical’, I mean issues that would make it difficult or even impossible to use the application on a regular basis. These issues need to be addressed if we are not to lose users to alternative products.

Install

After installation from the Ubuntu Software Centre, participants could not find Thunderbird to start using it. They did not see, in the product description, the bread crumb indicating the location of the download when provided.

Observation: After having installed a new application, users generally are excited about using their new software. The user experience would flow much better if, as the process of installation ends, the application opens automatically in the main window, allowing users to deal with their settings and messages right away. We need to keep users excited about Thunderbird. As it stands, it is a bit of a let down to not be able to find the new toy!

Create folders

One of the main challenges for participants was managing their many emails by creating folders.

Most participants did manage to create a folder by right clicking on the folder area. However, they could not find the folder once they’ve created it, and so couldn’t drop messages into it. This was because they had in fact not created a ‘folder’ (as promised by the menu label) but a ’sub-folder’. The sub-folder was not visible, because it was hidden under a folder.

Those participants who did eventually find the sub-folder they had created wanted to make it into a folder, but were not able to do so. Users normally organise their folders in a way that facilitates theiruse of emails. They tried to drag their sub-folder out of the parent folder and relocate it.

There are 4 main folders they want visible: inbox, sent, junk and draft. It is worth noting that the ’sent’ folder in Thunderbird is a sub-folder of gmail; this was confusing to participants. As a case in point, several participants failed at checking if a message they had sent me had really been sent because they couldn’t find the ’sent’ folder at all.

Participants also expressed a preference for ordering their folders. In addition to the point mentioned just above, some indicated that they like to create a work folder and a personal folder. They place these folders next to each other. They were not able to do this in Thunderbird.

Observations: Users manage their mailbox by customising folders. The level of customisation they need goes beyond creating and naming sub-folders. They want to create their own hierarchy of folders and sub-folders as well as to order them for convenience and visibility.

One more thing on this topic: participants were not clear about some of the words used to describe folders. For example, they did not know the difference between a ‘folder’ and a ‘local folder’.

Create filters

Most participants failed at creating a filter.

First, they didn’t know where to look to set filters up. Most participants looked under preferences and account settings. After looking generally at the menus they gave up.

Second, participants were unsure of the meaning of the dialogue boxes and of what was expected of them. They found the process of setting a filter unduly complex and they needed more feedback to measure their progress.

After participants managed to create a filter in the filter rules dialogue box, they clicked OK but didn’t know if the filter was actually set up or not. Additionally, they couldn’t figure out how to run a filter they had created. The issue was that, once having created a filter, when participants came back to the message filters dialogue box, the last filter set up is not selected – thus the run now option is not enabled. At the same time the enabled check-box is selectedindicating that the item has been selected.

Observation: After setting up a filter, users would like to run it to confirm that it works. Make the command ‘run now’ the next step in the process without users having to specifically select the filter to run it.

Find open and modify an attachment

None of our participants was able immediately to find the attachment in a message. They expected the attachment to be visible at the top of the message. While most participants eventually found the attachment, some didn’t, and consequently could not open and modify it.

When participants did not find the attachment, they consulted help, but were not provided correct information.

After some participants found the attachment, I asked them to edit it. They did not expect that the attachment would be in a read-only mode and tried to edit it without saving it first. The message warning them that the document is read-only only appeared after many attempts. It would have been friendlier for the message to be shown at the first attempt.

A few participants, after they attached a document, were not clear if the document was in fact attached to the message. They needed a stronger visual cue.

Observation: Sending, finding and reading attachments are fundamental activities on email. The user experience would be greatly improved if the attachments were located where users expect them, at the top of a message and/or if they would be more visible by changing the appearance of the link or using a colourful icon.Additionally, users would benefit from some immediate feedback on ‘read only’ documents as well as from a confirmation that a document has been successfully attached to an email.

In this case, for Thunderbird to be user-friendly, it would need to anticipate users’ needs, mainly need for visibility and for feedback at the first occurrence of an error. This anticipation of users’ needs would show the willingness of Thunderbird to collaborate with its users and to recognize their goals.

Search

Participants were unclear about the differences between the 2 search boxes at the top left of the screen.

Often they didn’t get results because the global search bar doesn’t suggest anything other than names.

Search doesn’t take into account misspellings – and so, when a word was misspelled, participants got no results.

Every time a participant performed a search, a tab opened automatically even if the search provided no results. As a result, participants opened many tabs that were not useful or wanted. They found that the tabs cluttered the interface and made it difficult to find such things as the inbox.

Observation: Users should know, before searching, what the fields will be actually searching. The area dedicated to filters is interpreted as a search and not a filter by participants. In part, the issue for users is that the boxes look virtually identical, and thus, from their point of view, should be interchangeable. A different visual treatment would greatly improve the usability of the different search boxes.

Less-than-critical issues

Participants also highlighted usability issues that were not critical, but that compromised their enjoyment of Thunderbird.

Mail account setup

Participants did not understand the message contained in the mail account setup dialogue box. They had to make a choice between:
IMAP – Access folders and messages from multiple computers
POP – Download all messages onto this computer, folders are local only

Uniformly, they did not understand the implications of this choice and went for the ‘recommended option’ – just because it was recommended. Most participants said that they would not read the message anyway and would just accept and move onto the next screen.

One participant chose the POP option, which caused her problems with search later.

In addition, the mail setup message has a button that says “create account”. This was confusing for some participants who thought they had already created an account and now were in doubt. Some wanted to go back to the signup page to check. There is no way to come back to the signup page, however.

Observation: While users are setting up their account, they are most eager to get the process over with. This is in part, because they want to see the application but also because they need to see what they will get, so to speak, before they can understand the pertinence of the various options proposed to them. In this case, it is good practice to make a recommendation – which simplifies the process. However, the choice should be clear, from the user needs perspective (so users don’t just choose what is recommended because it is simpler and don’t foresee the consequences of their choice).

Set up alerts

2 participants expected to be able to set up alerts in tools, but were not able to. Many participants were not able to find a way to set up alerts at all.

Create a signature

The majority of participants expected to be able to create a signature under ‘composition’. When that failed, they looked under tools, add-ons, preferences, insert and write. Most who wanted to did not succeed at creating a signature.

Some minor issues

‘Minor’ usability issues don’t compromise the main usage of an application or the integrity of the user experience. However, they can be annoying and irritating, particularly when the application is used on a regular basis.

Change the colour of the font of their message

Most participants either failed at changing the colour of the font for all their messages or were not sure they had succeeded after selecting a colour from the palette of ‘display’ in ‘preferences’.

In part, participants could not find the option to change colours. For those who found it, when they selected a new colour, the new selection was not reflected in the messages they wrote just afterwards. But also, after they selected a different colour, they were not sure if the chosen colour would appear in their message.

Observation: Many users like to personalise their communication. Playing with colours and lay-outs should be easier for them, with relevant options more visible. In addition, users need some feedback that their change will be immediately implemented. An ‘apply’ button or a confirmation that the new selection has been registered would reassure them.

Navigational issues

Participants did not know how to get back to their inboxes from either the address book or the ‘write’ screen. They didn’t understand that in these specific cases new windows were opened, instead of tabs, and that they needed to close them to go back to their in-boxes.

And more…

Participants had further suggestions for new features. They wished for:

A calendar on the side so they can see messages and their commitments at the same time

A way to compress large files directly from the email account

Some social networking, at least so that they could see that their friends are online

Hey Ubuntu One and Design fans! This is my first post here, and I have to say I feel priveledged to be able to write to you all. Recently I’ve been working with the Ubuntu One team on the desktop syncing apps, and trying to give them some special attention. I feel like these apps have the potential to be such an important part of not only the Ubuntu experience, but also the experience of users who may not have converted over to Ubuntu yet. As such, these are some of my personal motivations for my design work on Ubuntu One:

1. Make a good first impression

Ubuntu One has a great opportunity (as a cross-platform application and service) to be a bit of an evangelist for Ubuntu. Just like iTunes and Safari have been evangelists for the Mac experience on Windows, the Ubuntu One desktop app should introduce people to all the best things about Ubuntu. I want users under Windows to see and experience this amazing application and be hungry for more. I want them asking how to get more of our software on their computers. And with that, have an expectation that all the software we ship is going to be better than what they’re used to. Which is why I think it’s important that we…

2. Set a good example

Ubuntu One is one of only a couple of applications that users will see that was actually built from the ground up by Canonical. In this way, it should set a good example for others to follow. If we can’t provide applications with great design, we can’t expect our partners and community to. It has been said by Mahatma Gandhi that, “When the people lead, the leaders will follow.” In this way, I want to set a positive example with this small part of the desktop that every other application is going to be envious of and strive to acheive. This leads right into my next goal:

3. Offer a Superior experience

Above all, Ubuntu One should (as all of our work should) offer a superior experience simply for the intrinsic value of making our users happy. The Ubuntu One team firmly believes that the best way to attract more users and developers is to build something so great that people naturally want to be a part of it. As we heard from MPT (in his now quite infamous talk at UDS Natty), Ubuntu needs to be Useable and Keepable. That means providing a specific kind of desktop experience. Not only one that “gets the job done”, but one that people are going to prefer over any other available experience.

Two weeks into our wallpaper contest we’re seeing lots of really interesting entries for the photographic part. As those of you who read the previous post about the contest will know we’re also reserving at least three places this release for non photographic wallpapers. The plan _was_ to use another site to manage these entries. Unfortunately the dedicated community team working on this site have hit some issues with keeping it running and so we’ve decided to move all entries to Flickr. I’d like to take this opportunity to thank them for their hard work and we’ll explore this solution again next cycle.

Metal & Bolts Sculpture SW 6th 1 of 3 by Orb9220

In the meantime we’ve a Narwhal who simply has to look Natty and we need something robust and reliable so that you can all focus more on creating and getting your entries to our judging panel and less on trying to upload. As before with the photographic entries simply tag your image NattyWallpaper so we can easily find it when judging time comes.

While you’re hard at it snapping, sketching and creating our future desktops we’ll be contacting previous winners and asking them to prepare to look through all your entries. Good luck and have fun!

??’Appropriation’ – the taking of a product and using it for one’s own purposes, in ways unintended by the product creators – is implicitly at the core of the philosophy of opensource, because openness provides for change, adaptation and innovation.

Design specifications

Last year, I conducted several research projects to understand how developers work and how we can add design and a concern for user experience to their already very complex efforts. I’ve published some of the results already, for example, those coming out of our study of Empathy.

One of my inquiries asked how developers use design specifications. This research produced very rich results. We realized that developers’ approaches to our design specifications documents varied quite a bit, and often the documents were not made as central to the developers’ work as we had anticipated. So far, we’ve been able to characterize four different ways in which developers use our documents. These illustrate tendencies and not necessarily rigid approaches. Yet, they help us understand developers’ frame of mind when they deal with design information.

1) Some developers read meticulously specifications and try to figure out what the designer had in mind. These developers like to work closely and collaboratively with individual designers.

2) Some developers have a more organic approach to understanding specifications. They use them in combination with current conversations on chat networks about development topics and issues, and with other conversations that have taken place during past strategic meetings at UDS. They essentially make the written specifications a second resource in favor of what their colleagues and managers say about them. They fit the specifications in a dynamic broader context.

3) Yet other developers almost only look at screen representations of the specifications. They try to duplicate the visual guide that accompanies the specifications or simply to compare existing features of an application to the screen shots included in the document, trying to discern similarities and differences between them. Many of them use specifications documents as a simple reminder of past and current discussions and to get a general idea of what’s expected of them.

4) Finally, another group uses the “try and see” method. These developers implement changes as they see fit and rely on their colleagues to provide guidance once the development has been realized. Effectively, they hardly consider the written design specifications at all but like to follow their intuition.

Research, of course, doesn’t judge what people do because it appreciates that people do what they do for a reason. Furthermore, it doesn’t opine on which behaviour is the best – because people do things in a way that works for them in their situation. What research does is understand the complexity of individual situations and help designers fit seamlessly in people’s contexts and frame of mind what their products offer.

Based on these, and related, results, we have been rethinking our design specs tools and experimenting with new concepts derived from co-design principles, so that these specs become helpful to all developers and enhance their work and not represent merely an external constraint to the work they do.

This is all good. However, the issue is not restricted to our Ubuntu developers. We should not forget that, in the wider opensource community, many developers do not have access to the Canonical, or any other, design team or to anyone with solid design training. They are the developers who work on their own free time and produce amazing software. They have to wing design. Many wish they could access such skills to help beautify and enhance the user experience of their products. These contributors deserve our support.

So what?

To us, the solution appears to be ‘design appropriation’.

Our challenge: how can we create design specifications and design thinking tools that developers can ‘appropriate’, just as mobile phone users started to use their phones to text because it suited their needs even though texting was not considered by the phone first creators to be a very important feature?

How can we design for the unexpected?

Upcoming research this year will be concerned with what developers can teach us about ‘appropriation’ of design.

This represents for us a first step in the investigation of the potentiality of ‘appropriation’ for all opensource. Ultimately, appropriation should be possible not only for developers but for all end-users.

The theme of this week’s update is one of cool stuff that we came across this week in the world of open source.

You’ll know from Christian’s blog post a while ago that we’re really interested in the idea of interacting with your computer physically. Christian experimented with the webcam in a laptop but imagine what could be done with a Stereoscopic box of tricks like that clever Kinect setup for the Xbox!

Well, wonder no more because as it transpires we can play with it on Linux right now. Both Engadget and OMG! have written up stories and it’s quite exciting to see that this is working so soon after its release. If you’re interested in how it actually works you can head over to Wired who wrote an interesting piece about it for those with a taste for popular science.

Staying with exciting interactive technology John Lea on our team came back from a conference in Germany this week, The ACM International Conference on Interactive Tabletops and Surfaces – no less. Some representatives from a company called Archimedes Solutions demonstrated a tabletop surface running games. You can see a video from April this year showing off a castle defence game amongst others on Youtube.

Away from the new hotness the reason we’re a bit quiet is that we’re cracking on with the Natty cycle and getting our ducks in a row. There’s a lot of activity on Launchpad and we’re engaging with the artwork community. More on all of this later!

Phew! What a fortnight! Did you miss us? We all got back from Orlando, Florida this week and prepared to kick off the Natty development cycle in style. UDS-N was a great event and big thanks should go to Jono, Jorge, the Canonical admin and technical teams and everyone who attended sessions, took notes and participated. I took a lot of photos and I know others did too so go check them out as well as catch up on the planet to see what people talked about while they were there and now they’ve recovered from the jet lag!

The big news was obviously the move to Unity on the desktop and Mark has been blogging about other changes that will come to your favourite Linux desktop in April 2011.

Oh and finally I saw something exciting today to do with Ubuntu and robots! The Qbo project uses Ubuntu as the base for their development distro and they’ve released their first alpha. Charline, Otto and I sorta love robots and are secretly hoping that by talking about these guys they’ll send us our very own personal open source robot

We’ve been talking a lot about UDS but some of you out there might not know what it is. Duncan McGregor, one of my colleagues, has had this very conversation with a speaker who will be attending this UDS. He took some time to compose a blog post explaining what the summit is all about.

It’s out! We did it! YES! Well done everyone. Thanks for your support in helping make this latest release a good one. Our Meerkat is all grown up so what’s next?

Well, we have a Narwhal on the way and it’s up to all of us to make him that little bit Natty

For a kick off Andrea, a community member who has worked with us on the enhancements to the theme in the new release, has been hard at work thinking about the future of the Murrine theme engine. This engine is the beating heart of our gorgeous default themes and before we’ve even really started on Natty he’s upgraded it to work with the latest version of GTK. As we’re not sure what’s going into the next release just yet we can’t say for certain if all this work will make it in but what we can say is that if you’re a brave and heardy soul you can head over to his blog and get it for yourself.

The main thrust of our week has been getting ready for UDS in 2 weeks time. That has meant copious amounts of planning and preparation between us and the DX teams as we review the desktop and Unity and start to plan for the next cycle. We’ve also been conducting user research of Unity this week with Charline seeing what Joe Public thinks of our shiny new netbook sofware.

If you’ve not tried it yet you really should!

Have a fabulous weekend and we’ll bring you news from Orlando next week once we’ve prepared ourselves for an outstanding UDS!

Oh and one more little thing before we go. We’re going to have something exciting to talk about next week which can be used by anyone out there wanting to promote their love of Ubuntu. More on that when we’re a bit more ready and I don’t have a report to get out but in the meantime here’s a teaser

It’s release week! That means that the Millbank London office and home office locations this week are a buzz with activity getting things ready for our big release on Sunday.

The members of the kernel team, and others, are modeling our very swanky new banners which are going out to eligible loco teams in the not too distant future. They should stand out at promotional events and attract some attention

Away from cool assets being produced we have been focussed this week on the release and by now the beautiful linux desktop you early adopters are already looking at is hopefully starting to feel like home. We can’t wait to hear what you think and hope that you love using Maverick.

So there’s this big release happening on Sunday but I’ve just heard about another release that’s happening on Sunday that’s just as big! The new theme will hit the Ask Ubuntu stack exchange site and we’re really pleased with it!

Big thank you to Mat on our side for working on this and Jin Yang at Stackexchange for helping realise these designs in glorious HTML in time for the release. You’re a splendid fellow for working so hard and so late into the night!

More next week, we still need to do some work for the release, so enjoy, tell us about your release party and roll on UDS and Natty!