Open source usability

It appears that a bit of focus has been made on Linux Usability lately with articles by Eric Raymond here and a follow up here, and a reply by John Gruber here with comments on OS News here.

Like all the most interesting things in life, these matters are rarely as straight forward as they appear to be, and clearly the subject is one that touches close to my heart - after all, I've spent the last 4 1/2 years doing original research into HCI and usability (mostly of search engines but also of end-user applications) - so I've decided to respond to it. Whether anyone notices or cares though is a different matter! ;^)

The one aspect of all these that is apparent to me is the idea that open source is inherently ill-suited to generating usable software. The arguments in favour of this tend to revolve around the idea that usability is an art form that requires a dedicated effort from a usability expert (or a team of experts), and the argument is based on 2 premises: usability experts don't "do" open source projects, and dedicated effort is impossible for the open source development paradigm.

However, this argument reminds me of similar criticisms of the open source method when it first started becoming famous. These tended to follow the lines of "open source might be fine for small applications or scripts, but nothing major will ever come out of it." Clearly this argument is not true: open source has produced some very big and successful projects, from the Linux kernel upwards. Indeed, virtually the whole of the Debian distribution is built on open source software (aside: I am aware that a lot of the larger projects are funded commercially, like OpenOffice.org coming from Star Office which used to be proprietary, or Mozilla). Even Eric Raymond noted his surprise at seeing his first Linux distro working ("Open Sources: Voices from the Revolution").

So is there really a qualitative difference between software development (writing code) and UI design (designing interfaces)? An interesting comment from Eugenia Loli-Queru at the OS News website mentions her husbands fine work on BeOS which came about through regular contact and continual contact with the kernel engineer. This kind of setup is ideal and can lead to truly fantastic software being created. However, I sense that maybe there is a belief that interface design can only be done this way, and consequently any other method will crash and burn. For a long time, this idea was prevalent among open source skeptics, but the proof of the pudding is in the eating as they say, and Linux (for example) does work, so distributed software development can work.

But how can the distributed development model apply to something as artistic as UI design? The simple answer is that there are few examples of open source software being built with a usable design in mind, but it would be mistaken to confuse an absence of evidence for evidence of absence. As the OS community becomes more aware of end-user issues, focus on designing usable interfaces is becoming more important. Examination of the "help wanted" pages of SourceForge shows software in need of user interface experts.

But the experts aren't there, right? Well, there aren't that many involved right now, but they are there. MoreGroupWare (a php based groupware suite) recently advertised for help on the interface, and was rewarded with approximately 10 volunteers with many years of collective design experience from all areas of the field: from php coders wanting to increase their skills and learn a little, right through to the lead usability engineer for a large company with decades of experience at the sharp end of the business.

Clearly there are people around, and this situation may well improve in the future. After all, open source is a great place to get experience that looks good on a curriculum vitae: it shows motivation, an interest in the field, and a willingness to work with others as well as the chance to work on things that are outside of your normal experience.

But how well will the design process work within the distributed model of development? It should be remembered that the design process often doesn't work within completely commercial environments. With the exception of my own academic work, I have never worked on a project from inception (this is a refrain that I hear all too often in the field), though undoubtedly there companies with a more enlightened attitude that realise that HCI people are best employed from project inception.

The current success of the open source model in terms of generating code in undoubted, but there remains skepticism that this success will translate into the domain of interface design. In strict terms, there is no reason to assume this: just as developers don't need to work in close proximity in a well-ordered office, interface designers also do not need to work any more closely with other people in the project team than the code writers do.

But then, there is the issue of user testing. Open source, often being run by volunteers, cannot afford user testing, so this could be a problem. However, this ignores one obvious factor: the user base.

OS software is known for its user base being an effective source of bug reports, patches and feedback, and there are few reasons to assume that this population could not provide the same service for the user interface. The major point against this would, I imagine, be that the target audience of most developer tools differs from that of end-user applications. Fellow developers may be more likely to make an effective bug report than most end users, but this is just my assumption. It is worth noting that operating systems are increasingly encouraging feedback from end users. OpenOffice.org features a dialog to send a bug report to the project developers if it crashes for example.

Interface feedback can also be made this way. If users are willing to point out flaws in the design, any interface designers working on an open source project will have the breadth of usability feedback that most proprietary solutions can only dream of having. Of course, this leaves the problem of having to assess this information, but that's another story.

As counter-intuitive as it seems, user interface testing does not need to have user testing: methods have been developed to enable empirical assessments of interfaces (see the references for more information on how to use these methods). These methods are ways of examining an interface to work out how well it can be used. Of course, user testing is important, but in the early stages of the process, there can often be little need to have an amenable test population ready to hand.

Interface design is an iterative process. Often, a designer will produce several ideas for an interface, and fully expect to change them all at some point in the process. However, as long as the design process itself follows reasonable guidelines, there is often no more need to have the same designer in charge than it is to have the same coders working throughout a projects life.

In the real world, there are many restrictions on the quality of work that a designer can do, usually in terms of time, expenditure, and company politics. All of these can severely hamper the development of an effective design, but OS projects can offer benefits to interface designers. Because the open source development process is rarely time limited (rather, it's a continual process), the time factor becomes less of an issue which enables designers to really go to work. For any designers who are sick of doing hacked-up interfaces at work, I suggest trying an open source project for fun: you can really get the chance to do a bang-up job on something. Factors like having the marketing department nag you will be a thing of the past meaning you can lovingly construct the best interface you are capable of.

Finally, because an OS project might get a team of developers (like the MoreGroupWare example above), the skills and talents of the team will likely differ considerably. This diversity of opinions and backgrounds could well be a favour in exploring possibilities that a regular team may not be capable of.

I think OS can produce usable software. While at the moment this may not be as much in evidence as it could be, once more designers see the benefits of contributing, the future is certainly bright.

As a final note, it is worth noting that many programmers considered the open source development model as being incapable of producing anything of real value or size: indeed, some still do (reference). However, the truth is that open source has produced the kind of software that only a major company code hope to produce, and the barriers fall. Anyone care to bet against it this time?

interesting approach: the audience you intend to reach is both proprietary and open source developers.

one of the issues with open source development is that open source developers pretty much "do their own thing". they are driven by completely different motivations. for a lot of open source developers, if you approach them in the right way - through the right channels - then bug reports, criticism (phrased appropriately and positively), suggestions and coding enhancements will be accepted and welcomed.

the risk - to open source developers - is that they assume that because they are independent and becoming experts in one area, that their expertise therefore extends to other areas, such as product useability, product marketing, the look and feel etc.

or perhaps, i dunno, open source developers assume that because it's functional, that the responsibility for making it useable can be somehow left to someone else to do at a later date (and in the case of KDE, with its use of QT, that _is_ in fact the case!)

everyone in the world has expertise. not everyone in the world is capable of recognising - or in some cases willing to recognise - where their capabilities fall short.

i am presently endeavouring to set up a linux distribution. the presentation and useability of today's linux desktop is still roughly five to twenty years behind its competition. i am going to have to find expert graphical artists and people with expertise in presentation in order to make up the stark difference.

surely open source developers should make the lives of such experts easier (KDE / QT is a good starting point, with its theme capability designed in from the ground up).

recognition of lack of expertise is not a failing: it is the shining light of very wise individuals. except for donald rumsfeltd of course.

in a commercial company, the link from users "i don't like it therefore i won't buy it" and "i didn't like it therefore i'm going to bitch like hell, i damn well expect it to get better otherwise i'm demanding my money back and i'm going to complain to everyone i know to make sure they don't waste THEIR money" to the developers is much more direct.

there's a manager, a director, a sales team, a marketing team, all screaming at the developers to "get it done".

and if it ain't gonna get done, the developer ain't gonna get paid (because nobody else will be!)

in other words, the developers themselves very rarely call all the shots, and usually it's for the right reasons (or at least the right underlying driving force): commercial success or failure.

sometimes that works, sometimes it doesn't: the successful companies continue to make products that users pay for (whether they like them or not but they "Get The Job Done" :)

... where that gets _really_ interesting is where commercial companies meet open source development. we're seeing some success stories - but it's certainly not predominantly in the desktop market, but in the server market.

the only problem i have with people that call themselves "versed in interface development" is that most of the time they consider that a text console is not an interface, while an "interface" can only be a graphical based thing. i always found that very idea just stupid, to express myself pretty clearly. i have never found yet a graphical interface that offered more than my unix console under text when it comes to handle Unix-alike systems. and i became fed of morons that forget that there are blind users around, the same way some fucked-brain morons design our cities and transportations systems, hotels, restaurant and most places never thinking about accessibility for disabled people of any kind. that's the point. an interface is a presentation layer, not a control one to me. while the presentation might differ in its forms, when it comes to control i go the unix console way. and i became tired of explaining the same fact-based things to people that like to reinvent the world and believe they have the solution to all. the same way we got people coming with the "C remplacement langage" every year or so. i'm fed up :/

I know what you mean regarding console efficiency. I believe that my tutor for my undergrad degree (Jan Noyes) performed a study comparing the Unix console with the NT GUI interface, and guess which was most efficient? Yup, the console... I cannot find the study offhand but can let you know when I dig it up.

I think a lot of usability people concentrate upon the GUI as it is most users primary interface with the computer and it relies more upon recognition than recall. With a console, a user needs to know the command before it can be used - with one or two minor exceptions - whereas with the GUI even an inexpert user might be able to figure out what they have to do. Even though it is probably quicker to examine a man page and learn the correct command than spend hours hunting around for the correct places to click, users seem to be happier clicking around randomly trying to find their way rather than asking for clear directions. Many usability companies are becoming aware of access issues these days, so the world might be catching on (see http://www.stakes.fi/tidecong/112beck.htm for an example of what you state).

I find it curious that you mention blind users - it's one of my main criticisms of the zooming user interface which I'm currently investigating. Sadly however, you are correct in that access issues often take a poor second place with many investigators.

Finally, what did you think of the article? Was what you said just aimed at usability in general, because I couldn't see the relevance to the point I was trying to make (namely that the FOSS community has a potentially bigger test population than any usability company can access)? Sincere apologies if I've missed something.

salmoni: I couldn't see the relevance to the point I was trying to make (namely that the FOSS community has a potentially bigger test population than any usability company can access)?

I guess a criticism I could make of the article then is that this wasn't clear: I thought the main point you were trying to make was that FOSS was an appealing place for usability experts to hone their craft because of its freedom to make experiments.

Simply harness the user base as a huge test population for UI
innovations.

Try to attract HCI experts into developing good UIs.

I think the problem with both methods is one of rigour.

For method 1, one can't test if a UI is "good" in the same way that one can
test if this kernel driver works properly with that device. If a driver
dies, then it dies, period. But if a UI is "bad", it may just be because a
particular user has weird tastes. And very often, people who are satisfied
with a piece of software tend to keep quiet.

A similar problem exists for method 2. It's hard to separate the true
experts from charlatans whose UI designs are based on religion and circular
logic, as gilbou
pointed out. And the fact that Jef Raskin has become a sort of prima
donna in academia isn't helping much.

So I think while usability for OSS is possible, achieving it will require
some creativity.

lkcl: Linux is only
5 to 20 years behind some people's idealized notion of what the
"competition" is like. I think it's only 1 year or so behind MacOSX (perhaps
its low mindshare is because Macs are too expensive?), and Windows is
just a huge joke.

I have found that the best predictor of failure of a software project is it social organization. It's not a surprise that the projects with the best usability are those that have actively sought out and involved in a meangingful, respectful way the opinions of the target end users (customers, if you will).

Many, many, many open source projects are centred around a "hero" (or heroes--it could be a small clique) who puts too much emphasis on themselves in the project rather than splitting work up to others. These projects will have poor usability simply because the hero will code for him or herself rather than another party.

Alternatively, the usability of the software is very good for the target market, but the target market is a relatively narrow market, and so the software does not make sense for other market segments. It's a cliche that open source software is "programmer-centric" but you can also find other open source projects that serve niches other than programmers well, but not a general audience.

The complaint there is not enough money for user testing I think is a red herring. Similarly, there is not enoug money for software testing, but the software gets tested.

The simplest solution to the problem is simply then to build an ongoing relationship with the people that you want your software to serve.

I recognize that you probably knew that, but I just thought I'd reinforce that idea a little.

i have found a couple of fantastic little projects: fireflier and basket. both are not perfect by any means, but they represent an extremely useful step in the right direction.

the first one is called fireflier, and it is an on-demand firewall program that uses the [experimental] ip queueing module. packets are queued pending approval by a popup program.

fireflier can even track down the program that was doing the network traffic, and you can vet TCP and UDP traffic on a per-program basis. the only thing that this _doesn't_ catch is of course TCP "reset" packets, which come in _after_ a program has exited [unexpectedly?]. oops :)

fireflier is packaged up in debian so it's an apt-get install fireflier-server and apt-get install fireflier-client-kde/gnome etc. etc.

the other program is called basket, and it is a drag-drop program for being able to run other programs, store email addresses, URLs, contents of clipboard etc. etc. it is very smart, and extremely useful.

people who use it regularly don't bother clicking the start menu any more: they put everything they need in different tabs and access things from there.

the guy who wrote it isn't quite yet plugged in to the open source community: he just wrote a cool program and people use it. i'm very surprised that this program hasn't yet been debianised or made its way into the standard kde distribution.

New Advogato Features

New HTML Parser: The long-awaited libxml2 based HTML parser
code is live. It needs further work but already handles most
markup better than the original parser.