When to listen to inconvenient criticism

Off the Beat: Bruce Byfield's Blog

Dec 28, 2012 GMT

Bruce Byfield

Whenever users complain, the members of a free software project have a time-honored fallback. Happy users, they say, rarely bother to comment. Mostly, only the discontented are moved to voice an opinion.

However, like all pieces of popular wisdom, this one deserves to be questioned. Is the idea that feedback consists mostly of complaints true, or a rationalization seized upon by those who -- very humanly -- prefer to dismiss criticism of their hard efforts? If so, how do you know when to overlook criticism, and when to take it seriously? And what are the comments of ignoring inconvenient criticism?

The temptation is always for project leaders to ignore criticism. For example, when Mark Shuttleworth recently answered questions for Slashdot, he defended recent changes in Ubuntu by saying that, ""The Linux distro market has always been highly fragmented and ideological," and that "if you pursue the future, you have to accept that not everybody will agree with your vision."

Shuttleworth went on to say he was, "quite disappointed that more folk don't appreciate that we have a once-in-a-lifetime opportunity to shift the world towards a much more open platform than ever before, but that nasty flaming of individuals who lead that effort, whether it's me or anyone else, is totally counter-productive."

In other words, the results will eventually prove that he is right and his criticizers wrong, and everyone should accept his vision of free software's future -- despite the obvious fact that many don't.

Similarly, Karen Sandler, the executive director of the GNOME Foundation, blogged a couple of months ago that fielding more complaints was the result of operating in an open and transparent manner, and of people's natural slowness to appreciate anything new. Part of the problem, too, she suggested, was "how things often work in the free software press," particularly the tendency for all the media outlets to repeat criticism, especially when it is made by someone famous like Linus Torvalds.

Sandler's explanations are more worth taking seriously than Shuttleworth's. Users can be conservative, and the free software media is sometimes too focused on criticism. However, when believed and followed too strongly, such beliefs have much the same tendencies as Shuttleworth's, encouraging developers to ignore criticisms, and to keep on with what they are doing.

But my main reservation about such responses is that they tend to depict responses to criticisms as a question of either-or: either you listen to every piece of criticism, or you ignore every piece. This position means that developers might miss useful criticism for no better reason than shielding themselves from the pain of admitting they were wrong or could do better. Instead of rejecting all criticism, my own experience suggests that a more useful alternative is to try and decipher when you should listen to criticism and when you should ignore it.

Learning when to listenMoose Finkelstein, the chair of the 2012 Ohio Linux Fest, suggests that the idea that feedback consists more of complaints than compliments, is a truism borrowed from retail. My own long-ago experience in retail suggests that she is right, but I am not sure it is accurate even there. In a high stress, often minimum wage job, one critical word can hit you harder than a dozen compliments, because you are braced for attacks. But whether the number of complaints objectively outweigh the number of compliments is another matter altogether.

Nor am I certain that, even if the truism is valid for retail, it remains true for free software development. When I was working in software development, it didn't.

In my experience, free software users do feel more free to voice opinions than the average customers of proprietary software. Just as Sandler suggests, community standards are a great leveller, and the members of a free software project, even when well-known, seem more accessible than the CTO or senior programmer of a Fortune 500 company. With that accessibility comes greater frankness and less politeness, both of which can be annoying at times.

However, free software users are far from slow to accept a new idea if captures their imagination or they see any value in it. I do remember a small current of criticism, but I remember far more encouraging comments. Even before the software was available, potential users would write to express their conception of how a new feature should be implemented. Some would suggest features, or uses for the application that none of the development team had thought. Many wrote simply to express their anticipation and support.

Even when a general release did not meet expectations --and what first release ever does? -- the criticism was mild. Faults or incomplete implementations were pointed out, but gently, and usually with the hope that the next release would be better. It was only if problems persisted over a couple of releases that the general tendency of the criticism might turn hostile.

When feedback was hostile, naturally everyone on the development team felt tempted to dismiss it. To expect those who are putting in the long hours and careening from crisis to crisis as deadlines loom to respond favorably to catcalls from the sidelines is asking too much of the average person, especially when the comments turn abusive.

Yet I remember one or two occasions when even the hostile comments, that by the wording seemed designed to undermine rather than support, nonetheless made points that should to be addressed. Tone, in other words, was not always an indication of validity.

Just as importantly, ego needed to be put aside before assessing what was said. Besides the question of whether a complaint was technically accurate -- whether the problem could be verified, and whether it pointed to an actual defficiency or just the user's misperception -- the important question was how many shared the criticism.

One or two users making a complaint might indicate minority responses that could be ignored, or perhaps shelved for later consideration. However, the more users who agreed with a complaint, and the longer they voiced it, the more likely it was something that should be addressed.

I first began formulating these generalities thirteen years ago, but I see no indication that they have become inapplicable. That is why, although I have no hard figures about just how many discontented users GNOME or Ubuntu have, I have always considered their user revolts a phenomenon that shouldn't be ignored.

The fact that no one can pinpoint the level of content to two decimal points, or without a large margin of error is only a quibble. At a certain point, the level of criticism becomes so loud that questioning it turns into deliberate denial. At the very least, complaints so frequent and so loud indicate something worth addressing, even if it can't be quantified with any exactness.

Moreover, since this is an extremely rough metric, it is probably better to be overly cautious, and consider even borderline cases. After all, there is always a chance that you have under-estimated the situation.

The marketing falloutAnother point I discovered was that ignoring or dismissing hostile comments was the worst possible possible. Both these reactions simply created the impression that a point had been made that the development team could not respond to, and was trying to ignore

Just as importantly, these reactions not only created an added sense of grievance, but one that tended to spread to other users. Either reaction made it appear that the developers were out of touch with users, or had no interest in their feedback. In response, users would either stop using the software, or else intensify their attacks in the hopes that they could make the developers listen.

Instead, it was far better to respond -- if only selectively -- indicating that the developers were listening and would take the comment into consideration. They might even throw the question open to other users, to get a better idea of how widespread the reaction might be.I first formulated these truisms thirteen years ago, but nothing I've seen since suggests to me that they no longer apply. If anything, the longer that free software exists, the greater users' assumption that they have the right to have developers listen to them is likely to grow.

What has baffled me over the last few years is that long-established projects like GNOME or Ubuntu can overlook such self-evident facts.

GNOME, I'm pleased to say, has started showing signs of coming to accept them in recent decisions such supporting a GNOME-like experience through extensions, and announcing a new emphasis on security and privacy. Ubuntu, by contrast, is apparently going to take longer to sync with reality, assuming it ever does.