Random thoughts on policy for technology and on technology for policy

One of the main web2 trends in public services is about user-generated ratings. I see this is becoming quite spread, from patientopinion to fixmystreet to ratemycop and more. It’s one of the few web2 trends which is getting real foothold. Of course the key aspects is that this feedback is public (many-to-many).Today I read that the italian minister for public administration, mr Brunetta, proposed/decided (in Italy you never know) to let users rate public services via SMS smileys (positive or negative). It struck me because it’s an idea that came to my mind recently: imagine the impact when 3g costs will come down, and people will get used twitter live from the queue in a public office. This will change public services for good. There you have what I call “casual participation”.But I don’t think Brunetta’s proposal could work, first because it’s not about making this feedback public, but especially because quantitative feedback does not work well. You trust feedback when you can read between the lines and understand the person has good arguments. When you don’t have authentication such as in blogs, it’s the content that attributes trust to the person, not viceversa.Quantitative feedback is instead very simplistic and much easier to manipulate, as the problems eBay is experiencing now show. In particular, it can give rise to easier destructuve behaviour, as ratemyteachers.com showed.So in summary, the minister’s idea is nice but simplistic, qualitative feedback is better than quantitative, and feedback data should be public.

6 thoughts on “users ratings of public services? better qualitative than quantitative”

There’s a great deal of truth in this post.
After 4 years of working on Patient Opinion, we are coming to feel that stories have a lot more to offer than ratings. Why? Partly for the reasons you give:
– ratings are easily manipulated
– therefore trust is difficult to establish
– and there is a spurious idea of “accuracy” in a measure which must clearly be biased, either deliberately or simply because of response biases
– ratings are not actionable – it is very hard to say how we should “improve” the rating (apart from cheating)
– but people like them because they are easily compared and aggregated
What about stories, like the ones we publish on Patient Opinion?
– they are hard to summarise and aggregate: you have to read or hear each one
– but they are much more resistant to manipulation: especially if organisations are judged not on their stories, but on their responses to stories
– and they often tell you exactly what you need to do