Posts filed under ‘analysis’

Rules systems for moderating debate have been around since antiquity. In live debate this has always meant moderating time. One team gets to speak for a period of time and then the other gets to speak for a set period of time. There are many variations on this theme but generally speaking each party gets an equal amount of time. The only variation is in the length of the time segments and who gets the almighty last word.

Time is scarce commodity in live debate so moderating live debate is almost entirely the business of regulating time. The web is different. There are no time constraints and the storage capacity when it comes to text might as well be infinite. The only scarce resource on the web is human attention. Dividing the resources of human attention is the single goal of online debate moderation. Dividing this resource comes down to answering questions like who gets to have their argument highest on the page? Who gets to be left most. Where to rebuttals go? Where to arguments that back of premises go?

If you want to see why user moderation sucks for debate you need look no further than the Ron Paul echo chamber that Digg.com was for the period of Fall 2007 to the Iowa caucus. Take this article for example where the submission is some footage of Ron Paul that was apparently (I can’t say if it was) cut from the re-air of the Live debate.

“yep, I saw it on re-air as well. – innit great how people dugg you down for calling bullshit on the headline?

/not voting for him but I agree w/ everything RP said” — JimSwarthow

And the most dugg down comment on the whole page:

“The problem with this digg submission is an example of Ron Paul’s problem. . or I should say his followers. This was NOT cut from the debate. I and EVERYONE else who watched saw this part. This could have been a good submission, by blatantly lying is not gonna help Paul.” — Cyberdactyl

The unfortunate thing about Digg’s style of user moderation is that it inevitably ends up being used as a tool to shout down dissent. This does more then just discourage the unpopular side from participating further. It creates an illusion of consensus that breeds complacency in proponents of the locally popular idea.

It’s incredibly hard to get relevant opinion texts to rank high on a search engine. This is the case despite of the wealth of opinion texts on the web in blogs, newspapers, forums and elsewhere. Texts from a neutral point of view always seem to trump opinion texts in a search engine ranking. This is certainly a feature and not a bug. However, it makes a large part of human discorse invisible to what one could argue is our primary means of finding information on a daily basis.

It’s interesting to consider the history of Google’s citation-based ranking algorithm in this context. The model content that the creators considered for the algorithm during early development was scientific papers, where citations effectively increased the prestige of the cited paper. However, the purpose of a citation in the scientific world is to pull in data, not the conclusion, from antother researcher. That data became what search engines valued, not the arguments or conclusions.

Philosophy and the Humanities don’t have a culture of citation. One does not cite an argument someone else has made, one simply restates and maybe gives credit. Unlike citing someone else’s data it is a fallacy to cite the authority of someone esle in a debate. For these reasons opinion texts are unable to build the web of citations that search engines depend on to assign relevance.

Debate is something quite different from peer reviewed science process and Search becomes less usefull as we stray away from that model. The hard data relevant to a debate is typicically unambiguous. The conclusions that one can draw from those data is where debate begins.

It is unlikely and probably undesirable for the current generation search engines to change their algorithms to value relevant opinion texts. However, if someone sets out to create something that does then I think they’ll find that the current ways of doing things don’t work in the context of opinion.

Let’s say you have a forum thread with a small group of participants who are having an intelligent debate on a hot button topic of the day. The parties disagree but are polite and cordial and do not engage in ad hominem attacks of any sort. This goes on for awhile and a short time later other forum participants begin to take notice.

Among these observers are some real partisans who are offended by the very notion of a debate on the subject. Not content to sit on the sidelines they feel they must admonish the participant on the wrong side of the argument. These new participants soon start bickering amongst themselves in all the ways we’ve come to expect… occasionally punctuated by some thoughtful posts by the original participants. Godwin’s law manifests its inevitable result and the thread dies.
The previous scenario is an all too common occurrence. Debate threads have a really hard time scaling past a few people. In an unmoderated forum there are no means to block out posts or participants that decrease the value of the thread. This makes threads virtually unreadable to lurkers, participants and even search engines.

A low Signal to Noise ratio is not a problem that is unique to debate forums. We know from other disciplines that solving signal to noise ratio problems requires a comprehensive description of what signal means in a given context. Human discourse represents the toughest medium for coming up with such a description but I don’t think the problem is hopeless, just unsolved.

I have a lot to say about Helium.com, a site where user-submitted articles are submitted and moderated by users. The site has two features that I consider essential for debate; immutable posts and user moderation. Most of the site is dedicated to uncontroversial topics (example). These topics are organized if a fairly straightforward “Best Article at the Top” format.

Things get more interesting with their debate topics (example). On these topics the page is split down the middle with articles arguing for the proposition on one side and articles against the proposition on the other. Each side is moderated independently. The result of this design is as one would expect. The separated user moderation helps with the signal to noise ratio without resulting in noticeable shouting down. The immutable postings prevent the sort of edit warfare that we see on Wikipedia.

Yet despite all the potential I see here I must say that I am not impressed with the quality of the arguments made on this site. Many of the top arguments make outrageous or at least unproven claims as premises to their arguments. I can’t help but think that the absence of a rebuttal feature makes it possible for such claims to stand. Also, the moderation system seems to favor arguments that simply summarize many arguments in sort of a kitchen sink school of debate. These mini-arguments almost never back up their premises.

I also can’t help but think that the financial incentives (writers of top articles get a share of the ad revenue) put into the site has a corrupting effect on the moderation system. This is simply a sinking feeling that I cannot prove. However, credibility is hard to come by on the Internet and I can’t imagine nobody else will get the same impression.

Lawrence Lessig’s first video on corruption, his new area of study, has a line has been echoing through my head ever since I heard it. Here he is talking about presenting his ideas about copyright to congressmen:

What struck me was not that I was meeting congressmen who had a different view from mine. It was I was meeting congressmen who never even imagined there was another side to the story. So I would appear and make an argument and they were just befuddled because…Well, maybe because I wasn’t clear enough but maybe because they actually hadn’t ever contemplated the kind of points that were being made because never had the people making those points had the opportunity to appear in their office to talk to them about it….

You can find the line at 42:20 but I recommend the whole video, if only for context.

Congressmen deal in arguments. It’s their job. The information required to do their job comes in the form of arguments. Us working folk can just run a search to get half the information we need to do our jobs. Congressmen, however, have to read op-eds, listen to lobbyists and *shutter* watch talk shows to get the information they need to do their jobs. I actually kind of feel sorry for them.

As one of those condescending yuppies who doesn’t have a TV I’m at a loss to do any thoughtful analysis on this clever user moderated site about the TV show Lost. This is a shame because it has some unique properties. I can’t make sense of any of the content (which I’m told is an experience not too different from watching the show) or make any judgment as to its quality. I’d welcome any comments on this matter.

Most interesting is the section under the heading “debunked” for theories that have been disproved in some way. The mechanism to flagging something “debunked” appears to be from receiving a certain amount of “debunked” flags from readers. I’d be interested to know if there were theories the have been marked “debunked” yet a group of users continued to argue for it.

Will pop culture forums will lead us into the promised land of auto-moderated debate? If the ancient Kirk vs. Picard debate is ever resolved then stem cell research and global warming certainly can’t be far behind. Probably best not to hold our breaths though.