As I discussed in my post last month, it’s a skewed Web out there. A multitude of online social filters were developed over the last 15 years to address our perennial information overload curse. From Google’s page rank, we went all the way to tag clouds, social bookmarking, Twitter trending topics and Gmail’s Priority Inbox, trying to find ways to make what matters float to the top. However, most of these social filters are based on some variation of a “majority rules” algorithm. While they all contributed to keep information input manageable, they also skewed the stream of information getting to us to something more uniform. Will crowdsourcing make us all well-informed drones? Ultimately, it may depend on where you’re looking at, the center or the fringe of the beehive.

Almost two years ago, Clay Shirky boldly stated that information overload was not a problem, or at least not a new one. It was just a fact of life at least as old as the Alexandria library. According to Shirky, the actual issue we faced in this Internet age would be that of a filter failure: our mechanisms to separate the wheat from the chaff were just not good enough. Here is an excerpt from his interview at CJR:

The reason we think that there’s not an information overload problem in a Barnes and Noble or a library is that we’re actually used to the cataloging system. On the Web, we’re just not used to the filters yet, and so it seems like “Oh, there’s so much more information.” But, in fact, from the 1500s on, that’s been the normal case. So, the real question is, how do we design filters that let us find our way through this particular abundance of information? And, you know, my answer to that question has been: the only group that can catalog everything is everybody. One of the reasons you see this enormous move towards social filters, as with Digg, as with del.icio.us, as with Google Reader, in a way, is simply that the scale of the problem has exceeded what professional catalogers can do.

While some still beg to differ about information overload not being an issue – after all, our email inboxes, RSS readers and Facebook and Twitter streams never cease to overwhelm us–we tend to welcome every step in the evolution of smarter filters.

The whole lineage of social filters, from Google’s page rank, passing through Digg and Delicious, culminating with Twitter’s trending topics, mitigated one problem–information overload–but exacerbated another one: we were all getting individually smarter, but collectively dumber. By letting the majority or the loud mouths dictate what was relevant, we ended up with a giant global echo chamber.

br />This of course is not a new problem. Back in the early 1980s, MTV was running Michael Jackson’s 14-minute “Thriller” video twice an hour. The trouble here is just the magnitude of it. A potential downside of this mass-media-on-steroids uniformity is that a homogeneous environment is not the best place for innovation to flourish. Borrowing from paleontologist Stephen Jay Gould: transformation is rare in the large, stable central populations. Evolution is more likely to happen in the tiny populations living in the geographic corners: “major genetic reorganizations almost always take place in the small, peripherally isolated populations that form new species.”

If you are looking for the next big thing, or trying to “think different,” or to be creative and innovative, you need to look beyond the center. The core will tell you what’s up, so that you’ll be “in the know.” The fringe will show you what’s coming next. To paraphrase William Gibson, the future is peripherally distributed.