The Mechanisms of Online Emergence

When I talk about Enterprise 2.0 with company management teams, industry groups, and executive education students I usually start by asking people to raise their hands if it’s easier for them to find what they want on their company’s Intranet than it is on the public Internet.

No one has ever raised their hand.

I then ask them to think about how strange this is. Their Intranets are infinitesimally small compared to the Internet, and they’re also usually built and maintained over time by well-paid professionals. At many larger companies, in fact, the Intranet is the responsibility of a dedicated group whose job is to organize the site and make things easy to find.

I next ask how people find what they’re looking for on the Internet. Everyone responds "Google." (most people are polite enough to refrain from adding "duh.")

At this point I use all the classic case-method teacher techniques (standing still, cocking my head to one side, looking puzzled, scratching my scalp, etc.) to tip people off that the big question is coming. "But hold on — doesn’t your Intranet also have search capability? Isn’t there a little ‘search’ box up in the corner of each page? Why doesn’t it work as well? What’s so great about Google?"

This is a great question for getting the nerds in any group to self-identify. They explain to their colleagues that Google returns www.hbs.edu as the first result when searching for ‘harvard business school’ not because the HBS home page features those three words more prominently or frequently than any other page (think how easy it would be to game that system), but because it’s the page apparently about ‘harvard business school’ that’s most frequently linked to, especially by other sites that are themselves frequently linked to.

It relies heavily on people. Not people at Google — PageRank is automatic — but page builders all over the Internet who insert links to other pages. Over time, lots of these people have linked to www.hbs.edu. In other words, www.hbs.edu has a lot of backlinks. This high volume of backlinks is what places www.hbs.edu at the top of Google’s results for ‘harvard business school.’

Google’s founders realized that even though the Internet is extremely decentralized the Web still has a huge amount of structure thanks to links. This structure can be exploited not just for navigation (i.e. hopping from page to page via links) but also for search. Links provide so much structure, in fact, that the Web appears to us to be a very orderly place; we can find what we want on it.

The point of the Q&A described at the top of this post is to highlight how remarkable this is. As the Web was gaining steam about ten years ago its advocates (myself included) predicted that it would bring many kinds of benefits. Easy-as-falling-off-a-log searchability, however, wasn’t one of them. In fact, I recall a lot of handwringing about how we were ever going to be able to stay on top of it as it grew and changed.

What we didn’t realize is that thanks to links the Web is very much like an ant colony.

Ant colonies are also highly decentralized, but they appear tightly orchestrated. Colonies have complex social structures and use sophisticated strategies to forage, defend themselves, and make war. This happens because each ant is ‘programmed’ by its DNA to do certain things (carry an egg, fight an intruder, go to where food is) in response to local signals (usually chemical scents from other ants, eggs, intruders, food, etc.). As ants interact with each other and their environment they send and receive signals, and these low-level activities yield high-level structure.

Complexity science uses the term emergent to describe systems like this. Emergence is the appearance of global structure as the result of local interactions. It doesn’t happen in most systems; what’s necessary is a set of mechanisms to do critical things like connect the system’s elements and provide feedback among them.

The Web’s emergent nature doesn’t stem from the fact that it’s a huge collection of digital documents; if the Library of Congress were digitized tomorrow and put on line, it would not constitute an emergent system. The Web is emergent because it’s the dynamic creation of countless people around the world interacting with each other via links as they create new content.

This is a key difference between the public Internet and private Intranets. Public Web sites are built by millions of people, while most Intranets are built and maintained by a small group. Emergence requires large numbers of actors and interactions, but Intranets are produced by only a few people (even though they are passively consumed by many.). In addition, most Intranet pages aren’t as heavily interlinked as pages on the Internet.

Another important difference is that Web 2.0 has accelerated the rate of emergence on the public Internet. I think of Web 2.0 tools and technologies as accomplishing two important goals: increasing the number of people who are contributing content (and the ease with which they can do it), and increasing the number of ways to let content creators (and consumers) interact with each other. These new interactions are the further mechanisms, beyond linking, for emergence — for letting patterns and structure emerge from low-level behavior.

Tagging, as implemented on del.icio.us, Flickr, YouTube, Yahoo’s My Web, etc., is a new and clearly powerful way to let structure emerge. Tagging, like linking, fulfills all the standard criteria for emergence:

It’s conducted by many agents spread all over the Internet

These agents are acting independently and with great autonomy. I don’t pick my tags from any pre-defined list; I make up whatever ones are useful to me.

Agents are also acting in their own self-interest. My del.icio.us tags help me navigate my own bookmarks. The fact that they help reveal the Web’s structure to everyone else is peripheral to me, but central to the value of del.icio.us for everyone else.

The high-level structure of the del.icio.us folksonomy (which changes over time, and is visible in its wonderful and intuitive cloud views) can’t be predicted by observing low-level activities. My tags, in other words, won’t tell you anything about what’s going on across del.icio.us as a whole, just as watching a single ant won’t tell you what the entire colony is up to. Complexity science uses the term ‘irreducible’ to describe this sharp disconnect between low-level behaviors and high-level structure.

I also think automatic signaling mechanisms like RSS (that alert users when new content has been added to pages of interest) help foster emergence. Signals are another type of interaction between people and content, but with an interesting inversion. With links and tags, the person initiates the interaction; with RSS, the content does. Signals also meet all four of the criteria listed above.

In addition to tags and signals, online tools like Digg and the recently announced Google Co-op (briefly explained here) also accelerate emergence. Digg lets users vote on the merit of news stories, and Google Co-op provides a means to let people tell Google what different sites are about — what their content is. Google then uses that information to help guide searchers more precisely.

However, tools like these don’t meet criterion #3 above. They’re not about people acting in their own self interest, they’re about people doing things for the good of the community. That’s not a bad thing at all; look at how powerful and helpful user ratings and reviews are at eCommerce sites like Amazon, and how willing people are to build up eBay’s community by rating buyers and sellers. Reliance on altruism, though, does lead me to suspect that contributions to Digg and Google Co-op might grow more slowly than tags, links, and RSS subscriptions.

But things might well be different within the enterprise. As I’ve argued previously in this blog, employees in healthy companies are much more interdependent than strangers scattered across the Internet, and should also have a greater shared sense of mission. In addition, business leaders have the two powerful behavior-shaping tools of incentives and culture at their disposal. The cultures of Wikipedia, Digg, and other online communities grew up around the technology; within companies, culture already exists.

This implies that companies’ Intranets might be able to take faster and deeper advantage of all the mechanisms of online emergence than the broad public Internet. It’s not hard to imagine that teams of employees would start linking, tagging, subscribing to feeds, providing ratings, describing the content of both internal and external pages, etc. Perhaps the Intranet could even become a point of pride among employees, or an important part of the identity of the company. It’s also easy to imagine that the function of the Intranet group might shift from generating largely static and sparsely interlinked content to assisting the organization and navigation of employee-generated content.

A bit of tech work is required to enable this vision, but nothing too strenuous. What’s going to be much more difficult is explaining these tools to employees who are very accustomed to working with Web 1.0 Intranets. Everything I’ve seen indicates that the ‘activation energy’ required to get the current workforce comfortable with Web 2.0 tools, and so to create Enterprise 2.0, is pretty high. My executive education students usually have a deer-in-the-headlights look when we start talking about the new tools.

However, I’ve also heard and seen firsthand that once people do finally comprehend and get comfortable with the new generation of tools it becomes very difficult to stop them. These really are ratchet technologies, but they’re not intuitive to those of us who have spent more than a decade working with Web 1.0. Give us a bit of time and a bit of help to get used to them, though, and watch what emerges.

As I think about online emergence a couple questions come up. Please leave us a comment if you have any insight on any of the following:

Are there other mechanisms of online emergence in addition to the ones discussed here?

What companies/organizations are doing the most advanced work to bring Web 2.0 inside the firewall? I’ve heard about Motorola, the BBC, Microsoft, and IBM, and I’ve written about internal blogs and wikis at DrKW. But these can’t be the only examples. Who else is doing interesting and instructive work in this area?

What are the best ways to get a Web 1.0 workforce comfortable using Web 2.0 tools?