Who's Talking About Me?

Web experts are designing technologies that can collect scattered bits of online conversation.

Popular videos and articles get reposted or discussed on dozens or hundreds of sites. But Web experts are now thinking about how to keep track of online conversation in real time, even when it’s scattered all over the Web. A new crop of protocols aim to do just that.

The protocols provide notifications when new content is available, rather than passively waiting for search crawlers or feed readers to discover the content. For example, pubsubhubbub, an extension to the protocols used today to syndicate content, can push content out to feed readers as it’s updated. Building on that Salmon, a protocol first proposed last month, could allow comments to “swim upstream” to connect to the original post, regardless of where they’re made.

Technologies like this are needed for today’s Web, where content tends to flow from one site to another, said Kevin Marks in a keynote given last week at Defrag 2009, a technology conference in Denver focused on tools and technologies for handling online data. Marks cofounded Microformats, a set of standards for social information on the Web.

Getting scattered bits of data gathered quickly could be critical for people who want to participate in the conversation that goes on around the content they post online. About 64 percent of the total engagement with a piece of content happens in the first hour, according to Ilya Grigorik, chief technology officer of PostRank, a company based in Waterloo, Ontario, that helps clients monitor social activity around online content.

What’s more, people are increasingly likely to discuss and interact with content away from the site where it was originally posted. Grigorik studied 1,000 of the feeds his company has monitored for the past three years and found that about 60 percent of the interactions PostRank recorded happened on sites other than where the content was originally posted. “As a publisher or blogger, I want to see these conversations,” he says.

For the Salmon protocol to work properly, it would need to be adopted by both publishers of content and services that might subscribe to or discuss that content. When a post appears, the publisher uses pubsubhubbub to notify the subscribers that it’s present. Then, if a user makes a comment about the post on another site, the Salmon protocol sends this information back to the publisher. The publisher can in turn pass this comment downstream to all the other subscribers, keeping the conversation unified wherever it occurs.

John Panzer, the Google engineer who first proposed Salmon, notes that the protocol also includes ways of checking the identity of people making comments. While it’s possible to use a pseudonym to post, he says, it’s not possible to forge someone else’s signature. This feature would help prevent spammers from using the protocol as yet another means of distribution.

Earlier open standards helped make it easier for users to participate on sites across the Web by making it simple to transfer things like profile information and lists of friends, says Joseph Smarr, chief technology officer at Plaxo, a company in Mountain View, CA, that synchronizes contact information between Microsoft’s Outlook, other desktop e-mail programs, and a number of Web services. The new generation of protocols are solving the problems created by that success, he says, adding, “Now the question is, how do you end up not having the conversation be totally fragmented?”

Salmon does raise privacy questions, Smarr says. For example, it’s unclear whether users will be comfortable seeing their comments distributed to a wide variety of sites.

Marks says he’s seen a lot of concern recently as more sites have moved toward sharing information through business deals rather than through open standards. He says he remains optimistic about the new set of protocols, but adds, “Writing standards is easy. Getting people to adopt them is hard.”

Engineers have built a demonstration of Salmon, but the protocol is too new for any site to have formally adopted it so far.