11/17/2009 @ 6:00AM

Tracking A Million Conversations

Social media allows anyone to say anything they want and have it broadcast across the globe. Most of the time our conversations simmer in obscurity and then drift off into the nothingness of digital archives. But some ideas boil over. Perhaps United Airlines broke your guitar or you had an idea about Chad Vader, Darth’s younger brother. When such thoughts propagate and capture the attention of millions of people, they change perceptions. From a business point of view, these opinions and ideas can become the equivalent of marketing campaigns that would cost millions of dollars.

Marketing and media relations professionals are now tasked with tracking millions of conversations, keeping an eye on which ideas are growing in influence, and then determining how to respond. But social media is not just about media anymore. The way the customer service is delivered, the way sales and hiring prospects are evaluated, and the techniques for product development are all being transformed.

For example, it is common for someone who complains about a product on Twitter to then be contacted by the company that makes the product. Will customer service in the future start with a tweet because consumers expect companies to be listening? Sales staff now routinely evaluate prospects on LinkedIn. But how can all the other relevant evidence in hundreds of social media sources be found efficiently?

The first wave of experimentation with social media in business took place by brute force, using search technology to find conversations of interest and then to get involved. The vast number of sources and the mind-blowing volume of activity means that some sort of automated monitoring system is required. Here, we take a look at the technology involved in tracking social media and the different monitoring strategies they enable.

The systems we are about to describe work in two ways: The most common is based on some sort of guidance. You tell the system something about the conversations that are important to you, then receive a summary of related conversations. Most systems also allow you to ask, “What are the most active topics in social media right now?” and then get some summary. In both cases, you can then drill down and explore the connections between categories of conversations.

Radian6 leads the pack as an aggregator of many different technology approaches, including various kinds of search and some text analytics that can be applied to a wide variety of business processes. Marcel LeBrun, CEO of Radian6, describes his company’s software as a listening and engagement platform through which you can track conversations, get social media profiles of who is talking–”caller ID for the Internet,” LeBrun says–and most importantly, measure how rapidly attention is growing. An extremely negative post that is not generating responses may be less worrisome than a mildly negative one that has 100 comments in the last hour. Radian6 comes with workflow and integration with the usual suspects for CRM, ERP and content management so that important conversations can be channeled through a process for responding.

InsideView uses an aggregation of open source and proprietary technology for search and natural language processing to track social media related to the sales process. Social media is added to other online sources for news and financial information so the software can automatically recognize important events. The results are then delivered through InsideView’s dashboard or through integrations into existing CRM and ERP software. The software enables a sales person who is evaluating which leads to pursue, to make better decisions by seeing more information on the background and context of the company. “Knowing who is tweeting about their computer crashing is like a goldmine of insight for a rep selling cloud-based storage,” says InsideView CEO Umberto Milletti.

The most established technology related to tracking social media comes from Autonomy’s Intelligent Data Operating Layer (IDOL) platform, which has been scanning the Internet using techniques based on patterns and probability for more than a decade. IDOL is able to read millions of documents, understand what they say and then sort them into clusters. Positive and negative sentiments can be identified, as well as clusters that are growing or shrinking. Thousands of companies use IDOL directly or when it is embedded in other products. For example, a company called VMS has recently extended its use of IDOL beyond its competitive analysis and news monitoring business into social media. Mike Lynch, Autonomy’s CEO, points out that the IDOL platform does not rely on a model of content that must be maintained. Instead, a map of the meanings of what is being talked about is created and automatically updated as new content arrives.

NetBase is a new general-purpose linguistic search platform based on the idea of reading sentences and parsing their deeper meanings. This is a different approach from the statistical approach that Autonomy takes, which can be pointed at any text, regardless of language. NetBase, in a way, must learn to speak each language. Jonathan Spier, CEO of NetBase puts it this way, “Statistical approaches can get you to a document but not to a sentence.”

These are early days for technology that tracks and makes use of social media. Each of these technologies and approaches do something right. The big-picture value comes from putting a technique to use in the context of a business. The question is: How can a deeper understanding of social media trends and content help us do our jobs better? Patterns are emerging and processes are changing, but that is a topic for another day.

Dan Woods is chief technology officer and editor of Evolved Technologist, a research firm focused on the needs of CTOs and chief information officers. He consults for many of the other companies he writes about. For more information, go to www.evolvedtechnologist.com.