The data dig: Researchers mine mother lode of tweets, posts for clues about what makes us tick

By Tom Mooney

Friday

May 3, 2013 at 3:02 PM

Today Rhode Islanders will join 200 million other people around the world in posting a tweet or two — or 20.The latest Red Sox score, perhaps; a snarky remark about a politician on morning television;...

Today Rhode Islanders will join 200 million other people around the world in posting a tweet or two — or 20.

The latest Red Sox score, perhaps; a snarky remark about a politician on morning television; the link to that article about French bus drivers threatening to strike because their uniform pants are too tight. (True story.)

By day’s end, the total number of pithy bursts of commentary will approach a half-billion. And no matter how ephemeral their authors might have thought their public tweets were when they launched them into cyberspace, every single one — each teenage grievance, each Syrian’s cry to topple the regime — will be archived at the Library of Congress for future research.

That might surprise some people but not social scientists who study human behavior and changing social norms. With the proliferation of social media in recent years supplanting, in some cases, other forms of communication, they’re already hard at work mining the vast deposits of digital interactions.

“Our motto now is ‘Not irrelevant anymore,’ ” quipped research sociologist Marc Smith, who lives and works in Silicon Valley in California. He’s been developing computer tools to help his colleagues analyze the massive stores of social-media information, known as “big data.”

[social-media-poll5]

Facebook and Twitter are considered mother lodes of information, with Twitter yielding big payoffs at the moment. Computer programs can scan text for key words on the public Twitter feed and generate interesting findings about what vast populations are thinking and feeling. Analysis of Facebook isn’t so simple. Proprietary ownership and privacy settings have hindered some academic review.

Still, in conjunction with search engines and blogs, today’s study of social media is panning out in ways unimaginable even a decade ago, scientists say.

Back then, a sociologist’s dream would have been to equip everyone in society with a diary and have them write down how they feel, says Alexander Halavais, a social science professor at Arizona State University and president of the Association of Internet Researchers. Today, with social media, that dream has come true: “In some ways, we now have an ongoing window to the mind at any time.”

It’s estimated that 2 billion of the world’s 7 billion inhabitants are now connected to the Internet in some way. They e-mail. They tweet. They “like” things on Facebook. They “link,” they “follow,” they “connect,” they “share,” they “instant message.”

“When you do those things, sociologists get very interested,” says Smith in a phone interview. “We look at that and say, ‘Oh, you created a connection between you and somebody else.’ ”

[social-media-poll2]

In 2010 the Library of Congress announced that Twitter had agreed to donate its entire archive of public web messages, dating back to the company’s inception in 2006. The library said the archive –– more than 170 billion tweets and growing –– held extraordinary potential for research into contemporary life.

Reaction to Twitter’s donation was met with both surprise and excitement, says library spokeswoman Gayle Osterberg.

“Many commentators saw the involvement of the library as a validation of social media as a legitimate means of communication; others questioned whether what is transmitted on Twitter warranted archiving. For example, why would the Library of Congress care what I had for lunch?”

But, Osterberg says, a century from now “what people ate for lunch at the turn of the 21st century might be among the information that helps [researchers] paint with more colors and a finer brush.”

The archive is so enormous, and expanding at such velocity, the library doesn’t have the computer capacity to make archive searches easy. A single search could take 24 hours. The library hasn’t set a timetable for when the archive could be open for academic research.

Meanwhile, the library has received more than 400 inquiries from researchers around the world wanting access. Topics of interest range from studying the rise of citizen journalism and the effectiveness of politicians’ communications, to tracking vaccination rates and predicting stock market activity.

Other researchers aren’t waiting. Anyone using Twitter can see the real-time stream, but it is not searchable beyond a few previous days. However, some companies, such as Gnip, are capturing the stream and, for a fee, offering access going back further in time.

Researchers, teaming up with mathematicians and computer programmers, have tapped in to those Twitter repositories –– along with other social-media resources –– to uncover some interesting findings.

This past winter, researchers at Johns Hopkins University used computer tools to track people’s Twitter comments about being sick to accurately predict where the next flu outbreak would strike. The “real time” results were about two weeks faster than a regular influenza-prediction map published by the federal Centers for Disease Control and Prevention.

But research based on social media has limitations.

For instance, because most Twitter users are young and computer savvy, it is not an accurate representation of the general public; most flu sufferers were elderly and young children and the least likely to be posting tweets about their health.

Good data take hard mathematical crunching and time to generate, says Keith N. Hampton, a communications professor at Rutgers University. “It’s actually a bit frustrating because it takes a long time to figure some of this stuff out, and the technology use is changing so fast that it’s hard for scientists to gauge what is going on.”

One area of interest for Hampton is how Internet use and cellular phones have changed people’s social circles. His findings, along with some national surveys, he says, debunk one big misconception: that heavy users of social media are disengaged from real life.

[social-media-poll1]

On the contrary, “what we know is people who tend to use the technology the most tend to have more diverse networks and larger networks as a whole, both online and off.” Consequently, says Hampton, they are more trusting of others, more politically engaged, volunteer more and have more diverse personal connections. “This translates into benefits when you have a crisis in your life.”

Hampton also studies people’s interactions in public spaces — comparing, for instance, photographs of people walking through a city park from 20 or 40 years ago with photographs of today. While contemporary photographs would probably show many people staring at their smartphones and ignoring those around them, that doesn’t mean they are totally tuned out of their surrounding environment.

His research has shown that people who use social media in public places, say a coffee shop or a park, are actually more inclined to acknowledge others and events around them than someone talking on a cell phone.

About 20 percent of adult Americans use no digital communication at all, Hampton says. Research has shown those people tend to be less trusting of others and participate less in their community.

“I have a very hard time making an argument that there are many benefits to being completely disconnected,” says Hampton. “Critics say a text message cannot replace a big, warm hug. But what if you get a lot of text messages” at a moment of despair, or birthday greetings from all your Facebook friends? “Does that have some value?”

[social-media-poll3]

While research hasn’t answered that question definitively, Hampton says “people who share a lot of photos online tend to think their networks are more diverse.” And that makes them feel good.

Speaking of feeling good …

In 2011 sociologists at Cornell University conducted the first cross-cultural study of people’s daily mood swings using tweets of more than 2 million people in 84 countries.

They analyzed words with both positive and negative connotations. (“Awesome,” for instance, and “annoy.”) What they found was that people around the world are more chipper at breakfast and bedtime and share a drop in mood in the late afternoon. The findings suggested that mood swings were driven more by biological rhythms than environment or culture.

Topics of research appear as infinite as social-media use itself.

There are studies about whether posting status updates on Facebook increases or decreases loneliness (it decreases loneliness, according to a German study); and studies showing that most Facebook users receive more information than they post (the result of communal “super users” who have huge “friends” lists and feel compelled to share everything about their lives).

There is research comparing conversation patterns in advantaged and disadvantaged communities, and studies that ask whether incessant texting by teens makes them more “shallow” thinkers (OMG! It’s true, says a University of Winnipeg study).

But there are also studies that debunk that universal parental fear that kids are all talking online to child molesters.

[social-media-poll6]

Danah Boyd, a Brown University graduate who studies youth culture and social media, is a senior researcher at Microsoft Research, an arm of the computer giant that studies future online experiences. She says in her writings that kids aren’t addicted to social media; if anything, they are addicted to their friends. The vast majority of young people are not online meeting strangers, she says. They are communicating within their existing social circles and use social media to clear communication hurdles such as not being able to drive.

[social-media-poll7]

There is even research out there on the research.

One 2012 study by researchers at Washington University in St. Louis identified 412 relative articles on Facebook alone. The study sorted the works into five categories: descriptive analysis of users, motivations for using Facebook, “identity presentation,” the role of Facebook in social interactions, and privacy and information disclosure.

Nancy Baym, another Microsoft researcher, has been studying the Internet since 1991. More recently she has focused on the role social media plays in the relationship between musicians and their audiences.

Musicians realize the commercial value of tweeting, she says, but they are often surprised by the “really emotional, interpersonal” chords they hit with listeners who communicate back about what their music means to them.

That finding highlights a big misconception about social media, says Baym: “That there is some distinction between online and real life. There are reams of empirical research that shows [social media] relationships are deeply interwoven with what happens when we are not looking at the screen.”

When an appreciative music lover e-mails or tweets the artist and says: “ ‘Since my father died, all I can do is listen to your music,’ That is as real as it gets,” says Baym.

While some critics of social media say it has less value than face-to-face conversations, Baym counters that sometimes face-to-face conversations are “really boring” and exchanged with people “we don’t really like, and who we wouldn’t exchange an e-mail or phone number with.”

Despite all the study, Baym says the general social implications of all this social media is a “deep nest of contradictions, and there are no easy answers.”

[social-media-poll4]

Marc Smith says what’s happening now is reminiscent of what happened more than a century ago with the telephone.

“The telephone created a moral panic in America,” he says. “Suddenly, people in low status could talk to people in high status. … Men were talking to women. Women were talking to men.”

And worse: people could hide who they really were — and deceive — the same way those scam e-mails arrive today offering to share lottery winnings if you just send along your bank account number.

Thus was born the phrase: “phonies.”

Smith says there are many examples of new technologies “changing society and doing so in ways nobody expected.”

The proliferation of the automobile increased teenage pregnancy rates, he says, and gave criminals a way to avoid police, which in turn created a need for an expanded tax base so society could buy patrol cars to chase the bad guys.

In this world of pervasive social media, one area of study remains unexplored: how will relationships in the future die?

Not since we left the feudal age of villages hundreds of years ago, says researcher Hampton, “have we had social ties that are persistently with us all the time.”

It used to be when we graduated from high school, then college and moved away, we lost chunks of our earlier social networks. There were benefits to that separation: the loss of peer pressure and conformity helped us become independent thinkers.

We took new jobs, started new friendships, became our own person with our own perspective and beliefs.

“Now we grow up with these people, and it’s very hard to shake them,” says Hampton, “and I think we have a lot to understand about how that constant access will affect us. … If I see [on Facebook, for instance] that all these people I went to high school with are considerably more conservative than I am, does that instill in me that my liberal views are more right, or do I think: ‘Wow, maybe I should readjust how I view the world?’ ”

“We don’t know yet.”

Check Facebook sometime in the future.

Never miss a story

Choose the plan that's right for you.
Digital access or digital and print delivery.