A Twitter Experiment

I’m a long-time Twitter skeptic. It’s difficult for an economist to see a 140 char lmt as a ftr. My journalist friends tell me I’m dead wrong. And a recent long and boozy evening with co-founders Evan Williams and Jason Goldman convinced me to give it a try. Is Twitter worth the hype? Let’s find out.

Today I’m beginning my Twitter Experiment. I’m now tweeting @justinwolfers. I’m going to keep this up for a couple of weeks as a “burn in” period—basically so that I can learn the ecosystem before my experiment begins. Then on the morning of August 1, I’m going to wake up, and flip a coin. Heads, I’ll open Twitter; tails I won’t. And I’ll do the same on August 2, and then every day for three months. If the coin comes up heads, it doesn’t necessarily mean that I’ll tweet, just that it will be a Twitter-aware day; I’ll consume the stream, and tweet away if I feel the need. Tails, and I’ll simply tweet “Tails, goodbye,” close the stream (unless I need it for research) and then resist the urge to tweet for the rest of the day.

So far, designing the experiment is easy. But what are the right outcome variables to track? I can think of three things that Twitter does:
1. Distracts me, sucking valuable time from serious research.

2. Broadcasts me, perhaps increasing my influence.

3. Informs me, which may make me a better economist.

Here’s my question for you: How can I measure the influence of my Twitter Experiment on each of these outcomes? I think measuring distraction is easiest—I’ll just evaluate how productive I was each day, using a subjective 0-10 scale. I have some ideas about measuring influence, but none are particularly compelling. And I’m baffled by how I might measure the extent to which Twitter is informing me, making me a better economist.

Crowdsourcing can surely improve my experiment. Do you have any ideas for how better to measure distraction? Or ideas for useful indicators of influence or impact that I can track? And how best to measure how informed I am? And does anyone else want to join me in this experiment? More data = more insight.

Please add your ideas in the comments. Or if your idea is ez 2 expln, just tweet me. #twitexpt.

COMMENTS: 34

Regarding one way to measure point number 2 (whether it broadcasts you), ask the person who runs the Freaknomics blog to compare traffic to your blog posts from Twitter compared to your pre-Twitter numbers or, better yet, the traffic on non-Twitter days.

If you tweet someone, say someone of influence, or someone without using your economical knowledge, you could perhaps make a difference in their decision making process.

Twitter is all about expanding the pipeline from fat cats to the users (people, firms, etc). So by becoming a user, you theoretically have the potential to alter the behavior of the fat cats, as you now have a voice.

Under the current scope of your experiment, you’re not going to do very well at all, because you’re looking at it from the wrong lens, “What do I gain?” The internet, and Twitter for that matter, are all about contributing to a larger collective.

You need to look at it more so in what you have to offer, rather than what you personally can gain- how scientific is that?

How about adding to the experimental design by having someone you work with (another economist or a research assistant) assess your daily productivity instead of you doing it yourself? You would need to keep a daily list of things accomplished, which you would report to this colleague at the end of every day without telling him/her whether it was a Twitter or non-Twitter day. Then he/she would grade your productivity on your 1 to 10 scale. This would add an element of blindness to the experiment.

First of all, welcome and good luck
I have a comment about your #1 variable – productivity
It is inaccurate to measure it so simply
I think you will find being “twitter aware” keeps you on edge writing wise and more tuned to everything else.
In the mean time I offer myself to participate as a content provider that may or may not affect you
@theregjoe

I think you have too many subjective variables for such a small sample of people. To measure how twitter distracts or informs you can be influenced by:

* Personal feeling towards Twitter as a social media on that given day
* Who you follow
* Who follows you
* How your interests correlate with those of people you follow/who follow you
* Mood and productivity ignoring the possible influence of Twitter (would I just have found something else to distract me today?)

Points 1 & 5 are easily reconcilable by adding them as data points but 2, 3 & 4 can only be accounted for by widening the sample, in my opinion. Perhaps you could measure distraction by comparing time spent on Twitter to how productive you felt off of Twitter (you have already conceded you’ll look at Twitter even when you get a tails – why bother flipping the coin at all?). Similarly you could perhaps tally how many ideas Twitter gave you in a given day. Yet neither of these can take into account who you follow and who follows you.

Similarly measuring how well you are broadcast creates issues. How will you measure this? The most common way is to see how often you are retweeted but, as mentioned before, that depends on your followers being the right people. If you are tweeting about economics and yet have very few economists following you, you may either have a massive amount of retweets or none at all.

You have already tried to create a control group with the tossing of a coin. Perhaps what you really need is to use your twitter account to encourage more people to join in the experiment. That way the number of participants would help you judge how well you are broadcasting (keeping people interested in the experiment). Also if you created a questionairre asking people about Twitter habits, followers, the people they follow etc., it might help you discover *HOW* to make Twitter less distracting and more informative by comparing this to their results.

You should be able to measure the volume of traffic (visits) to your blogs/articles coming from twitter and compare to your pre-twitter time. Google anaytics is great for this. Should be a good guage of increasing influence