Can Klout Really Measure Influence?

When I first met Joe Fernandez (@JoeFernandez), CEO and Cofounder of Klout, I thought little more of Klout than a new Twitter score. And to be honest, that is what they were for some time. A few minutes with Joe and you gain a lot of confidence in what they are trying to accomplish.

To reach their goal of becoming “the standard for influence”, they had to start somewhere and Twitter, the realtime back channel of the internet, was an obvious choice. Now they have added Facebook and have plans to add LinkedIn, Foursquare, and maybe Myspace very soon. Additionally, several other online metrics and social networks are on their radar to improve the scope of Klout.

They are trying to build an algorithm (or more accurately a set of algorithms) that will measure ability of someone to drive people to action online. In an attempt to provide insight, let’s break down how Klout produces their main influence score.

Currently Klout measures 30+ variables on Facebook and Twitter to give you a 1 to 100 score “with higher scores representing a wider and stronger sphere of influence”. Having only a Twitter account or Facebook account will not negatively affect your score, but will shift the weight of your score to that network’s variables only.

What are these variables? Well they fall into 3 separate categories.

1. Making People Act

They are pretty much anything that can be determined as an “action” in to response to something you have done. Klout calles this “Amplification Probability” and describes it as the likelihood that your content will be acted upon.

Unique Retweeters

Unique Messages Retweeted

Likes Per Post

Comments Per Post

Follower Retweet %

Unique @ Senders

Follower Mention %

Inbound Messages Per Outbound Message

Update Count

2. Network Size

Next, Klout looks at the size of your network. Think of this as the potential size of the audience that may be exposed to your audience. Consider several circles here. First are your direct followers, friends, people following your lists or lists your on, etc.

Then consider retweets and replies and comments as they may expand your potential audience greatly. Then consider the meta data of this potential audience. How many are you following? And how many are unique interactions, compared to one person interacting multiple times.

Klout refers to this as “True Reach” and describes it as the size of your engaged audience. The official list from their website of variables that control this score are:

Followers

Mutual Follows

Friends

Total Retweets

Unique Commenters

Unique Likers

Follower/Follow Ratio

Followed Back %

@ Mention Count

List Count

List Followers Count

3. Network Quality

Lastly, Klout looks deeper than just the size of your potential audience and more closely at who makes up that audience. Basically answering the question, how influential are the people you are engaging? They call this “Network Influence” and describe it simply as the influence of your engaged audience. Variables measured here include:

List inclusions

Follower/Follow Ratio

Followed Back %

Unique Senders

Unique Retweeters

Unique Commenters

Unique Likers

Influence of Followers

Influence of Retweeters and Mentioners

Influence of Friends

Influence of Likers and Commenters

The Secret Sauce

With all these variables to consider, you might start to get a sense of how complicated a task it is to try and measure online influence. Above all the variables and data points is another secret sauce. Klout then has to attach a weight to each variable. This is the part that is not public. How much a retweet affects a score verses a Facebook comment and so on.

If you are a doubter, this is likely where a lot of your distrust might creep in. Who is Klout to determine how much a retweet or a new Facebook friend should count toward your influence score?

“One thing we really want to avoid is trying to tell people how to use Twitter or Facebook” said Fernandez.

When you talk to Joe about the algorithm, it is clear that he is all about the data. About taking clear measurable results and weighing them against other results.

Testing For Accuracy

Another item they consider a sample of is click through rates.

We perform significant testing to ensure that the average click-through rate on links shared is highly correlated with a person’s Klout Score. (http://klout.com/kscore)

To do this they access publicly available click through data via Bit.ly. In fact they do a lot of this “testing” at Klout. At one point, of 22 employees at Klout (likely more now), 18 of their staff were engineers. That is a huge emphasis on the science behind their algorithm and how seriously they take the responsibility measuring influence.

When you talk to Fernandez he will emphasize that the engineers working on the algorithm hunt for outliers a lot. They try to find people that have abnormally high or low scores and see if their algorithm has a loophole that are missing these accounts and correct it accordingly.

How Klout Changes

They update the algorithm monthly right now, and have to account for a constantly changing online landscape. For instance, when Twitter implemented their oficial retweet button, retweets became a lot more common and therefore had to be devalued. Just as Google is constantly tweaking and improving their massive search result algorithm, Klout has to constantly adjust for changing technology and human behavior.

It is confirmed that Klout will add Foursquare and LinkedIn to the algorithm very soon. And last I spoke to them they were still considering Myspace. They have also looked into measure more click through data, website traffic, and commenting activity through services like Disqus and Intense Debate among others.

Even measuring things like book sales on Amazon for authors or Google Search trends to see who is being talked about are possibilities for Klout.

It is clear that the Klout team understands the vastness of the challenge of measuring influence. And they understand that influence is not only online. They have a few ideas of how they can ultimately begin to take into account offline activity as well.

Context Of Influence

Klout scores are calculated on a rolling 30 day timeline. As of last month, the score is calculated and updated nightly. Klout also lists topics on every profile it scores as the categories that they think a person has the most influence in. Expect this to play a much larger roll in the future as Klout emphasizes niche influence.

Can Klout measure influence? Who knows. From everything I have seen, yes. But is it perfect? By no means. The biggest miss right now is that there are plenty of influencers out there with scores that are not so high. It all comes down to context and making human decisions based on the scores they are creating.

What Klout has done is put a massive amount of data together in a very smart way. And a lot of that data is highly desirable for marketing situations. I think anyone marketing products online should be paying attention to Klout and looking for an opportunity to use their data.

I’ve found that Klout has a few consistent problems. My score, along with others, seems to “freeze” for periods of time and then suddenly jump around; the scoring process needs some smoothing over. Also, their most recent update seemed to boost many peoples’ scores dramatically, giving fairly high scores to fairly inactive users. If Klout can continue to adapt and improve, then it will become a really useful tool. For the time being, the small issues it has still bother me. (Another note, the topics that Klout claims I tweet about frequently include ones that I haven’t mentioned in months.)

My unofficial “Paws on the Ground” research says it takes about 3,000 online friends to get 1 to do anything offline, ie, go to a store and buy something, attend a rally, get to a meeting, fly to Darfur, etc. A really good “influence” score would be how effective is someone in getting offline action done. Anyone can click a “Like” or a “Join” button or push along a tweet. It take real influence to pry someone else’s butt off a sofa.

I hate to sound like a broken record here (my friends on twitter know exactly why I’m calling this out), but I agree with Rufus Dogg. My biggest issue with Klout is that certain elements of my Klout profile never change (while others ceemingly change regularly.

My biggest pet peeve is with the “influenced by and unfluencer of” sections that have literally not changed since my very first profile calculation. I’m not sure what’s worse, the fact these have not changed or the fact I’ve gottent no response to my inquiries about the problem for weeks (despite using their internal help system ticket log, and despite multiple calls for help on twitter itself).

Their focus recently has been adding Facebook to the mix (and soon Foursquare/LinkedIn) and getting the overall score to update daily. that in itself is a massive scale issue. The amount of data they take in daily to accomplish this is impressive.

but I agree, a large portion of their profile data, outside of this main score, is static. The best info I hear on this is that they are rolling out a redesign that will have these sections updating more. It is of course their goal to make all the data more relevant and as realtime as they can get. I think they are just fighting a man hour vs priorities battle right now.

They have just made major updates to the main Klout score so most of these flux issues should be fixed. The algorithm now updates daily on a 30 day rolling calendar. Even not tweeting for a few days will not drastically affect your number right now.

Other score changes might happen monthly when they update the algorithm, same as how Google updates their search algorithm weekly and results change accordingly.

Can you point to a couple examples of these inactive accounts that score so high? I am sure Klout would want to know so they can adjust accordingly.

My influencers and influencer of sections have not updated since the very first time I accessed klout several years ago. As a result, some of the people listed there are people I do not talk to much anymore, people who have left twitter, or people who have changed their twitter names. It is frustrating to see a service with such potential having significant issues in a few areas and makes me question the accuracy of some of the less easily verifiable sections.

I think that if there are areas they cannot provide any level of accuracy on, they should simply remove them from the public view. That would give it more credibility, IMO.

In addition to the inaccuracies already mentioned, I also wonder whether some of the metrics top out at a certain point or get rounded off. Either way, having more information on that would help. For example, it says I am a member of 500 lists. I am actually a member of 953 lists. Does that number bump up to 1K once I reach that and stay at 500 until then? Because all of my other achievements are really round numbers too, like 1K, 100K, 10, etc.

Those are their badges. Perhaps they need more clarity there. It is saying more or less that you are a member of the 500+ club. And yes, will likely jump to the 1000+ list club when you get to that number.

I don’t see Klout as good measure of influence at the moment. My Klout score is really high, but I don’t feel more influential because of it. Klout is headed in good direction, but for the time being, they aren’t good reference for “influence”.

Anonymous

The concept of Klout is great, but if you only judge people by an algorithmic score, no young entrepreneur, small business, etc stands a chance. Yes, Klout is cool if you have tons of the things they deem cool, but it doesn’t take into account being authentic, being hard working and having potential. I don’t think any website will ever be able to do that…

I question the validity of including Foursquare into the mix too. I consider myself up on social media but simply don’t like using Foursquare (I prefer Gowalla). So now my Klout rating gets dinged because I don’t like a service they measure on? I think they should give you the option to include the social platforms you want included that get attached to your name. Some people don’t use Facebook for anything but personal connections, why not give the option of LinkedIn then? I don’t use LinkedIn either, just bringing it up as a validator of connections/influence/etc. Speaking of Facebook, I wonder how Klout interprets multiple Facebook accounts? Some of us have to have fan pages because we’ve outgrown the Facebook 5,000 limit on friends.

I’m not 100% against Klout, I just think more people need to worry less what websites rate and think of them and focus more on putting out good work. Everyone is so caught up in having MORE connections instead of having the RIGHT connections.

I know plenty of young entrepreneurs with small followings and high Klout scores.

But I would agree overall Klout will never be an end game. It is another metric to help with human decision making.

Foursquare is an addition, not a requirement. If you are not on Foursquare or very inactive it does not reduce your score, it would only adjust the algorithm to account for where you are the most active.

Last I spoke to Klout about multiple Facebook accounts and pages (or even multiple Twitter accounts for business/personal, etc) they were still working through that problem. They will integrate Facebook pages for sure. But I get the sense they will begin to delineate the personal from the professional, or easily allow users to. Their algorithm already discovers if an account is a bot, business, publication, normal person, etc. but doesn’t publish it yet. Should be interesting to see where that all winds up.

In regard to having MORE connections vs the RIGHT connections. Klout definitely steers clear of weighting just the size of your followers. It is more about the quality and ratio if anything. I see plenty of people with small audience getting high scores. And you are right, it is because they might have a small social circle, but the people in it have a lot of influence.

The best description I have on how they will do this, and I know this is currently how the Facebook inclusion works, is that if you avoid adding Foursquare or LinkedIn accounts OR even if you do and are not very active on those accounts, Klout does not penalize you for a lack of influence there. Instead they adjust the algorithm to weight your score where you are active. This seems like a fair solution to me.

I agree with all the pros and cons for Klout. From the influenced to influencers, facebook, foursquare etc. Until I see something close to what I have been able to acquire through Klout (for free) I will be patient and continue to give them my feedback until they get it right. I will admit it has been updating my score without my having to go in and update which is nice but it has been going up and down daily so there seems to be some changes & updates happening as we speak. Does it make a difference in my bottom dollar? No, & I don’t expect any “free tool” to give me anything other than data I can review and fine tune

I agree with all the pros and cons for Klout. From the influenced to influencers, facebook, foursquare etc. Until I see something close to what I have been able to acquire through Klout (for free) I will be patient and continue to give them my feedback until they get it right. I will admit it has been updating my score without my having to go in and update which is nice but it has been going up and down daily so there seems to be some changes & updates happening as we speak. Does it make a difference in my bottom dollar? No, & I don’t expect any “free tool” to give me anything other than data I can review and fine tune

Anonymous

You eloquently sum up the value of klout in your last paragraph when you ask: “Can Klout measure influence? Who knows.” Frigging genius. Thanks for all the other worthless information, though…

I literally just finished reading Jim Mitchem’s article on Klout when this popped up in my twitter stream. I do not like playing the numbers game. We are being defined by a number and that number is based from an algorithm that is based upon popularity. If I get a big boots from my buddy Chris Brogan with an article that he retweets, I get a lot of retweets from his followers. Awesomeness! But do they ever take action, comment, or become clients?

Gauging our influence on line is not easy. People want a metric to say that their actions and time online is measurable in terms of how great they are. Brands want/need to tie into that greatness (aka influence) and maximize their efforts through you. Brands need us to be influential to promote them and we need us to be influential to justify the time spent online.

As I begin to start looking at Klout differently as it is not a passing phase that is going away like the flu, I still think that gauging our influence as a number and being defined by that is not a good measure. Do I have a measure that would be better? No. There is no way to ultimately measure the success of our activity online as we can say that sales were up this month when we tweeted more but can we attach the increase in sales solely to tweeting more. We know better than that. People wanted a number to connect with influence and give reason to continue with social media and Klout has given us that.

I am not sure I am on board yet but starting to warm up. Great article Jason. Really enjoyed it.

Thanks Suzanne. I actually have another post brewing about why people have such a hard time warming up to Klout. And I get it. For many influence easily equates to worth and that is emotional to attach a number to.

I think the most important thing to consider here is that Klout is a tool like any other. What is most important is how you use it. If you depend upon Klout 100% to make certain decisions, you remove the human relationship and the human decision making. For most marketing efforts, I would not advise that. But Klout CAN provide a lot of data to inform better decisions.

Klout is smoke and mirrors for investors. Make it look like its active with high profile users, that it has the full breadth and representation of twitter users, but once you pull back the fancy curtains of celebrity profiles, nothing is going on in the background (ie the accuracy and updating of regular profiles). Oh Snap. Exposed.

One of the most influential people I know doesn’t have a Klout score, but you can bet when he tells people to do things the % of people who act on his advice is very high ~50%. He only recently got a LinkedIn profile since he didn’t want people begging him for money. How are these types of people accounted for in Klout? I noticed that Donald Trump (http://bit.ly/f0aGMc) has only a 64 score on Klout. I searched “Bill Gates” on Klout and go no results. So I guess the question really is how does a Klout Score compare to real influence, online or offline?

An important thing to remember is that (as of now) Klout scores are basically the ability to get a reaction on Facebook and Twitter.

Does Bill Gates or other influencers have the potential to drop major announcements or statements that bring a large reaction? Certainly. In general their current Klout scores are pretty much a ratio for how interesting their followers find them on these platforms. Meaning sometimes they say unimportant things or nothing at all. This offline influence is a gap in the algorithm.

In the future Klout wants to better account for offline influence. But currently it does not. There are ways for an algorithm to take this into account but today they do not.

Keep in mind though. People looking for Donald Trump or Bill Gates as amplifiers of a a brand message, really need to either be a friend or have a major ad budget. Klout is not a part of either of those processes.

I have to come down on the whole issue of Klout and its worth somewhere in the middle. First of all, everyone needs metrics and ways to understand what’s happening when it comes to managing any business and that’s especially true when it comes to something as nebulous as on line influence and social media. And Klout has sure worked hard to provide a true metrics and verifiable set of standards to our understanding of how we and our followers interact on Twitter. So that is the plus side of Klout.
However on the minus side Klout can’t and for that matter no one can take into account all the different variables, methods, and reasons people use Twitter. And so their “rating” is just an opinion based on various aspects of interaction on Twitter that may have little or no meaning for many Tweeters.
As, for example, in my case being and tweeting in an area that doesn’t cause that much viral interest. (Face it tax accounting and business consulting aren’t exciting subjects to most people.) And thus the whole issue of re-tweets becomes of less importance than the spread of information of a specific nature that is not really subject to a lot of re-tweet interest. So naturally my re-tweet percentage isn’t what some teen stars’ site is. And so my over all Klout score will show that in a negative way.
Even in the area of follow backs I take a hit under the metrics and standards Klout uses. For again, many just aren’t interested in a stream of boring and non interesting facts or links to articles about tax issues. That’s to be expected. I’m like your dentist, someone you only see because you have to see them. If there was any way most people could avoid the whole issue of taxes and business accounting they would do it. And my work ranks right up there with getting a root canal on the popularity scale.
But I follow a lot of people to stay attuned with what going on and to be able to reply to Tweets I see so as to engage people in dialog. Most of them however have little interest in paying attention to my Tweets or following me back until I hit on some subject that is of importance to them.
That again means that Klout’s metrics for follower/ follow back ratio’s doesn’t really fit my particular circumstance. And my Klout score reflects that as does most accountants and CPA’s Klout scores on Twitter. You just won’t see many of us in this industry hitting the higher numbers on Klout.
What many of us need is for Klout to better take into account not only these numbers and ratio’s as monitored by their analysis and algorithms but to also better show how my Klout ranking relates to others in my industry. I’m never going to have a Klout rating that is what most would consider great and still be truly Tweeting within my area of expertise.
So Klout scores are nice and they give us a measuring standard to judge some aspects of our on line impact but only as relates to a very general definition of clout. And until Klout can share with me how I rate considering some kind of industry specific comparison its true value for me is less than it might be for many other Tweeters. The day Klout shows my score against an average score for business and individuals Tweeting on my subject matter their number is interesting but without a way to compare it to anything meaningful to me.

Still, I find Klout motivates me to interact more and more effectively. What I’d like is an app that can sort my followers by Klout. That would be helpful. I’m often surprised by people I happen to notice are up in the 50s and I had no idea.

Excellent post on Klout – seems to be a hot topic. I wrote a post inspired by a Tweet I picked up from Jeremiah Owyang. The post discusses the opportunity for Klout scores to influence the offline world.

For Finance Marketers

Stay Current

About Social Fresh

Social Fresh is a social media marketing education company. Our goal is to inspire people to create better businesses through social media. Social Fresh produces some of the top social media conferences and online training programs in the industry.