Twitter Engagement Unmasked: A Study of More than 4M Tweets

What you see before you is the result of analyzing more than 2 million Western language tweets out of over 4 million Tweets we collected. In this paper we will outline the results of analyzing that data to learn and understand what drives behavior on Twitter. One of the main goals: to see what really drives retweets, favorites, and replies. (The remaining non-Western tweets will be the subject of future studies.)

For example, did you know that including images in your tweets could raise the number of retweets you get by 4 times or more? There is much more in the study. Factors we considered included:

Inclusion of media (images or video) – we also broke this out by number of pieces of media

Character count

Inclusion of Hashtags – we also broke this out by number of hashtags

Hashtag length

Inclusion of Links – we also broke this out by number of links

The Domain Authority of the shared link

Time of Day

Inclusion of Mentions – we also broke this out by number of mentions

In addition, we used Followerwonk to measure the social authority of the person issuing the original tweet to see how that impacted behavior. We broke that out by 10 levels: 1-9, 10-19, … 90-99. It was interesting to note how much the impact of various tactics differed by social authority. This is one of the more important and unique aspects of this study.

In today’s post, I am sharing the results of the Western language based results. The regions of the world included are: North America, South America, and Europe. Tweets in script-based languages, such as Japanese, Hangul, Arabic, Cyrillic, Thai, and others are excluded in today’s post. However, we will be publishing follow-up posts covering each of the 5 above-mentioned languages in the near future. (Update: Our Japanese Twitter study is now published!)

Scope of Study Results Published Today

The 4 million tweets pulled included:

1,943,231 Original Tweets in Western Languages

1,164,958 Original Tweets in Script Based Languages

The balance of the data was in the form of Retweets and Favorites

The study data presented here focuses on the 1.9 million original tweets in Western languages only. As noted above, we broke this all out by social authority. The splits of accounts by social authority was as shown here:

The total number of original tweets posted by the users in each social authority grouping was as follows:

The following chart containing the social authority levels of people you may know, to help you get a better sense for each of the social authority levels:

The Importance of Statistical Significance (Why We Looked at So Many Tweets)

Why do we need to look at so many tweets? The primary reason this is important is that small amounts of data are going to provide us with inaccurate conclusions (because they are not “statistically significant”).

For illustration purposes, imagine you send out a single tweet, and it gets 1M retweets. Can you then conclude that every tweet gets 1M RTs? Of course not, because the sample size of 1 is not statistically significant. When you evaluate so many different categories of data as we did, you also can’t draw meaningful conclusions based on 100, 1,000, 10,000, or even 100,000 tweets, because some of the smaller categories we evaluate will end up with too little data.

For example, in our 90-99 category we had a total of 21,856 Retweets from 266 users, and if we had a much smaller sample (say 100,000 in stead of 1.9M, this could reduce our sample set of tweets for the 90-99 range down to around 1,000 total tweets from 12 or so users. Then only some of those will include images, or hashtags, or mentions. We could have ended up with categories we were trying to evaluate with a couple of hundred tweets from 4 or 5 users.

When you look at factors like media vs. social authority level, or time of day vs. authority level, or hashtag lengths, and these types of things, you need to have a lot of data to make sure you can provide accurate conclusions and insight.

This does make the project much more daunting and difficult to do, but having more data does allow us to dig a lot deeper into the way things work.

Chances of Any Engagement vs. Total Engagement

For all of our data we looked at it from two perspectives:

1. Chances of Any Engagement: This concept focuses on analyzing what percentage of tweets get at least one RT or at least one Favorite. That’s useful in understanding how you can create an initial reaction, but it does not measure the depth of that reaction (i.e. how many tweets or favorites you get)

2. Total Engagement: Here we measure the total number of Retweets or the total number of Favorites received. For example, a single tweet might get 27 RTs and 36 Favorites if we build it without an image, and 115 RTs and 188 Favorites if we add an image.

This may still be confusing, so let me illustrate with an example scenario:

We actually did find some differences between behaviors that increase chances of any engagement vs. total engagement, so both are well worth understanding, and we addressed both these dimensions throughout the study. This proved to be particularly important when it came to mentions.

Methodology

All data was pulled via the Twitter API. We pulled tweets directly from the main Twitter feed and kept going until we had a large sample of users. We ended up with 27,143 western language users and 15,317 script based language users. As noted before, today we are publishing the results just for the western language based users, and we will address the script based language users in one or more posts coming soon.

Impact of Social Authority

As you might expect, social authority has a huge impact on your ability to get Retweeted or Favorited. The following chart shows the scope of that impact:

This is one of the reasons we broke out all of our tests by social authority. The rules of the game are quite different for those that have everyone’s eyes on them already.

The Power of Images

It’s been documented by others that media (images and video) have a big impact on social media engagement. Just how much? How does that vary by social authority? Our first chart shows you how your chances of getting at least one Retweet or Favorite are increased by using media:

Here is a more detailed look at how images drive the chances of getting at least one Retweet, as broken out by Social Authority. (The percentages in the vertical axis are the percentage of tweets that got at least one retweet at that Social Authority level):

Here is a more detailed look at how images drive the chances of getting at least one Favorite, as broken out by Social Authority.(The percentages in the vertical axis are the percentage of tweets that got at least one favorite at that Social Authority level):

In lower authority levels, your chances of getting an RT or Favorite more than double. However, it gets even better than that. Let’s look at how including images can increase the total number of retweets you might get. (The vertical axis is the multiplier; that is, it shows how many times more likely a Retweet is to occur when an image is included.)

Here we can see a chart of the gain in total Favorites resulting from including images. (The vertical axis is the multiplier; that is, it shows how many times more likely a favorite is to occur when an image is included.)

At lower authority levels including an image will get you 5 to 9 times as many Retweets and 4 to 12 times as many favorites than you will if your tweets don’t include an image. Hopefully, you were sitting down when you read that. Note that high authority levels also benefit as well, though for the 90-99 range the gain is relatively modest. For those high authority accounts, people are already hanging on their every word.

If the difference between chances of getting at least one RT (or Favorite) vs. the Total RTs or Favorites is confusing to you, please jump back up above to read the section on Chances of Any Engagement vs. Total Engagement.

The short conclusion: if you are trying to get RTs and Favorites for your tweet, you’d have to be insane to not include an image with it.

Having said that, this does not mean you should include random, crappy images with your tweets. It remains very important to make them relevant, and something that helps evoke an emotional response from the viewer. There is no doubt in my mind that these huge upside numbers result because there are many people out there who are very good at doing that, and their average gain levels are probably even higher.

Engagement on Twitter is about Retweets and Favorites

As you saw in the prior section, Retweets and Favorites are frequent events on Twitter. 36% of all tweets get at least one RT, and 43% of all tweets get at least on Favorite. The story with replies is quite different. Only 0.7% of tweets get a reply.

That means there is a whole lot of broadcasting behavior occuring on Twitter. Engagement is primarily defined by RTs and Favorites.

Certainly, there are lots of people who have regular dialogues on Twitter, but the platform appears to be predominantly a place for sharing news and information. Inclusion of images does appear to have a very real impact on replies as well; it’s just that the numbers are still quite small:

Character Count

Prior studies have shown that character count impacts Retweet rate. We found the same: the longer the tweet, the more likely it is to receive at least one Retweet or Favorite, but the effect levels out after 120 characters.

Use of Hashtags

This is also a long held popular belief – that including hashtags impacts your chances of getting RTs or Favorites:

As you can see, hashtags do increase Retweets and Favorites, but at a rate significantly less than images do.

Here is a look at the impact of hashtags on RTs as broken out by Social Authority:

Here is a look at the impact of hashtags on Favorites as broken out by Social Authority:

The statement appears to be generally true, except at very low and very high social authorities, where including a hashtag does not seem to increase your chances of getting at least one RT or Fave at all!

Hashtag Length

Here we looked at the length of hashtags, to see if that made a difference. Our initial look showed no major variance based on hashtag length. (The groups represents the ranges of hashtag character lengths):

As with many of these types of criteria, it is likely that the most important factor is the context and content of your tweet. If you plan to include a hashtag, pick a hashtag that makes the most sense for your tweet.

Including Links in Tweets

Does including one or more links in your tweets impact engagement with your tweet? On an aggregated basis, the data tells us that including a link has a minor impact on your chances of getting Retweets or Favorites for your tweet:

Here is the impact of Links on Retweets broken out by Social Authority:

Here is the impact of Links on Favorites broken out by Social Authority:

However, if you dig deeper and look at how it impacts the total number of RTs a given tweet might receive, the story gets even more interesting:

If your social authority is 50 or higher, or 9 or lower, the total number of tweets and favorites you get appears to go down if you include a link. Probably worth letting that one soak in for a moment! The bottom line here is that whether or not you include a link should not be driven by a desire to get more Retweets or Favorites, but should be dictated by the content needs of your post.

Time of Day

This is one of the most popular myths (hint) about Twitter – that there is a best time of day to tweet. Let’s look at the data at an agreggated level:

This line is almost flat as a pancake. This suggests that time of day has near-zero impact on your chances of getting at least one Retweet of your tweet.

However, it’s certainly possible that the results might be different if sliced based on different market verticals and the time zone of the person doing the tweeting. That is not something we looked at here, but on an aggregate level, it’s clear that this is not a major factor in Twitter engagement.

In addition, people tend to tweet in their own time zone, and during their normal waking hours. This could explain why our aggregate look at the data shown no indication of a “best time of day” to tweet. Other studies that used different parameters may yield different results.

Keep in mind that our chart shows the chances of being retweeted at any particular time of day. However, the volume of tweets will almost certainly vary with time of day in your own time zone. It is reasonable to expect that there are going to be fewer tweets and fewer people online viewing them in the middle of the night than during the day in your own zone. But our results show that no matter when you tweet, the chances that you will be retweeted are essentially the same.

Twitter is an extremely accessible platform. Our belief is that you don’t need to worry about tweeting at 10 AM in the morning or 5 PM in the afternoon. In general, tweeting during your normal working hours will probably be just fine. However, if you are breaking major news, different guidelines might apply (for example, we published this post at 11:30 EST to be during the morning hours of the work day for North America).

Including Mentions

What about mentioning someone (including an @ with their username)? Does that have a positive impact? Let’s look at how mentions increase your chances of at least one Retweet:

For the 1 to 59 authority range, it looks like the impact is positive, in terms of getting at least one RT or Favorite for your post. The story chances for higher authority users, where the impact becomes more or less a wash. Let’s take a look at how Mentions increase the total number of Retweets you might get:

Here the conclusion is different. While our first chart showed that your chances of getting at least one RT go up, our second chart shows us that the total number of Retweets any particular tweet will get will likely go down.

Why would this be the case? Chances are that it’s likely that you will draw a reaction from someone that you mention in your tweet. However, since that feels more like a targeted conversation, it may be that fewer other people would want to participate in it. Intuitively, this makes sense.

Takeways

It’s important to remember that this study looked at tweets in a highly aggregated manner. The specific content and context of your tweet is a vital factor to consider in developing your tweet.

Whether you mention someone, include a link, include a hashtag, or when you post it, should largely be guided by the intended content and context.

However, if you want to get real engagement with your tweet, you absolutely should include images. Character length seems to be worth paying attention to as well.

The art of a great tweet can benefit from what we have revealed here today, but it should not be a sole determining factor. Whether or not you include hashtags, mentions, links, or post at a specific time of day – that should be determined by the needs of your content. What image you include, should also be determined by the needs of the content as well.

Remember too, you can’t just throw any image in your tweet. You need to think carefully about what you do, and ideally create an image that is highly relevant, and that can evoke a reaction from people that see your tweet.

If you are interested in speaking with someone at Stone Temple Consulting about help with content marketing, social media consulting (including how to get more engagement on social media platforms), or SEO, please contact Us Here. You can also use that email address to request addition to our mailing list. People on the mailing list get proactive notification of our studies (yes, there are a LOT more coming!).

Really insightful, thanks for posting. Did you also happen to compare the audiences of say Rand to Bieber to see if there’s any difference in propensity to generate RTs, Favs, etc based upon a similar authority?

This is a nice writeup that confirms many of my suspicions about Twitter. I’ve especially noticed that my image tweets get far more attention than my text-only tweets. I’ve even seen some people tweeting images of text and those do better than many text-only tweets, oddly enough.

However, I do take issue with your discussion of statistical significance. You write:

>Why do we need to look at so many tweets? The primary reason this is important is that small amounts of data are going to provide us with inaccurate conclusions (because they are not “statistically significant”).

>For illustration purposes, imagine you send out a single tweet, and it gets 1M retweets. Can you then conclude that every tweet gets 1M RTs? Of course not, because the sample size of 1 is not statistically significant. You also can’t draw meaningful conclusions based on 100, 1,000, 10,000, or even 100,000 tweets.

That’s just plain wrong. I mean, of course you can’t do statistics on 1 tweet, but 1,000 tweets — randomly sampled — could come out statistically significant when you analyze them. And just because you analyze 1,000,000 tweets doesn’t mean that whatever you analyze is suddenly statistically significant, either.

You also somewhat contradict yourself with this statement by suggesting that analyzing only 100,000 tweets wouldn’t be statistically significant, yet when they show the distribution of tweets that you analyze (broken down by Social Authority), several of the bins have <100,000 tweets.

Hi Randy – with a post like this I have to walk a fine line between how much of the various calculations we did that I put in it. This is the reason that I did not put error bars into the post. It would clutter the post.

Also, for the record, the 90-99 social authority bin had 266 users and 21,865 tweets. As you say, that is less than 100,000 tweets. However, my point was, if we had tried to do this study with a grand total of 100,000 tweets, instead of almost 2M, after we sampled users and split them into social authority buckets, a category like the 90 to 99 one might have had only 10 users and less than 1,000 tweets in it. While you could argue that the 1,000 tweets could offer statistically significant data in it, clearly the 10 users can’t. It is way too susceptible to a sampling bias, even if you attempt to be random with how you approach it.

But, because we used a total study size that comprised almost 2M western language tweets, our smaller categories still had enough data in them to represent statistically significant data. I probably could have explained that better, and I will update how I describe this in the post. Thanks for helping draw that out for me.

@Randy – Re “tweeting images of text and those do better than many text-only tweets, oddly enough” – not odd really, just that images (even if just containing text) tend to catch your eye more; same reason I still stop and look at the pictures in boring books 😉

Other than Rand and Chris are those truly “high social authority” users? Do they tweet and retweet on their own? Do they create and take part in conversations through @ replies? Do they respond to people who @ mention them? I don’t assign any authority to users like Justin Bieber, Lady Gaga or others who have mountains of fans by virtue of their celebrity but who use social media as a one-way marketing tool. They get “engagement” not because they provide useful or interesting material but because they have legions of fans who love everything they do.

Hi Hans – it’s not really about a single retweet. It’s about the chances of “at least one retweet”, so the overwhelming majority of the time the results will be more than one retweet. We also measured the impact on the total number of retweets too. For example, including an image increases the total number of RTs you get by 5 to 9 times. That’s pretty significant!

I understand that it’s not about getting a single tweet. I was tuning in here, because I was looking into the efficiency of twitter marketing lately and to be honest, I have the feeling it is just an illusion. The number of visitors a normal account creates are so small. What kind of companies can drive business this way? The profit per visitor must be huge, if it’s worth it.

I see so many twitter accounts with thousands and even tens of thousands of followers but their tweets get favorited only once or twice. So effectively, nobody is reading it anyhow, right?

So I was reading the study and was thinking: Ok, you can analyze all you want. But what is it good for if the absolute engagement numbers you get from Twitter are effectively zero? A favorite here, a retweet there, a visitor to your website every other tweet… how does anybody make real money out of that?

Hi Hans – it’s a great question. Making money off of Twitter directly is, in my opinion, a non-starter for the great majority of people. You have to view it as a platform on which you can build your reputation and visibility, and have the lead to later conversions. The impact is an indirect one.

Ok, so it’s not about driving traffic but about building “reputation and visibility”. That sounds a lot like brand advertising.

But since this study is about how to improve engagement, I still wonder: why? If the absolute engagement numbers are effectively zero, why put effort into it? If you can only get 0.1% of your followers to engage – why bother?

Looking at Barrack Obama, he get’s about one retweet per 50,000 followers.

Tweets from non celebrity accounts seldom get even 10 retweets. Looking at @stonetemple which is probably highly optimized, I see maybe 4 retweets on average.

Is that really making an impact? Can I measure that impact? If not, what would stop me from thinking it’s all just an illusion and it’s better to spend my time on other, measurable, forms of advertising?

Great point Hans. Most of my web site’s traffic comes from social media. Despite the fact that I have 40k+ followers on Twitter (and regularly receive 20+ RTs/favs), the traffic and engagement that I receive from Twitter is minuscule compared to Facebook, Hacker News, SlashDot, and especially Reddit.

IMO, Twitter is better used as a professional networking tool rather than a traffic and engagement driving tool. Unless you can get a celebrity with 1M+ followers to tweet something for you, it’s just not worth the effort if your goal is traffic and engagement.

(Of course, IMO it’s invaluable for a company to have a customer service presence on Twitter to handle the oh-so-common customer complaint tweets.)

Great post! Some interesting results in there. Surely the time of day results depends on what industry you are in though, there will be quite a few industries when posting at the right time of day as a big impact. I’m thinking deals websites here as there customers aren’t going to be as active on social media during the day as they will be during the evening.

Glad you enjoyed the post, Karl. Yes, as with any big data study, “your mileage may vary” as they say. These results are averaged across a great many accounts of all types.

However, keep in mind that unlike most other studies, what we’re showing here in “time of day” is not the average number of retweets you can get at each hour, but the chances (or probability) of getting retweeted at a given time. So what we’re really revealing here is that while there certainly may be times of day when you will get more RTs (because more of your followers are active), you shouldn’t ignore the off hours when your chances of getting at least one RT may be higher.

Having developed proprietary tools for social analytics and social campaign optimization methods using large scale data collection, this is one of the best recent public studies presented in a non-juried publication format that I have seen.

What an amazing article! We spend so much time working with our clients twitter accounts, it’s always so cool to see these types of reports that make actionable steps to be taken from raw data.
Thanks again!

Fantastic work! I do wonder about confounding factors. For example, did you control for presence of images when looking at hashtags or vice versa? Is it possible that simply *better* tweet content is likely to be accompanied by hashtags or images?

I do not yet fully understand how to best use twitter but I did try to add pictures and hash tags and the engagement did increase slightly. Clearly there is much more to learn about using this social media platform especially on the business side. An article on how to best use it for business purposes would be enlightening

Great post! Some interesting results in there. Surely the time of day results depends on what industry you are in though, there will be quite a few industries when posting at the right time of day as a big impact. I’m thinking deals websites here as there customers aren’t going to be as active on social media during the day as they will be during the evening.