Most websites load reasonably fast when visited by their average number of users. However, performance rapidly deteriorates when a site is overwhelmed by peak traffic (the times when the site’s traffic is the highest) and during traffic spikes.

In a quest to learn about the art and science of peak traffic estimation, I began to study some publicly available data to see if I could try to discover a connection between peak traffic and the average traffic of a website, as well as the type of traffic it can receive.

Summary and Data

For the purpose of this study, we will only look at traffic data from the US. We also obtained the average and maximum daily traffic over a 6-month period. This data was first retrieved on February 12, 2011. Only MRC accredited data was used.

The following summarizes the data obtained.

High-Ranking Websites (About 400,000 Visitors Monthly)

Rank

URL

Monthly visitors

Average daily visitors

Maximum daily visitors

*Factor

4,988

collegecandy.com

409,400

13,647

61,700

5

4,912

myplick.com

415,400

13,847

39,900

3

4,908

djbooth.net

415,600

13,854

29,500

3

5,009

wndu.com

407,500

13,584

104,000

8

4,937

joycemeyer.org

413,600

13,787

53,000

4

4,978

mapmyrun.com

410,200

13,674

50,600

4

4,964

techdirt.com

411,700

13,724

80,800

6

5,010

curezone.com

407,400

13,580

18,700

2

5,072

stupidvideos.com

403,000

13,434

25,300

2

*Factor is the maximum daily visitors divided by the average daily visitors.

Notice that the maximum daily visitors are 2-8 times higher than average daily visitors. This is from monthly traffic in the 400,000-visitors range.

Middle-Ranking Websites (About 190,000 Visitors Monthly)

Rank

URL

Monthly visitors

Average daily visitors

Maximum daily visitors

*Factor

9,938

koinlocal6.com

195,300

6,510

56,400

9

9,943

charitynavigator.org

195,300

6,510

28,900

5

9,948

fivestaralliance.com

195,200

6,507

10,400

2

9,952

justapinch.com

195,100

6,504

12,600

2

9,967

futilitycloset.com

194,800

6,494

26,300

5

9,969

profootballweekly.com

194,800

6,494

21,700

4

9,973

schaeffersresearch.com

194,700

6,490

22,900

4

9,978

temptalia.com

194,600

6,487

19,100

3

9,916

clatl.com

195,900

6,530

18,300

3

10,155

famoushookups.com

191,200

6,374

62,900

10

Note that at this traffic range, maximum daily visitors are now 2-10 times higher than average daily visitors. Monthly traffic that each site received was about 190,000.

Lower-Ranking Websites

Rank

URL

Monthly visitors

Average daily visitors

Maximum daily visitors

*Factor

19,895

miamiandbeaches.com

89,900

2,997

5,100

2

20,146

michaelmoore.com

88,700

2,957

110,000

38

17,722

talenthouse.com

103,200

3,440

88,300

26

19,996

nmh.org

89,500

2,984

6,100

3

20,025

campusgrotto.com

89,300

2,977

7,400

3

19,920

times-standard.com

89,800

2,994

16,200

6

20,181

slantmagazine.com

88,500

2,950

6,700

3

19,904

ldoceonline.com

89,900

2,997

8,600

3

19,763

yipit.com

90,600

3,020

13,200

5

20,173

pregnancyguideonline.com

88,500

2,950

6,500

3

You may notice that maximum daily visitors are greater than average daily visitors by a factor between 2-38 times. The variance has thus increased dramatically. A simple explanation would be that higher-ranking sites with high average monthly visitors are less likely to be affected by sudden surges in traffic.

Large sites like CNN, TechCrunch and Mashable have high, stable visitorship, and the impact of a popular news story such as protests in Egypt or a new Apple product would be much lower as compared to a site with low average visitorship.

On the other hand, a site like michaelmoore.com can be flooded with traffic if a particular piece of news about the site owner becomes viral.

Let’s take a closer look.

Three Types of Traffic Patterns

There are three main types of traffic patterns.

High Stable Increasing Traffic

WordPress.com has high stable traffic of 44.1 million monthly US visitors. It has a peak daily traffic of 2.2 times of its daily average traffic.

Extrapolation of the rising trend puts their estimated peak daily traffic to be about 4 million by June 2011.

Seasonal Peak Traffic

Some sites experience spikes in traffic due to seasonal promotion or normal business cycles. Let’s take a look at data from Holidays Central.

Notice that there is seasonal peak traffic. In this case, we can estimate based on past data, but we need to also factor in growth in traffic.

Low Stable Traffic

On December 14, 2010, Michael Moore decided to contribute $20,000 to Julian Assange’s bail, resulting in a lot of press, which coincides with the drastic spike in the site owner’s website traffic. This news was reported on major news channels, causing traffic to spike 38 times higher than normal.

Defend Against Sudden Traffic Spikes

It is always hard to predict peak traffic for sites like michaelmoore.com because there is always a possibility that some of its content becomes viral.

According to Scott Galloway, Clinical Associate Professor of Marketing at NYU, there are 3 elements of viral content:

Authenticity

Humor

Social Debate

In Michael Moore’s case, we can see these elements coming into play.

What is interesting, though, is that most of the time, going viral normally catches people by surprise. They are not prepared for sudden fame. Neither are their websites.

Imagine your website being hit with 100,000 visitors in an hour — you should be overjoyed, right? But many site owners actually have a bad experience because the site is slow or unavailable.

If your website falls into the category of having low, stable traffic with the chance of going viral, you should not hesitate to estimate that your requirements would be more than 30 times your average traffic.

Controlled Traffic Spikes

Besides being viral, normal marketing and promotion can also cause a spike in traffic. As a site owner, this traffic spike should be more predictable to you and is more likely to happen than accidentally achieving viral site traffic levels.

For example, the site completebody recently did a Groupon promotion. This resulted in a traffic spike from 150 regular daily visitors to 7,000 in one day. That is more than 46 times more than the normal traffic.

Conclusion

We can follow these guidelines for estimating peak traffic:

If you do not have prior peak traffic data, if your site has low visitorship, and has content that could go viral, account for peak traffic that can be up to 30 times your average daily traffic.

If your site has high and stable visitorship, peak traffic can be up to 5 times your average daily traffic.

If you have prior peak traffic data and timing of peak traffic is predictable (seasonal traffic and controlled traffic spikes), use past data and add a percent growth in traffic to arrive at the final number.

Developers and website owners should determine their site’s capacity by using load-testing tools such as Load Impact (full disclosure: I work for Load Impact) to simulate traffic to their website. That way, they will be able to understand how the website performs under high traffic and tune the website before peak traffic arrives.

18 Comments

A very interesting analysis and thanks for sharing. We also see large peaks in traffic when we run Email Marketing campaigns for our clients which is always expected. It is a good idea to invest in cloud hosting, despite being more expensive it allows you to easily monitor your bandwidth and simply upgrade your processor power and memory allocation instantly. It’s also a good idea to have website monitoring so if you suddenly become famous overnight, you’re the first to know and can ensure your website it working. Hope that helps someone.

It’s interesting to see how popular some of these sites are. One of the concerns for websites that aren’t used to a huge spike of traffic is whether there hosting provider can handle this amount of traffic.

Many of these websites will be running many dedicated servers of even using the “Cloud”.

Rhoda Bernstein

Also, folks might want to check out node.js and Joyent. I’ve been doing some contract work for them (marketing) and if you look at their almost counter-intuitive approach to autoscaling, it helps deal with unanticipated spikes. I am only pseudo techy, so I could be completely off base. But, I am interested to know what those of you who are truly scaling geeks (yes, I am not worthy) think about Joyent’s approach and node.js.

One of my sites, recently got stumbled, and as a result of the stumble had 2 other articles “stumbled” pushing a ordinary site up from a small visitor count to just over 90 000 visitors for Feb. The climb has steadily increased in the past three months. with new highs being reached every month.

I thought that it will be a nice way to see when the site would crumble, but to my surprise everything went well apart from me having to jump in and get extra bandwidth a time or two.

This all happened on a shared hosting plan with not even caching or anything.

I think that a proper host is prepared for more than we might imagine.

Wow, nice info and detailed research. Loved it and glad I can see it before it happens. I had the biggest spike a few days ago and it was about 5 times my usual traffic. But fortunately it didn’t crash my blog :)

Jack

I think the key issue is not whether the website is able to load, but rather how fast does it load? That is a more qualitative feature since most people get irritated by slow loading websites. Maybe your bandwidth can handle the load, but because of your server caching settings, inefficient code, database errors, redirects.. the website still loads slowly under high traffic.

It’s about time somebody wrote about this. What do you think of these “grid services” web hosting providers are offering? I was under the impression that these grid services were a solution to sudden peaks in traffic.