Hidden overselling

Traditionally, hosting packages have two very basic, measurable dimensions:
the disk space and the amount of monthly data transfer. Following these are
the plethora of other features which are basically permissions to use specific
server based applications: email, PHP, MySQL, ASP, FTP and many others.

While this might seem to most people like the complete image of a shared hosting
package, it really isn't. Something (or some things actually) are missing from
this picture. It will be quite surprising for some of you to find out what's
missing. Most likely it never crossed your mind because it's just too basic
to think of hosting without them.

The missing link

Once you sign-up with a host you will be allocated a slice of space to use
on the servers hard disk. The next thing you do is upload your website, set
things up and you're done. Once the new nameservers propagate your website will
be up and running.

All you have to do next is to promote your website and get as many visitors
as you can, right? Right! However, as your website grows it might need more
space or more data transfer allowance. The solution is simple: upgrade to a
bigger account, with more space and more data transfer.

Do you see what's missing from this picture yet? Do you see anything else that
your website is using besides data transfer and space? A missing link? Most
likely you don't see it, but your website is using something else. Imagine that
your website is transferring 10,000GB per month. Still not finding what is missing
from this picture? Well, that something is generally referred to in the hosting
industry as "resources".

Come again? Resources? What are those? They're the things that make the whole
server work; basically the CPU (the processing unit, the "brain" of the computer)
and the internal memory. Fact is that the whole "shared hosting" concept
almost revolves around these two things as they are two of the most important
things that all the websites hosted on a server share.

Because the access to these shared resources is basically unrestricted, a website
or script will use as much CPU and/or memory as it needs - sometimes to the
point where the other websites on the server are affected.

Obviously the administrator cannot and will not let a client use resources
up to that point. In the Terms of Service the host will reserve the right to
temporarily suspend and even completely terminate an account that uses too many
resources. It's basically a "fair use" policy. After all, you cannot
expect to use the whole processing power of a CPU when you're paying only for
a share.

However, trouble is no one knows what that share is (besides the host of course)
so this opens a very big door for abuse.

Overselling resources

No matter how you look at it, the CPU and the memory are oversold.
Why? Well, for one because there's no real/exact limit to their use.
The host will approximate an "average" or "normal" usage of resources
per unit of transferred data and will suspend a website that uses too
much over that value, so that it affects the performance of the server.
It will then ask the customer to upgrade. Websites will generally be
allowed to use higher than average amounts of resources from time to
time, but not all the time.

Somehow this whole thing remembers me of a Seinfeld episode where George dipped
his chip, took a bite and then dipped again. Let's assume that the data transfer
and the hard disk space are the chips and the dip represents "the resources".

What a host does is sell you and the other clients a number (quantity) of
chips, put a bowl with dip in front of all of you and say: "Dipping is
free, but the bowl is only this big. Everyone can dip according to his needs,
but if we notice that someone abuses dipping, up to the point where it's affecting
the other clients using that same bowl, they'll get a slap over their hand.
If they do it again we'll not only slap them, but we'll point them to the door."

Sure this example is about products rather than about services (as is the case
with hosting), but it's just meant to be a way of giving you a more vivid picture.

Going a bit further now, one cannot overlook the fact that these resources
that we're talking about are often strong selling points. Many hosts mention
the fast processors that they use and the vast amounts of internal memory that
their servers are equipped with. This supposedly means that the client gets
access to a powerful computing system so his website will "fly like the
wind".

Well, considering that the CPU power is not oversold, just how much of it should
a customer be allowed to use? Here's an example:

Say a CPU is shared by 200 websites (equal accounts in terms of space and data
transfer). That means a website should be allowed to use a maximum of 1/200
(or 0.5%) of the processing power of the CPU. Less even, because a server becomes
unstable at high CPU usage, so a maximum figure of 0.25% is more likely.

However, a host will not close websites for using just 0.25% of the processor,
even on a consistent basis. You might get to read about limits of 5% or 10%
CPU usage in the TOS of some hosting companies. (Yet another sign of overselling.)

Most hosts allow the customer to use high amounts of resources if he did not
use much of his data transfer quota. This is done in an effort to show that they're
not overselling data transfer, which may very well be true, but in this way
they actually prove that they're overselling the other resources.

Translation: one user can "consume" more CPU than others and even
much more that his money have actually bought. If customers pay for 0.25% each,
but are allowed to use up to 5% on a consistent basis, it's obvious that the
host places a bet on the fact that very few websites will actually use their
0.25% share of the processing power. The very basis of overselling: because
users on average don't actually use what they pay for, the provider can promise
a lot more - as long as just a few of the customers actually use it.

What if all/most of those websites suddenly start to use 1-2% of the CPU? Obviously
the server will not be able to cope with this and the company will have to redistribute
the websites on other servers and cover the costs, but for a while it will be
unable to provide the services. Why? Because they are sold beyond he means of
delivery or, in a single word, oversold.

Extremes make great examples

Many hosts boast the fact that they allow users to consume high amounts of
"bandwidth" and "resources" without asking them to upgrade,
yet they still have a "resources abuse" policy in their TOS.

Translation: "we do our best not to limit usage but because we're overselling
the resources, at some point we have to tell you that you're using much more
than what you paid for and ask you to upgrade."

When hosts make the packages they basically add the costs of running the server
and decide on a selling price. That price is then divided into pieces (accounts)
according to allocated hard disk space and bandwidth.

Trouble is that a website using 100GB of monthly data transfer could very well
need a whole server in terms of CPU usage. Sure, when you're paying for 100Gb
of data transfer on a shared server, you're not supposed to use 100% of the
CPU, but, as long as CPU and memory usage are not clearly defined/limited, measured
and sold, you are at the mercy of the host. If the host says that you were using
too much then that's it. You will either have to upgrade (often this means to
get a dedicated server) or you'll have to change hosts hoping (as strange as
it might sound) that the current host lied in an attempt to make you pay more.

Sure, most hosts will not like my statement but some will agree to it, I'm
sure. There are hosts out there that are definitely against any kind of overselling
and they most likely know by now that this resources thing is a gray area. They
are not overselling space nor data transfer, but they can't really do anything
about the resources because the tools for measuring their usage in the way that
bandwidth and space are measured are not available.

Sure, the VPS (virtual private server) might seem a good solution, but that's
not really shared hosting and very small VPSs are not really efficient. Plus,
a VPS generally requires more knowledge from the part of the user, a knowledge
that the average user simply doesn't have.

Is overselling a bad thing?

Tough question to answer! As this is a very controversial issue all I can do
is express my opinion on the matter (again). I do believe that overselling can
be done smartly, even when it comes to resources. One proof is that even though
all "shared hosts" oversell the CPU's capacity there are lots of good
hosts... successful hosts out there, proving that it can be done smartly.

This article is not about overselling being "bad", it's about the
fact that as a hosting customer you're not given the specifics of your account.
You don't know what's your share of CPU and memory, as you do with space and
bandwidth.

The future

Probably sometime in the future (not a very near one though) this issue will
be dealt with. However, I don't see a movement towards this end - not from the
part of the hosting companies nor from the part of the customers. Most people
seem to be satisfied with the current status-quo. Without a driving force behind
it, such a change will not happen.

I do believe that the real complete package (with the CPU and memory usage
clearly specified in some way) is a thing that could help hosts and customers
alike and I'm looking forward to see it become a reality. All I can do though
is write this article, wait, and hope that it will help people realize that
there are things that need to be improved.