Welcome to the IT Computing Section at AnandTech

Benchmarking is an art form. The subtle nuances of methodology design and test
bed engineering can take months, even years, to master. With the nearly endless
variety of PC usage scenarios, it's easy to make a mistake and draw the wrong
conclusion. One false assumption and your results are worthless.

Case in point: The Corporate Enterprise. Mainstream enterprise computing is
different from hobbyist/enthusiast computing. The expectations are different.
The usage model is different. You need to understand both before you can formulate
an effective testing methodology.

Sadly, many fail to grasp this simple truth. These "expert wannabes"
approach virtually every testing project the same way: Load-up their favorite
linear productivity and gaming benchmarks and fire-away. They give little or
no thought to how a system will be used in the real-world and so often miss
the point of why a platform is designed a certain way.

In an effort to educate novice readers on the realities of enterprise testing,
I've compiled the following list of guidelines to help bring them up to speed:

1. Enterprise customers do not overclock. Period. Overclocking is something
hobbyists do in their spare time with their own hardware. End users who over-clock
their corporate desktops tend to find themselves unemployed shortly thereafter.

2. Enterprise customers do not run games. Again, gaming is something enthusiasts
do at home in their spare time. As far as enterprise computing is concerned,
gaming benchmarks are (or at least should be) irrelevant.

3. Enterprise customers do not operate in a vacuum. Virtually all enterprises
are connected via some form of network infrastructure. These typically include
various network-level services, like encryption, firewall protection, proxy
redirection or network caching. Test scenarios that fail to incorporate connectivity
(either online or offline, via network caching) to a realistic network environment
are irrelevant.

4. Enterprise customers do not work in a "bare iron" environment.
The typical enterprise client computing "stack" includes multiple
layers of systems management, security and connectivity software. Many corporate
login scripts are mini-applications in themselves, with synchronized single-sign-on
to multiple, back-end resources. Simply loading up the OS and a benchmark
script doesn't cut it. The PC must be tested within the environment for which
it is being deployed, using features and functions relevant to its day-to-day
usage model.

5. Enterprise customers do not single-task. Outside of the odd data entry
kiosk or Point of Sale (POS) system, most enterprise users run more than one
application at a time. Often these applications are active in the background,
retrieving data, monitoring processes and integrating the user into the flow
of corporate information. The old model of serial task switching between individual
productivity applications is simply invalid.

Mastering the art of enterprise testing can be a challenge. You need to spend
some quality time supporting an enterprise-caliber IT operation before you can
fully grasp the unique needs of the modern knowledge worker. At CSA,
we've spent years working with such organizations in an effort to refine our
testing methodologies and create new tools that simplify the benchmark design
process.

The first fruit of our labor, Benchmark Studio, represents our vision for a
sophisticated load simulation and performance-testing platform, one that encompasses
the full range of enterprise computing usage models. Our follow-on solution,
OfficeBench 2001, is an attempt to distill our methodology even further by providing
an easy to use complement for other currently available benchmarks.

Together, they give us a powerful toolkit with which to tackle the tough IT
questions of PC platform scalability and client/server application performance.
And thanks to Anand, we now have the opportunity to share our experiences with
one of the strongest and most vocal communities on the web. We look forward
to working with Anand and his talented staff and to making our own small contribution
to the AnandTech community.

The IT Computing section on AnandTech will house articles that deal with the
trials and tribulations of computing in the IT/Enterprise worlds. These articles
and editorials will serve as a complement to AnandTech's current content, hopefully
to provide you with a better overall picture of the computing industry for all
aspects of your life both work and play.