httpd-dev mailing list archives

I haven't actually looked at ZD's previous benchmarks before in
detail because I never saw real graphs.
Now that I look again, they have better detail online.
http://www.zdnet.com/pcmag/features/webserver/_intro.htm is their
latest test, from... erm... July 97? Not sure of the date. That's
the last-modified anyway. Oh, this tested 1.1.3.
A reasonable quality PDF version of their results is at:
ftp://ftp.zdnet.com/pcmag/1997/0527/labsdata/in1610ba.pdf
This shows a few things.
First, Apache had decent latency throughout their tests.
The two servers they tested on Solaris had almost identical latencies
just below the 200ms mark, and they were identical right from 1 to 60
clients. Whenever you see 200ms constants around TCP you should be
suspicious; that is just obvious. This is, if I remember, the test that
led to the documentation of the slow-start "disagreement" between Solaris
and NT. The fact that ZD did no more than say "hmm, that looks odd"
and ask Sun doesn't give me confidence. These are the things you need
to look at, at the protocol level if necessary, to ensure that you are
not intorducing bogus factors into your tests. Blackbox doesn't work.
For single processor static content throughput, Apache was middle
of the pack. About the same as Netscape Enterprise and Fasttrack
on IRIX, but below a bunch of others. It was around 5000kbytes/sec.
IIS hit the best, hitting up to ~13000kbytes. Everything else
(eg. FastTrack and Enterprise on NT, DGUX) topped out at ~9000kbytes.
Apache was right in there growing at a linear rate with the number
of clients, like most other servers, until it hit 12 clients, then it
hit a cap and stopped increasing.
In their NSAPI test, Enterprise on Solairs showed almost the same
rate for both NSAPI and CGI, while on all other platforms there
was a drastic difference. If that doesn't set off red lights...
Oh yea, I like this:
Some products, such as O'Reilly's WebSite, have higher latency
numbers than others because of the way the Web server handles
incoming requests. This product goes to disk to read a fresh version
of the requested HTML page every time a request comes in. Going to
disk every time a page is requested results in a much slower response
time than caching the page right away (but possibly still acceptable).
Gee, Apache had pretty good latency even in your tests but you didn't
complain about it always going to disk now did you...
Apache showed almost zippo (but did have a little bit) increase adding
another processor. IIS showed almost zippo increase. Enterprise on NT,
OTOH, showed a good increase to bring it right up to IIS.
IIS's bouncy results that can't really be fit to a curve concern bit a bit
about IIS. Enterprise under NT, for example, shows a very well fit
increase.