Carl and I grabbed an interview with Dino Esposito in a quiet room during the conference,
his viewpoint on Silverlight and ASP.NET technologies is always interesting.

Dino's session on "What Partial Rendering is not AJAX" rang true for me as well -
his point is that the essence of AJAX is pushing page rendering to the browser, rather
than computing it on the server. But partial rendering still computes the HTML on
the server and sends it to the browser to display. This undermines the goal of AJAX.

I had last session of the day (and conference) and a huge crowd for my load testing
talk today, as usual there were relatively few folks in the audience that had done
load testing before, so a lot of my talk focused on the fundamentals of why and where
for load testing. The data we've gathered around Strangeloop is great stuff for getting
people started.

DevReach Day Twohttp://www.campbellassociates.ca/blog/PermaLink,guid,ea779f7e-4d25-43d5-9b07-a59e94cd607b.aspxhttp://www.campbellassociates.ca/blog/PermaLink,guid,ea779f7e-4d25-43d5-9b07-a59e94cd607b.aspx
Tue, 02 Oct 2007 21:41:14 GMT<p>
Carl and I grabbed an interview with Dino Esposito in a quiet room during the conference,
his viewpoint on Silverlight and ASP.NET technologies is always interesting.
</p>
<p>
Dino's session on "What Partial Rendering is not AJAX" rang true for me as well -
his point is that the essence of AJAX is pushing page rendering to the browser, rather
than computing it on the server. But partial rendering still computes the HTML on
the server and sends it to the browser to display. This undermines the goal of AJAX.
</p>
<p>
I had last session of the day (and conference) and a huge crowd for my load testing
talk today, as usual there were relatively few folks in the audience that had done
load testing before, so a lot of my talk focused on the fundamentals of why and where
for load testing. The data we've gathered around Strangeloop is great stuff for getting
people started.
</p>
<img width="0" height="0" src="http://www.campbellassociates.ca/blog/aggbug.ashx?id=ea779f7e-4d25-43d5-9b07-a59e94cd607b" />http://www.campbellassociates.ca/blog/CommentView,guid,ea779f7e-4d25-43d5-9b07-a59e94cd607b.aspx.NET Rocks!ASP.NETPodCastingSpeakingTestingTravelhttp://www.campbellassociates.ca/blog/Trackback.aspx?guid=1458f2e8-afc8-4152-bcb3-df576f63a16ehttp://www.campbellassociates.ca/blog/pingback.aspxhttp://www.campbellassociates.ca/blog/PermaLink,guid,1458f2e8-afc8-4152-bcb3-df576f63a16e.aspxhttp://www.campbellassociates.ca/blog/CommentView,guid,1458f2e8-afc8-4152-bcb3-df576f63a16e.aspxhttp://www.campbellassociates.ca/blog/SyndicationService.asmx/GetEntryCommentsRss?guid=1458f2e8-afc8-4152-bcb3-df576f63a16eFailing From Your Own Successhttp://www.campbellassociates.ca/blog/PermaLink,guid,1458f2e8-afc8-4152-bcb3-df576f63a16e.aspxhttp://www.campbellassociates.ca/blog/PermaLink,guid,1458f2e8-afc8-4152-bcb3-df576f63a16e.aspx
Fri, 11 May 2007 17:25:52 GMT<p class=MsoNormal style="MARGIN: 0cm 0cm 0pt">
<span lang=EN-US><font color=#000000>While I’m on the subject of load testing, this
is the horror story I hear over and over again from developers: The site is ready,
it looks great, the client loves it. But you’ve focused all your energy on features
and on getting the thing out the door, and almost no energy thinking about how it’s
actually going to run under load. So almost invariably, it’s not until ship date,
when your investor or whoever’s paying for the site sends the link to everybody he
knows, and says “Look guys, I’ve succeeded!” that the site finally runs under load.
And it’s a disaster. It’s like one developer I know who built and tested a site completely
internally, and it wasn’t until the day it went live that he realized that the average
page size was 1.5MB. Or that no one actually tested the transactional integrity of
the web pages with the database, and the first time two people try to enter data at
the same time, bad things happen. </font></span>
</p>
<p class=MsoNormal style="MARGIN: 0cm 0cm 0pt">
<span lang=EN-US><?xml:namespace prefix = o ns = "urn:schemas-microsoft-com:office:office" />
<o:p>
<font color=#000000>&nbsp;</font>
</o:p>
</span>
</p>
<p class=MsoNormal style="MARGIN: 0cm 0cm 0pt">
<span lang=EN-US><font color=#000000>So this is an important question to ponder: What
does it look like when your site fails under load? It’s not like there’s an explosion,
or a sign that pops up out of the server that says “Help!” Failure doesn’t look like
anything in particular. It’s an inelegant thing, and it’s an inconsistent thing. You
see all kinds of bizarre messaging, usually for stuff that’s unrepeatable. You get
errors in code that doesn’t actually have errors (no wonder you can’t reproduce them!).
It’s just that the environment you’re living in under load is so different. People
can waste a lot of time pursuing those errors. You can make yourself completely crazy
chasing down these phantom “bugs,” not recognizing that they’re just symptoms of an
overloaded web server. </font></span>
</p>
<p class=MsoNormal style="MARGIN: 0cm 0cm 0pt">
<span lang=EN-US>
<o:p>
<font color=#000000>&nbsp;</font>
</o:p>
</span>
</p>
<p class=MsoNormal style="MARGIN: 0cm 0cm 0pt">
<span lang=EN-US><font color=#000000>Of course, even a great load tester can’t <i style="mso-bidi-font-style: normal">really</i> show
you what failure looks like. You get a report showing how long it took a page to render,
but what does that actually look like? What pieces came up and what didn’t? What did
the error message look like? Those kinds of things are very tough for a load tester
to capture.</font></span>
</p>
<p class=MsoNormal style="MARGIN: 0cm 0cm 0pt">
<span lang=EN-US>
<o:p>
<font color=#000000>&nbsp;</font>
</o:p>
</span>
</p>
<p class=MsoNormal style="MARGIN: 0cm 0cm 0pt">
<font color=#000000><span lang=EN-US>That’s why Sean Wilson, our QA manager at Strangeloop,
recommends that even when you’re running a load test, you actually go in and use the
site yourself during the test. You may be the only “real” person on the site, but
you’re experiencing it as if there’s another 10,000 people using it. It’s worthwhile
to capture that human viewpoint. The impact of a 120-second response time is nowhere
near as significant when your Spirent Avalanche reports it as it is when you’re actually
sitting there watching the thing freeze for two minutes and the only thing on the
page is a partially drawn banner. Or worse yet, a partially rendered form that a user
may <i style="mso-bidi-font-style: normal">think</i></span><span style="mso-ansi-language: EN-CA"> they
can start working with, only to find a minute or so later that it goes nuts.<o:p></o:p>
</span></font>
</p>
<p class=MsoNormal style="MARGIN: 0cm 0cm 0pt">
<span lang=EN-US>
<o:p>
<font color=#000000>&nbsp;</font>
</o:p>
</span>
</p>
<p class=MsoNormal style="MARGIN: 0cm 0cm 0pt">
<span lang=EN-US><font color=#000000>Which brings me to the next point to ponder:
Fail gracefully. As developers, we tend to think that the correct answer is always,
“No bugs, we’ll just fix everything.” But the reality is that you’ll never do it.
It will never, ever completely go away. So put the cycles into dealing with failure
well. </font></span>
</p>
<p class=MsoNormal style="MARGIN: 0cm 0cm 0pt">
<span lang=EN-US>
<o:p>
<font color=#000000>&nbsp;</font>
</o:p>
</span>
</p>
<p class=MsoNormal style="MARGIN: 0cm 0cm 0pt">
<span lang=EN-US><font color=#000000>The basic definition of a graceful failure is
a failure where everybody knows what happened. Or at least, you’ve controlled the
message explaining what happened. Take the default IIS “Server Too Busy” message.
It’s not a pretty message, but it gets the job done. It conveys the information. It’s
not some weird ASP.NET error that just makes your customer angry at your incompetence,
and sends you off trying to debug something that can’t be debugged. The customer may
still be annoyed that you weren’t adequately prepared for your bandwidth demands,
but at least they know what happened. And you can go from there.</font></span>
</p>
<p class=MsoNormal style="MARGIN: 0cm 0cm 0pt">
<span lang=EN-US>
<o:p>
<font color=#000000>&nbsp;</font>
</o:p>
</span>
</p>
<p class=MsoNormal style="MARGIN: 0cm 0cm 0pt">
<span lang=EN-US><font color=#000000>Obviously, you want to provide some more content
than the default messages generated from IIS or from ASP.NET. At least a better apology.
Give the customer a sense that this shouldn’t have happened, we’re sorry about it,
we have a record of it happening, and we’ll deal with it. Ideally, you don’t want
to need any message because everything’s handled. But that’s an ideal, and should
be treated as such. You have to have something in between.</font></span>
</p>
<p class=MsoNormal style="MARGIN: 0cm 0cm 0pt">
<span lang=EN-US>
<o:p>
<font color=#000000>&nbsp;</font>
</o:p>
</span>
</p>
<p class=MsoNormal style="MARGIN: 0cm 0cm 0pt">
<span lang=EN-US><font color=#000000>Of course, nobody likes thinking about failure,
but this isn’t about failure. It’s about success. These failures under load are the
kinds of things that happen to successful sites. With the Web in resurgence again,
with ASP.NET becoming more and more popular, and with sites getting bigger and bigger,
the load is only going up from here. You should be planning on being successful, and
that means thinking about these things. If you haven’t thought about them, they’re
going to think about you. </font></span>
</p>
<p class=MsoNormal style="MARGIN: 0cm 0cm 0pt">
<span lang=EN-US>
<o:p>
<font color=#000000>&nbsp;</font>
</o:p>
</span>
</p>
<p class=MsoNormal style="MARGIN: 0cm 0cm 0pt">
<span lang=EN-US>
<o:p>
<font color=#000000>&nbsp;</font>
</o:p>
</span>
</p>
<p>
<font color=#000000></font>
</p>
<img width="0" height="0" src="http://www.campbellassociates.ca/blog/aggbug.ashx?id=1458f2e8-afc8-4152-bcb3-df576f63a16e" />http://www.campbellassociates.ca/blog/CommentView,guid,1458f2e8-afc8-4152-bcb3-df576f63a16e.aspxASP.NETDevelopmentStrangeloopTestinghttp://www.campbellassociates.ca/blog/Trackback.aspx?guid=1097b5f2-c5e1-4cbe-b7c1-59b473c50076http://www.campbellassociates.ca/blog/pingback.aspxhttp://www.campbellassociates.ca/blog/PermaLink,guid,1097b5f2-c5e1-4cbe-b7c1-59b473c50076.aspxhttp://www.campbellassociates.ca/blog/CommentView,guid,1097b5f2-c5e1-4cbe-b7c1-59b473c50076.aspxhttp://www.campbellassociates.ca/blog/SyndicationService.asmx/GetEntryCommentsRss?guid=1097b5f2-c5e1-4cbe-b7c1-59b473c50076Peeking Over the Fence into the Networking Guys' Backyard Reveals a Brilliant Load Testing Solutionhttp://www.campbellassociates.ca/blog/PermaLink,guid,1097b5f2-c5e1-4cbe-b7c1-59b473c50076.aspxhttp://www.campbellassociates.ca/blog/PermaLink,guid,1097b5f2-c5e1-4cbe-b7c1-59b473c50076.aspx
Sat, 05 May 2007 00:40:47 GMT<p class=MsoNormal style="MARGIN: 0cm 0cm 0pt">
<span lang=EN-US><font color=#000000>We’ve been going through beta testing at <a href="http://www.strangeloopnetworks.com/">Strangeloop</a><?xml:namespace prefix = o ns = "urn:schemas-microsoft-com:office:office" />,
which means I’ve had the chance to do some serious scaling of ASP.NET. One of the
interesting experiences that keeps coming up in this process is the reaction we get
from customers when we’re helping them do load testing.
<o:p></o:p>
</font></span>
</p>
<p class=MsoNormal style="MARGIN: 0cm 0cm 0pt">
<span lang=EN-US>
<o:p>
<font color=#000000>&nbsp;</font>
</o:p>
</span>
</p>
<p class=MsoNormal style="MARGIN: 0cm 0cm 0pt">
<span lang=EN-US><font color=#000000>One of the things we can offer our early beta
test customers is the opportunity to load test their site, with and without Mobius
in the loop. We need the test data anyway, and quite a few candidates don’t really
have much in the way of load testing resources ready to go. And then we test their
site in our lab with our <a href="http://www.spirent.com/analysis/technology.cfm?media=7&amp;WS=325&amp;SS=109&amp;wt=2">Spirent
Avalanche</a>, and they go “Wow! I need one of those!”<o:p></o:p>
</font></span>
</p>
<p class=MsoNormal style="MARGIN: 0cm 0cm 0pt">
<span lang=EN-US>
<o:p>
<font color=#000000>&nbsp;</font>
</o:p>
</span>
</p>
<p class=MsoNormal style="MARGIN: 0cm 0cm 0pt">
<span lang=EN-US><font color=#000000>So what’s a <a href="http://www.spirent.com/analysis/technology.cfm?media=7&amp;WS=325&amp;SS=109&amp;wt=2">Spirent
Avalanche</a>, you ask? Funny you should ask… It’s 3Us of load testing love.<o:p></o:p>
</font></span>
</p>
<p class=MsoNormal style="MARGIN: 0cm 0cm 0pt">
<span lang=EN-US>
<o:p>
<font color=#000000>&nbsp;</font>
</o:p>
</span>
</p>
<p class=MsoNormal style="MARGIN: 0cm 0cm 0pt">
<span lang=EN-US><font color=#000000>Josh Bixby, our VP of product development, noticed
it when he was at trade shows. One of the benefits of having our feet in both the
development camp and the networking camp is that we naturally see things on the network
side that a lot of developers don’t. Josh pointed out that virtually every company
making networking appliances had one of these 3U boxes in their demo racks. But I’d
never heard of it before. So we checked it out, and realized it was the best answer
I’ve ever seen to doing load testing. I know that load testing isn’t something people
want to think about unless they HAVE to think about it. But if you do have to think
about it, you have to check this out.
<o:p></o:p>
</font></span>
</p>
<p class=MsoNormal style="MARGIN: 0cm 0cm 0pt">
<span lang=EN-US>
<o:p>
<font color=#000000>&nbsp;</font>
</o:p>
</span>
</p>
<p class=MsoNormal style="MARGIN: 0cm 0cm 0pt">
<span lang=EN-US><font color=#000000>I don’t need to emphasize how much of a pain
load testing is. Typically, you have two options, both of which suck: If you’re doing
it yourself, you may be spending literally a week setting up a load test farm, and
you’re probably spending more energy making the configuration work than actually doing
the test. Which is no surprise, since most likely you’re using any piece of junk you
can find, trying to network together a bunch of machines with different NICs, different
performance, different speeds, etc., before you even begin to configure the test.
I had one customer that bought me ten identical, dedicated servers for load testing
- for about the same cost as an Avalanche - but that’s the exception, not the rule.
And it still gives you much less control, you have to do all your own analytics, etc.<o:p></o:p>
</font></span>
</p>
<p class=MsoNormal style="MARGIN: 0cm 0cm 0pt">
<span lang=EN-US>
<o:p>
<font color=#000000>&nbsp;</font>
</o:p>
</span>
</p>
<p class=MsoNormal style="MARGIN: 0cm 0cm 0pt">
<span lang=EN-US><font color=#000000>It’s easy to think “Oh, I’ll just use <a href="http://www.mercury.com/us/">Mercury
Interactive</a> (sorry, <a href="http://h71028.www7.hp.com/enterprise/cache/447066-0-0-0-121.html?rd=mercury">HP
Mercury</a>) to do my load testing.” Easy until you see the price. Paying six digits
for load testing with a 20% annual maintenance contract isn’t so easy. And that’s
just for software – you still supply the hardware. I don’t think anyone told Mercury
that the Dot Com Boom was over.
<o:p></o:p>
</font></span>
</p>
<p class=MsoNormal style="MARGIN: 0cm 0cm 0pt">
<span lang=EN-US>
<o:p>
<font color=#000000>&nbsp;</font>
</o:p>
</span>
</p>
<p class=MsoNormal style="MARGIN: 0cm 0cm 0pt">
<span lang=EN-US><font color=#000000>So taking a page from the network guys, there’s
a third way to do load testing: You get a Spirent Avalanche, hook it up, and let it
do the job. One 3U box with four gigabit Ethernet ports that can generate nearly two
million users by itself. So you’ve got the hardware and the software all in one box.<o:p></o:p>
</font></span>
</p>
<p class=MsoNormal style="MARGIN: 0cm 0cm 0pt">
<span lang=EN-US>
<o:p>
<font color=#000000>&nbsp;</font>
</o:p>
</span>
</p>
<p class=MsoNormal style="MARGIN: 0cm 0cm 0pt">
<span lang=EN-US><font color=#000000>Of course, the Avalanche isn’t cheap either,
although they’ve nailed the gradually pregnant business model well – you can rent
the gear, and those rental charges get applied to a purchase. We spent less than $100,000
on our 2700 with all the features we needed to do web testing. It also uses TCL-based
scripting, which is usually the realm of networking guys, not developers, and can
be difficult to understand. TCL provides the Avalanche with the flexibility to do
load testing on a lot more than just web stuff.<o:p></o:p>
</font></span>
</p>
<p class=MsoNormal style="MARGIN: 0cm 0cm 0pt">
<span lang=EN-US>
<o:p>
<font color=#000000>&nbsp;</font>
</o:p>
</span>
</p>
<p class=MsoNormal style="MARGIN: 0cm 0cm 0pt">
<span lang=EN-US><font color=#000000>However, bundled with the Avalanche is a product
called <a href="http://www.sstinc.com/webpro_spirent.html">TracePlus/Web Detective
(Spirent Edition)</a>, made by <a href="http://www.sstinc.com/">System Software Technology
(SST)</a>. SST makes a variety of different TracePlus products for networking and
web, including this version specifically for working with the Avalanche. TracePlus
provides the classic capture mechanisms that you see with most load generating tools,
where the tool captures your navigation of the web pages and captures them as HTTP
commands. The Avalanche internally converts this to its TCL commands.<o:p></o:p>
</font></span>
</p>
<p class=MsoNormal style="MARGIN: 0cm 0cm 0pt">
<span lang=EN-US>
<o:p>
<font color=#000000>&nbsp;</font>
</o:p>
</span>
</p>
<p class=MsoNormal style="MARGIN: 0cm 0cm 0pt">
<span lang=EN-US><font color=#000000>The Avalanche has some ability to do reporting
internally (pretty graphs), but the main way we’re using it is in “Excel mode”, where
it generates CSV files that we can load into spreadsheets for analysis.<o:p></o:p>
</font></span>
</p>
<p class=MsoNormal style="MARGIN: 0cm 0cm 0pt">
<span lang=EN-US>
<o:p>
<font color=#000000>&nbsp;</font>
</o:p>
</span>
</p>
<p class=MsoNormal style="MARGIN: 0cm 0cm 0pt">
<span lang=EN-US><font color=#000000>We’re also finding that the Avalanche doesn’t
understand ASP.NET things like viewstate very well, but then, neither does <a href="http://www.microsoft.com/technet/archive/itsolutions/intranet/downloads/webstres.mspx?mfr=true">WAST</a>.
We’re using <a href="http://msdn2.microsoft.com/en-us/teamsystem/aa718823.aspx">Visual
Studio 2005 Team Edition for Testers</a> to get really smart functional testing around
specific ASP.NET features.<o:p></o:p>
</font></span>
</p>
<p class=MsoNormal style="MARGIN: 0cm 0cm 0pt">
<span lang=EN-US>
<o:p>
<font color=#000000>&nbsp;</font>
</o:p>
</span>
</p>
<p class=MsoNormal style="MARGIN: 0cm 0cm 0pt">
<span lang=EN-US><font color=#000000>Even with these complications, it’s such a better
way to do load testing than setting up servers, and infinitely better than letting
your paying customers do the testing. So if you’re doing load testing, why aren’t
you using one of these? Why don’t more people know about this? This is pretty standard
equipment if you build networking gear. It’s not like the Avalanche is some new, earth-shattering
product. It’s not even mentioned on the main page of Spirent’s Web site?!?</font></span>
</p>
<p class=MsoNormal style="MARGIN: 0cm 0cm 0pt">
<span lang=EN-US>
<o:p>
<font color=#000000>&nbsp;</font>
</o:p>
</span>
</p>
<p class=MsoNormal style="MARGIN: 0cm 0cm 0pt">
<span lang=EN-US><font color=#000000>I have yet to find anyone else in the ASP.NET
world using a Spirent Avalanche. I really think it’s just a cultural issue, where
great stuff is getting lost in translation between the networking world and the Web
development world.
<o:p></o:p>
</font></span>
</p>
<p class=MsoNormal style="MARGIN: 0cm 0cm 0pt">
<span lang=EN-US>
<o:p>
<font color=#000000>&nbsp;</font>
</o:p>
</span>
</p>
<p class=MsoNormal style="MARGIN: 0cm 0cm 0pt">
<span lang=EN-US><font color=#000000>Important lesson: If you’re not paying attention
to the networking space, you should be. You may just be wasting your time wrestling
with a problem that other smart people have already solved. That’s one of the cool
things about working with Strangeloop; we really get to straddle the line between
those two worlds.
<o:p></o:p>
</font></span>
</p>
<p class=MsoNormal style="MARGIN: 0cm 0cm 0pt">
<span lang=EN-US>
<o:p>
<font face="Times New Roman" color=#000000>&nbsp;</font>
</o:p>
</span>
</p>
<p>
</p>
<img width="0" height="0" src="http://www.campbellassociates.ca/blog/aggbug.ashx?id=1097b5f2-c5e1-4cbe-b7c1-59b473c50076" />http://www.campbellassociates.ca/blog/CommentView,guid,1097b5f2-c5e1-4cbe-b7c1-59b473c50076.aspxASP.NETDevelopmentNetwork GearStrangeloopTesting