Since SP3 was supposed to be mostly a bug-fix/patch consolidation release - unlike w/Vista SP1, Microsoft made no promises of improved performance for XP - the unexpected speed boost comes as a nice bonus. In fact, XP SP3 is shaping-up to be a "must have" update for the majority of users who are still running Redmond's not-so-latest and greatest desktop OS.

Of course, none of this bodes well for Vista, which is now more than 2x slower than the most current builds of its older sibling. Suffice to say that performance-minded users will likely choose to stick with the now even speedier Windows XP - at least until more "Windows 7" information becomes publicly available.

Sunday, November 18, 2007

Note: We've updated our data set to include results from Vista with 2GB of RAM and also with Office 2003 instead of 2007. Check out our revised numbers here.

With the initial performance characteristics of Windows Vista leaving much to be desired (see our previous post on the subject), many IT organizations have put off deploying the new OS until the first service pack (SP1) is released by Microsoft early next year. The thinking goes that SP1 will address all of these early performance issues and somehow bring Windows Vista on par with - or at least closer to - Windows XP in terms of runtime performance.

Unfortunately, this is simply not the case. Extensive testing by the exo.performance.network (www.xpnet.com) research staff shows that SP1 provides no measurable relief to users saddled with sub-par performance under Vista.

How We Tested

The above conclusion is based on an analysis of the RC0 (v.658) build of Service Pack 1 for Windows Vista. Testing was conducted on a dual-core Dell notebook with 1GB of RAM. The staff ran a variety of test scenarios against both "before" (RTM w/no updates) and "after" (RTM w/SP1 installed) configurations, using the DMS Clarity Studio testing framework to capture scenario scoring and metrics data for upload to the exo.repository.

During office productivity testing, the staff used the DMS Clarity Studio OfficeBench test script to drive Microsoft Office 2007 through a scripted set of productivity tasks, including creating a compound document and supporting workbooks and presentations materials.

To test multitasking performance, the staff used the ADO, MAPI and WMP Stress modules - all part of DMS Clarity Studio - to generate a multi-process workload scenario involving client/server database, workflow and streaming media tasks.

Note: DMS Clarity Studio is available as a free download from the exo.performance.network (www.xpnet.com) site. Simply register for your free DMS Clarity Analysis Portal account to access these and other free tools from xpnet.

Test Results

During OfficeBench testing we noted a statistically insignificant delta (~2%) in favor of the SP1-patched configuration. CPU Saturation, Memory Pressure and I/O Contention factors were all comparable, as were process specific metrics - including the Thread Utilization and Thread Growth Potential Indices.

Figure 1 - OfficeBench Completion Times (Seconds)

The multitasking scenario was also comparable, with the ADO and MAPI Stress workloads showing a delta of less than 1% in favor of the SP1-patched configuration. As with the OfficeBench test scenario, system and process metrics for CPU, Memory and I/O were all nearly identical between the two configurations.

Figure 2 - ADO and MAPI Avg. Transaction Times (Seconds)

Note: For more information on the various system and process metrics employed in this article, please login to your private DMS Clarity Analysis Portal site and refer to the Glossary section of the Online Help

Not yet a member of the exo.performance.network? Sign up today! It's free, and you'll be helping us to build the world's first global repository of computer performance-related knowledge and data.

Conclusions

After extensive testing of both RTM and SP1-patched versions of Windows Vista, it seems clear that the hoped-for performance fixes that Microsoft has been hinting at never materialized. Vista + SP1 is no faster than Vista from the RTM image.

Bottom Line: If you've been disappointed with the performance of Windows Vista to date, get used to it. SP1 is simply not the panacea that many predicted. In the end, it's Vista's architecture - not a lack of tuning or bug fixes - that makes it perform so poorly on systems that were "barn-burners" under Windows XP.