EDIT: funny how nivia said 3d mark is usuless for testing graphics performance because of bla bla bla when GF FX 5800 was out, they even droped out of the beta program and wrote articals against fruture mark just because their products are inferior in it compared to the readon. They must have realized that gamers do look at the benchmark and as a result they cheated!

Well you probably all have witnessed the release of 3DMark03 patch 3.3.0 which disables all identified alterations Nvidia Detonator 43.51 and 44.03 introduced. For all of you interested in the details of the 'cheats' click read more.
Futuremark identified an unusual 8.2% difference with Catalyst 3.4 on a Radeon 9800 Pro. Investigation continues...

--------------------------------------------------------------------------------
Aren’t These Cheats Just Optimizations That Also Benefit General Game Play Performance?
No. There are two reasons.
Firstly, these driver cheats increase benchmark performance at the expense of image quality. Only the user and the game developer should decide how a game is meant to be experienced, and not the hardware developer. An act by hardware developer to force a different experience than the developer or the user intended, is an act that may mislead consumers, the OEMs and the media who look to our benchmark to help them make purchase decisions.

Secondly, in well-designed benchmarks like 3DMark03, all cards are instructed to do the same amount of work. Artificially reducing one card’s workload, for example, by using pre-set clip planes or using a lower precision shader against the program’s instructions, is only aimed to artificially manipulate the benchmark test result. Please note, that the cheating described here is totally different from optimization. Optimizing the driver code to increase efficiency is a technique often used to enhance game performance and carries greater legitimacy, since the rendered image is exactly what the developer intended.

1. The loading screen of the 3DMark03 test is detected by the driver. This is used by the driver to disregard the back buffer clear command that 3DMark03 gives. This incorrectly reduces the workload. However, if the loading screen is rendered in a different manner, the driver seems to fail to detect 3DMark03, and performs the back buffer clear command as instructed.
2. A vertex shader used in game test 2 (P_Pointsprite.vsh) is detected by the driver. In this case the driver uses instructions contained in the driver to determine when to obey the back buffer clear command and when not to. If the back buffer would not be cleared at all in game test 2, the stars in the view of outer space in some cameras would appear smeared as have been reported in the articles mentioned earlier. Back buffer clearing is turned off and on again so that the back buffer is cleared only when the default benchmark cameras show outer space. In free camera mode one can keep the camera outside the spaceship through the entire test, and see how the sky smearing is turned on and off.

3. A vertex shader used in game test 4 (M_HDRsky.vsh) is detected. In this case the driver adds two static clipping planes to reduce the workload. The clipping planes are placed so that the sky is cut out just beyond what is visible in the default camera angles. Again, using the free camera one can look at the sky to see it abruptly cut off. Screenshot of this view was also reported in the ExtremeTech and Beyond3D articles. This cheat was introduced in the 43.51 drivers as far as we know.

4. In game test 4, the water pixel shader (M_Water.psh) is detected. The driver uses this detection to artificially achieve a large performance boost - more than doubling the early frame rate on some systems. In our inspection we noticed a difference in the rendering when compared either to the DirectX reference rasterizer or to those of other hardware. It appears the water shader is being totally discarded and replaced with an alternative more efficient shader implemented in the drivers themselves. The drivers produce a similar looking rendering, but not an identical one.

5. In game test 4 there is detection of a pixel shader (m_HDRSky.psh). Again it appears the shader is being totally discarded and replaced with an alternative more efficient shader in a similar fashion to the water pixel shader above. The rendering looks similar, but it is not identical.

6. A vertex shader (G_MetalCubeLit.vsh) is detected in game test 1. Preventing this detection proved to reduce the frame rate with these drivers, but we have not yet determined the cause.

7. A vertex shader in game test 3 (G_PaintBaked.vsh) is detected, and preventing this detection drops the scores with these drivers. This cheat causes the back buffer clearing to be disregarded; we are not yet aware of any other cheats.

8. The vertex and pixel shaders used in the 3DMark03 feature tests are also detected by the driver. When we prevented this detection, the performance dropped by more than a factor of two in the 2.0 pixel shader test.

We have used various techniques to prevent NVIDIA drivers from performing the above detections. We have been extremely careful to ensure that none of the changes we have introduced causes differences in either rendering output or performance. In most case, simple alterations in the shader code – such as swapping two registers – has been sufficient to prevent the detection.

What Is the Performance Difference Due to These Cheats?

A test system with GeForceFX 5900 Ultra and the 44.03 drivers gets 5806 3DMarks with 3DMark03 build 320.

The new build 330 of 3DMark03 in which 44.03 drivers cannot identify 3DMark03 or the tests in that build gets 4679 3DMarks – a 24.1% drop.

Our investigations reveal that some drivers from ATI also produce a slightly lower total score on this new build of 3DMark03. The drop in performance on the same test system with a Radeon 9800 Pro using the Catalyst 3.4 drivers is 1.9%. This performance drop is almost entirely due to 8.2% difference in the game test 4 result, which means that the test was also detected and somehow altered by the ATI drivers. We are currently investigating this further.

The 1.9% performance gain comes from optimization of the two DX9 shaders (water and sky) in Game Test 4 . We render the scene exactly as intended by Futuremark, in full-precision floating point. Our shaders are mathematically and functionally identical to Futuremark's and there are no visual artifacts; we simply shuffle instructions to take advantage of our architecture. These are exactly the sort of optimizations that work in games to improve frame rates without reducing image quality and as such, are a realistic approach to a benchmark intended to measure in-game performance. However, we recognize that these can be used by some people to call into question the legitimacy of benchmark results, and so we are removing them from our driver as soon as is physically possible. We expect them to be gone by the next release of CATALYST.

Originally posted by AlphaWolf None of them are saints, ATI got cought on an even more blatant cheat last year, they fixed the drivers so that when quake3 was run, it would drop the texture detail.

I'm starting do develop a half-peeve on this one.

It wasn't intentional.

The original optimisation was for the old Radeon R6 cards, and worked great.

When they carried that optimisation to R200, hoping for a quick and easy performance boost, they didn't have time to test it visually on R200 before sending out the drivers (R200 shipped with early beta drivers)... turned out the same optimisation needed tweaking on R200 - it didn't work as it should; it wound up causing the core to misinterpret the LOD requests.

We've had a pretty hefty (4 page, 65 replies) discussion on this at Beyond3D, linked here if you want to read through the bulk of it.

I'd hardly call it blatant "cheating" though anyway, since the very next driver set had none of the anomalies (EVER), and had higher performance as well. And if you think IHV's never optimise their drivers to avoid performance hiccups in popular games, you are very horribly wrong.

-Tagrineth, flying above at the speed of an African swallow, because European swallows are wimps!

'What happen ?' 'Somebody set up us the bomb.' 'We get signal.' 'What !' 'Main screen turn on.' 'Its you !!' 'How are you Gentlemen?' 'All your base are belong to us.' 'You are on the way to destruction.' 'What you say !!' 'You have no chance to survive make your time.' 'Ha ha ha ha ....' 'Captain !!' 'Take off every 'ZIG' !!' 'You know what you doing.' 'Move 'ZIG'.' 'For great justice.'

"Since NVIDIA is not part in the FutureMark beta program (a program which costs of hundreds of thousands of dollars to participate in) we do not get a chance to work with Futuremark on writing the shaders like we would with a real applications developer. We don't know what they did but it looks like they have intentionally tried to create a scenario that makes our products look bad. This is obvious since our relative performance on games like Unreal Tournament 2003 and Doom3 shows that The GeForce FX 5900 is by far the fastest graphics on the market today."

"Since NVIDIA is not part in the FutureMark beta program (a program which costs of hundreds of thousands of dollars to participate in) we do not get a chance to work with Futuremark on writing the shaders like we would with a real applications developer. We don't know what they did but it looks like they have intentionally tried to create a scenario that makes our products look bad. This is obvious since our relative performance on games like Unreal Tournament 2003 and Doom3 shows that The GeForce FX 5900 is by far the fastest graphics on the market today."

More FUD, by the way.

nVidia is basically comparing their Pixel Shaders 2.0 and 1.4 (floating-point) performance with pre-floating point games (don't even try to tell me that DOOM3's NV3x path is using FP. Carmack never said so, and with the performance it's getting compared to the FP ARB2 path, I highly doubt it's going past PS 1.1)

-Tagrineth, flying above at the speed of an African swallow, because European swallows are wimps!

'What happen ?' 'Somebody set up us the bomb.' 'We get signal.' 'What !' 'Main screen turn on.' 'Its you !!' 'How are you Gentlemen?' 'All your base are belong to us.' 'You are on the way to destruction.' 'What you say !!' 'You have no chance to survive make your time.' 'Ha ha ha ha ....' 'Captain !!' 'Take off every 'ZIG' !!' 'You know what you doing.' 'Move 'ZIG'.' 'For great justice.'

Originally posted by Tagrineth
I'd hardly call it blatant "cheating" though anyway, since the very next driver set had none of the anomalies (EVER), and had higher performance as well. And if you think IHV's never optimise their drivers to avoid performance hiccups in popular games, you are very horribly wrong.

Then how come when toms hardware renamed it to quaff3, the driver wouldn't drop the image quality? LOD whatever request or not, the driver only cared if the app that was running was named quake3.

Originally posted by AlphaWolf Then how come when toms hardware renamed it to quaff3, the driver wouldn't drop the image quality? LOD whatever request or not, the driver only cared if the app that was running was named quake3.

And keep in mind that at the time Quake3 was a very popular game to play (and it still is, to an extent).

Then there's also the fact that a few revisions later, other Q3-engine games had similar improvements out of nowhere.

Back on track with a proper rebuttal:

Yes, I know it did drop the image quality though.... but as I said:

It did NOT drop any quality at all on R6-based Radeons, which was the target for the optimisation anyway. It was the same optimisation, initially, and it was thrown onto R200 on the assumption that it would have the same results - which it didn't, much to the chagrin of ATi. It was fixed immediately, and I'm sure ATi would've fixed it themselves even if nVidia themselves hadn't provided the info to review sites.

Yes, nVidia pointed out their opponent's 'cheating'. Just like ATi never pointed any fingers at nV for the current 3DMark03 hassle.

-Tagrineth, flying above at the speed of an African swallow, because European swallows are wimps!

'What happen ?' 'Somebody set up us the bomb.' 'We get signal.' 'What !' 'Main screen turn on.' 'Its you !!' 'How are you Gentlemen?' 'All your base are belong to us.' 'You are on the way to destruction.' 'What you say !!' 'You have no chance to survive make your time.' 'Ha ha ha ha ....' 'Captain !!' 'Take off every 'ZIG' !!' 'You know what you doing.' 'Move 'ZIG'.' 'For great justice.'