Introduction

This series of articles presents an objective analytical analysis of language performance for Java and C# .NET (2.0, 3.0, and 3.5). The aim of these articles is too end the age old dispute of .Net is better than Java or vica versa. The article analyses performance of standard base mathematical operators common to all languages but specifically mathematical operators that form the basic components of most programs.

Scope of the Analysis

In this article the following functions are tested for performance;

Function

Java Equivalent

C# .Net Equivalent

Add

+

+

Subtract

-

-

Multiply

*

*

Divide

/

/

Power

Math.power(x, y)

Math.Pow(x, y)

Testing Procedure

Key Points

The testing is performed via a console all cases with system load at minimal prior to test start

All system auto-updates are disabled

Task manager is run and set to show cpu usage of all processes ordered by cpu usage descending; if any other process exceeds 5% usage then the test is restarted

The test process is set to a single core affinity (to reduce cpu effects)

All tests are performed 5 times to reduce data noise and increase accuracy

All tests are performed on the same multi-core system (reduces influences of other processes)

Each mathematical function is tested 1000000000 times to reduce inaccuracy in system time calculation and increase accuracy of measures

Effects of looping over 1000000000 times is removed in control case

Test System

CPU

Intel Core 2 Extreme QX6700 - Kentsfield, 2.66Ghz

Motherboard

ASUSTeK - P5WDG2 WS Pro

Memory

DDR2 - 5120MB, Dual channel

Test Code

Control Case

The control case is intended to be used to remove the effects of the loop over the test function. To prevent compiler optimisations from removing the loop the loop performs a simple add function which is used in all function tests (see below). Furthermore the value of the add function is printed at the end, again the prevent compiler optimisations from removing y due to non-use and then the loop itself. The control case is repeated 5 times and the average of all (controlEnd - controlStart) is used as the controlTime. y is reset to 0 at the start of each test.

The test case is kept as similar to the control case as possible, only with the addition of the mathematical function being tested. In this example the add function is being tested, and like the control case the value of x is printed at the end to prevent removal of x due to compiler optimisations. Furthermore comparison of resultant x values may be used to confirm equal functionality between the various languages being tested. The test time is calculated as ((addEnd - addStart) - controlTime).

The other tests are performed as follows;

Function

Code

Add

x = x + 1;

Subtract

x = x - 1;

Multiply

x = x * 1.1;

Divide

x = x / 1.1;

Power

x = Math.Pow(x, 1.1);

Results Analysis

All times are in milliseconds and are averages of the 5 test passes.

Lower values are better.

In these graphs relative performance of functions is shown, and counter to perhaps popular belief is that in all but the power and multiply cases Java actually appears to out-perform C# .Net languages. It is worth noting however that the cases of add and subtract for java are actually lower than 0 indicating that either the control cases have run particularly slowly or perhaps some optimisation has occurred. Since the control cases for Java show good cohesion, it is unlikely to be the former.

Lower values are better.

In this graph the total time for completing the control cases are shown over the 5 passes. It is interesting to note that some form of optimisation has occurred in the .Net languages due to the consistent reduction of overall time from pass 1 to the latter passes; this effect cannot be seen in Java which appears to perform consistently better (perhaps due to the control case being based on the add function itself which according to the results above performs significantly better).

Summary

The control case, i.e. a loop and add function, performs consistently better in Java

.Net performs some optimisation on repeated functions, while Java does not

Java performs better at divide, add, and subtract functions (although further testing is required to confirm)

.Net performs better at multiply and power functions

Other References

Check back soon for other articles on comparison of disk access and interface update performance.

Share

About the Author

I originally studied for a masters in engineering of software engineering at The University of Birmingham during 2000-2004, of which I received a 2:1. I continued at Birmingham University working with Civil Engineering and Rail Research UK where I am currently in my final year of a 3 year PhD project developing a Computational Intelligent Approach to Railway Intervention Planning. Although my work has a significant focus on railway engineering and associated practices much of my work is with data mining (on SQL Server 2008) and computational intelligence (CI) techniques. My key areas of expertise in CI are clustering algorithms (including Rival Penalised Competitive Learning) and evolutionary algorithms.

Outside of my formal work I enjoy testing the latest technologies such as .NET 3.5 and the many frameworks of which it comprises (mainly WPF). I have several projects on the go including a .NET and DirectX port of Quake 3 and many utility libraries. I also maintain an extensive website coded in Cold Fusion which is regularly updated; more information is available about me there.

Yes, that is odd, although I would be interested in hearing a reason other than a rather generic optimisation reason. In the article I suggested:

Derek wrote:

It is worth noting however that the cases of add and subtract for java are actually lower than 0 indicating that either the control cases have run particularly slowly or perhaps some optimisation has occurred.

Reasons aside, that's how Java performs compared to .Net. Strange, but true. Could other people run the demos and let me know the results, do other systems and platforms show the same results?