REALTHINCLIENT SDK TEST RESULTS

RealThinClient SDK v6.05

REST + JSON Server built using the RealThinClient SDK v6.05 (available to developers with a RealThinClient Subscription ) has been stress-tested inside a 100 MBit LAN with 9 Client PCs (1 – 8 + X) and 1 Server. Clients 1 to 8 have a 662 MHz CPU, Client X has a dual-core 2,2 GHz CPU, the Server has a dual-core 2,8 GHz CPU. In this test, all Client PCs have started by opening 10 connections, sending 20.000 requests to the Server from each connection in a single “batch”, then closing all connections and measuring the time it took for each Client to complete the batch (open all connections + send all requests + receive all responses + close all connections). The number of connections was increased every batch by 10 per Client PC and the procedure was repeated.

For comparison reasons (to see how much the CPU on the Server can affect performance), here are the results of the same stress-test
when the Windows Resource Manager (perfmon.exe) is RUNNING on the Server and draining the CPU by 25% – 45% …

RealThinClient SDK v3.76

RealThinClient SDK v3.76 (available to developers with a RealThinClient Subscription ) has been extensively stress-tested using 9.900 concurrent connections over HTTPS with a stress-test running one full week without a break (24/7). There were NO errors throughout the whole stress-test, there were NO problems closing any of the RTC Clients nor the RTC Server and there were absolutely NO memory leaks on either of the Clients nor the Server (FastMM 4.94 used as Memory Manager with memory leak reporting and StreamSec Tools 2.1.9.240 used for SSL encryption).

In this stress-test, the RTC Server has responded to more than 394 million requests from RTC Clients, which have opened and closed more than 2,7 million connections to the Server. I have been changing test parameters on all Clients once every 24 hours, so that most RTC SDK properties and settings have been tested for cca 24 hours each.

Here is one Screenshot of a RTC Server showing how the RealThinClient SDK v3.76 is being stress-tested by simulating 9.900 concurrent clients, where each Client is sending a remote function call to the Server, waiting for the result and then sending the next call in a continuous loop, flooding the Server with requests (image shrinked here to reduce file size):

What you see above are 11 Client PCs, each running 3 instances of the AppClient demo, each AppClient instance using 64 threads in MultiThreaded mode to send remote function calls through 300 concurrent connections to the Server (11 x 3 x 300 = 9.900 connections). The Server is running in MultiThreaded mode and using 64 threads to handle all connections simultaneously.

Each vertical line on each AppClient instance represents a connection to the Server, where the red color shows the percentage of calls to be made in the current loop and the green/cyan color shows the percentage of results received in the current loop. AppClients were all working in multi-threaded mode and were set up to use different settings, while every connection is set up to make 50 remote calls to the Server per loop, and each loop is repeated one second after the last one was finished, which results in an endless loop of remote calls (until stopped).

RealThinClient SDK v2.47 –Extensive Stress-Test

Test running time: ~ 14 Days and 14 Hours (350 Hours and 22 minutes)
Connections in use: 2.000 (10 x 200)
Server: Dual-Core AMD X2 at 4,4 GHz (2 x 2,2 GHz) with 2 GB RAM
Average processing speed: 518,5 req/sec
Average data transfer rate: 33,1 MBit/sec
Total requests processed: 653.960.000
10 Client PCs (200 connections each):
7 x Pentium III at 667 MHz with 256 MB RAM
3 x Pentium III at 650 MHz with 384 MB RAM
– connected to one 24-port 100 MBit switch
The purpose of this test was to see how stable the Clients and the Server are
when generating Extreme Load for a very long time (350 Hours), without a pause.
There were no problems, the test would still be running if I hadn’t stopped it.

For performance tests, please check other test results (below).

Test running time: 1.261.336 seconds (14 days, 14:22 Hours)

CPU
[MHz]

Data
Structure

Data
Format

Auto-
Encrypt

Auto-
Compress

Requests
processed

Bytes/
Request

Bytes/
Response

Req/
sec

Bytes per
second

Memory
in use

Clients with 200 connections active (all at the same time)

667

Simple

RTC

No

No

81.120.000

180

54

64,3

15.046

43 MB

667

Simple

RTC

Yes

No

90.000.000

192

65

71,4

18.350

44 MB

667

Simple

RTC

No

Yes

90.160.000

180

54

71,5

16.731

43 MB

667

Simple

RTC

Yes

Yes

83.880.000

192

65

66,5

17.091

43 MB

650

Simple

XML

No

No

86.760.000

855

176

68,8

70.933

42 MB

667

Complex

RTC

No

No

45.280.000

13.975

2.321

35,9

585.026

47 MB

667

Complex

RTC

Yes

No

54.600.000

13.961

2.330

43,3

705.400

47 MB

667

Complex

RTC

No

Yes

46.240.000

2.432

2.321

36,7

174.435

49 MB

650

Complex

RTC

Yes

Yes

45.280.000

2.443

2.332

35,9

171.423

46 MB

650

Complex

XML

No

No

30.640.000

53.268

52.428

24,3

2.568.413

57 MB

Server with 2.000 connections active

4.400

Both

Both

Yes

Yes

653.960.000

518,5

4.342.848

206 MB

Since all Clients were connected to the same Server during this test, all parameter combinations have been in use at the same time, sending simple and complex data structures to the server, receiving simple and complex results. Because of this and very high network load during the test, this was a good stress-test in a flooded environment, but it should NOT be seen as a performance test, since it is not.

Information provided in the table above is only to show you how load was distributed between clients when they are sending different-size requests. All results should be evaluated as a whole, showing you what would happen if all these clients would be hitting the Server continually for 14 days, without a pause.

Since the RealThinClient SDK supports 12 simple data types (boolean, integer, largeint, float, currency, datetime,string, text, widestring, variable, exteption, byte stream) and 3 complex data types (records, arrays and datasets),
where every element inside a complex data type can be of ANY other data type, including complex data types,the resulting structure used in this stress-test was quite big (14-53 KB). And to test how sending and receiving of all those data types works, the same data was packed again on the Server and sent back to the Client.For performance tests, please check the other test results (below).

Note: If you shold be asking yourself why there’s a difference between the amount of data being sent and received, the explanation is rather simple: request data size is larger than response data size, since requests from the client in this test send information needed to call a few remote functions (function names and all parameters), while response from the server only has to return a single value as a result from those functions.

RealThinClient SDK v1.6 –Tests done by our Beta-Testers and Partners

A number of tests were only possible with help from companies and developers interested in using RealThinClient components for their applications. We would like to THANK everyone who has participated in those tests and made global testing possible.These tests were done using the AppClient demo opening multiple connections and constantly flooding the AppServer with remote function calls. All tests were running constantly for several hours, having processed millions of requests during that period. This kind of testing was very time-consuming, because in each test client and server had to run under maximum load for several hours (most longer than 12 hours).

One hour of such flood-testing is equivalent to one full day of maximum application usage in a very productive environment.

MORE …

A number of smaller tests over the Internet and different LAN’s have also been completed without any problems. You are also welcome to test RTC SDK in your environment. The easiest method of doing so is by using the ready-to-use AppClient and AppServer demos, which you can download here. Feel free to post your test-results to the RTC Newsgroups.