Benchmark

Method

Every benchmark was executed 120 times for each logging framework. The 10 best and 10 worst runs were discarded to avoid that outliers that could distort the whole result. The final result is the average of the remaining 100 runs.

All logging frameworks were configured to write log entries with the level info or higher to a log file. The output of log entries with the level trace and debug was disabled. After each benchmark run, the created log file is automatically checked for completeness and correctness.

The output format is: Date [Thread] Class.Method(): Message

For all logging framework, the default synchronous and - if available - the asynchronous output were tested. Most logging frameworks offer mechanisms for asynchronous output of log entries. tinylog uses a writing thread, Log4j 1.x and Logback use an asynchronous appender and Log4j 2 the promoted asynchronous loggers. If asynchronous logging is activated, log entries are written buffered to the log file.

Maximum logging performance

This benchmark measures how fast each logging framework can output log entries. The benchmark just creates log entries with the levels error, warning, info, debug and trace in a loop one million times.

tinylogwith writing thread

3 s 144 ms

1 s 693 ms

JUL

7 s 73 ms

Log4j 1.xwith AsyncAppender

9 s 233 ms

5 s 449 ms

Log4j 2with AsyncLogger

11 s 966 ms

5.868 ms

Logbackwith AsyncAppender

12 s 371 ms

7 s 60 ms

Influence on compute-intensive application

In practice, the maximum logging performance is probably less important than the influence of logging on the performance of the actual application. Thus, the latter is tested in a second benchmark that calculates all prime numbers from 2 to 10,000,000. The calculation is done in 16 threads to use all cores at full capacity. All found prime numbers are logged as info log entries and all others are dropped as trace log entries.

tinylogwith writing thread

2 s 830 ms

1 s 603 ms

Log4j 2with AsyncLogger

5 s 435 ms

4 s 99 ms

JUL

11 s 623 ms

Log4j 1.xwith AsyncAppender

15 s 250 ms

9 s 059 ms

Logbackwith AsyncAppender

18 s 622 ms

3 s 637 ms

Conclusion

In comparison to other logging frameworks, tinylog is the fastest in both benchmarks. The logging performance can be further improved by using asynchronous and buffered output of log entries. On the other hand, this has the disadvantage that possibly the last - and thus the most important - log entries are lost after a crash of the JVM.

The benchmark program is available on GitHub as open source for the reproducing the results or adaption of the benchmarks to own requirements. The benchmarks are parameterized, making it possible to configure the output format of log entries or the stack trace depth, for example.