Log4J 2 is a logging framework designed to address the logging requirements of enterprise applications. If you are new to Log4J2, I suggest going through my introductory post on Log4J 2, Introducing Log4J 2 – Enterprise Class Logging.

Log4J 2 introduces configuration support via JSON and YAML in addition to properties file and XML. I’ve written about the different Log4J 2 configuration options in the following posts:

In this post, we’ll take a look at asynchronous loggers (async loggers) introduced in Log4J 2.

Asynchronous Logging: Introduction

Performance is critical for enterprise applications and nobody wants the underlying logging framework to become a bottleneck. In small programs with little volume, the overhead of logging is rarely an issue. However, enterprise services can see significant volume. If the service is getting invoked hundreds or even thousands of times per second, the overhead of logging can become significant. In such scenarios, two fundamental performance-related concepts are:

Latency: Time required to perform some action or to produce some result. Time of a transaction, or service invocation.

Throughput: The number of some actions executed or results produced per unit of time.

For increased logging performance, we want lower logging latency and higher throughput. The asynchronous logger in Log4J 2 does this by decoupling the logging overhead from the thread executing your code. An async logger has consistently lower latency than a synchronous logger and high throughput of logging messages at 6 – 68 times the rate of a synchronous logger.

I/O operations are notorious performance killers. This is because of locks and waits which are typical when dealing with I/O operations. I/O operations can be executed in a separate thread, thereby freeing the main thread to perform other tasks. With the multicore architectures of modern CPUs, multithreaded operations are an ideal way to improve application performance.

Multi-threaded logging was present prior to Log4J 2 through asynchronous appenders, and its support still exist. The new asynchronous logger differs from asynchronous appender in how work is passed by the main thread to a different thread. Async appender uses an ArrayBlockingQueue – A first-in-first-out (FIFO) queue to hand off the messages to the thread which performs the I/O operations. The ArrayBlockingQueue class internally uses locks to ensure data integrity and data visibility between threads. As locks introduce latency, ArrayBlockingQueue is not the most optimal data structure to pass information between threads. Async logger is designed to optimize this area by replacing the blocking queue with LMAX Disruptor – a lock-free inter-thread communication library. The use of Disruptor results in higher throughput and lower latency in Log4J 2 logging. Martin Fowler has written an excellent article on the architecture of LMAX Disruptor here.

Maven Dependencies

To use async logger in your application, you need to add dependency of LMAX Disruptor in addition to the required Log4J 2 libraries to your Maven POM, like this.

Next, we will use XML to configure Log4J2. The log4j2.xml file is this.

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

<?xml version="1.0"encoding="UTF-8"?>

<Configuration status="debug">

<Appenders>

<Console name="Console-Appender"target="SYSTEM_OUT">

<PatternLayout>

<pattern>

[%-5level]%d{yyyy-MM-dd HH:mm:ss.SSS}[%t]%c{1}-%msg%n

</pattern>>

</PatternLayout>

</Console>

<File name="File-Appender"fileName="logs/xmlfilelog.log">

<PatternLayout>

<pattern>

[%-5level]%d{yyyy-MM-dd HH:mm:ss.SSS}[%t]%c{1}-%msg%n

</pattern>

</PatternLayout>

</File>

</Appenders>

<Loggers>

<Logger name="guru.springframework.blog.log4j2async"level="debug">

<AppenderRef ref="File-Appender"/>he precedingc

</Logger>

<Root level="debug">

<AppenderRef ref="Console-Appender"/>

</Root>

</Loggers>

</Configuration>

In the code above, we added the
status="debug" attribute to the
<configuration> tag to output internal Log4J 2 log messages. This is required to verify that log messages are indeed getting logged asynchronously. We then configured a console and a file appender. We also configured an application-specific logger and the root logger to use the file and console appenders respectively. Notice that we haven’t written any asynchronous logging configuration code as of yet.

All Async Loggers

The simplest way to enable asynchronous logging in Log4J 2 is to make all loggers async. This involves setting the Log4jContextSelector system property. On the command line, you can set it like this.

In the preceding configuration code, the application-specific logger will asynchronously log messages to the file, while the root logger will synchronously log messages to console.
To make the root logger async, use
<AsyncRoot>.

Random Access File Appender

A discussion on asynchronous logging won’t be complete without the mention of the random access file appender. A random access file is similar to the file appender we used, except it’s always buffered with a default buffer size of 256 * 1024 bytes. The buffer size, as of the current release, is not configurable. This means that once the buffer is pre-allocated with a size at first use, it will never grow or shrink during the life of the system. You can override the default size with the AsyncLoggerConfig.RingBufferSize system property. The random access file appender internally uses a ByteBuffer with RandomAccessFile instead of a BufferedOutputStream. This results in significant performance improvement. It is reported to have 20-200% more performance gain as compared to file appender.

Log4J 2 also provides the rolling random access file appender for high performance rolling files. This appender, similar to random access file, is always buffered with the default size of 256 * 1024 bytes, which is not configurable.

I have discussed configuring rolling files here, and also here. To configure a similar rolling random access file appender, replace the
<RollingFile> tag with
<RollingRandomAccessFile>.

Conclusion

In this post, I’ve discussed configuring asynchronous logging in Log4j 2 using the Log4jContextSelector system property (for all async loggers) and through
<AsyncLogger> and
<AsyncRoot> (For mix of sync and async loggers). One common mistakes that programmers make is to mix both of them. Although it works, you will end up with two background threads – an unnecessary thread in the middle that passes a log message from your application to the thread that finally logs the message to disk.

The average Java application will not need the performance benefits of Log4J 2’s asynchronous logging. In many cases, it would simply be overkill. However, Java and the Spring Framework are often used for highly scalable applications processing enormous amounts of information. When you’re developing enterprise class applications, optimal performance does become critical. The option for asynchronous in Log4J 2 is a tool you can use to optimize the performance of your Java and Spring Applications.

Log4J 2 introduces configuration support through JSON and YAML in addition to properties file and XML. If you are new to Log4J2, I suggest going through my introductory post on Log4J 2, Introducing Log4J 2 – Enterprise Class Logging.

For the different Log4J 2 configuration options, you can explore these related posts:

Maven Dependencies for YAML Configuration

To use Log4J2, you need to add the required Log4J 2 dependencies in your Maven POM, as described here. For YAML configuration, you additionally need Jackson, a suite of data-processing tools for Java. The YAML support for Log4J 2 uses two Jackson packages: Jackson data format and Jackson data bind whose dependencies must be present in your Maven POM.

The following code snippet shows the dependencies required to use YAML to configure Log4J 2.

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

<dependency>

<groupId>org.springframework.boot</groupId>

<artifactId>spring-boot-starter</artifactId>

<exclusions>

<exclusion>

<groupId>org.springframework.boot</groupId>

<artifactId>spring-boot-starter-logging</artifactId>

</exclusion>

</exclusions>

</dependency>

<dependency>

<groupId>org.apache.logging.log4j</groupId>

<artifactId>log4j-api</artifactId>

<version>2.5</version>

</dependency>

<dependency>

<groupId>org.apache.logging.log4j</groupId>

<artifactId>log4j-core</artifactId>

<version>2.5</version>

</dependency>

<dependency>

<groupId>com.fasterxml.jackson.dataformat</groupId>

<artifactId>jackson-dataformat-yaml</artifactId>

<version>2.7.3</version>

</dependency>

<dependency>

<groupId>com.fasterxml.jackson.core</groupId>

<artifactId>jackson-databind</artifactId>

<version>2.5.4</version>

</dependency>

...

Creating a Logger

Before we start configuring Log4J 2 using YAML, lets create a logger class that uses the Log4J 2 API to log messages.

Configuring Console and File Appenders

In order to configure Log4J 2 using YAML, you need a YAML configuration file, named either log4j2.yaml or log4j2.ym in the project classpath.

The skeleton of a YAML configuration file is this.

YAML

1

2

3

4

Configuration:

Properties:

Appenders:

Loggers:

The syntax of the log4j2.yaml file above is composed of a
Configuration key with a list of properties:
Properties,
Appenders, and
Loggers.

Let’s start by configuring two appenders to write log messages to the console and a file. We will also configure an application-specific logger and the root logger to use the appenders, like this.

YAML

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

Configutation:

name: Default

Properties:

Property:

name: log-path

value: "logs"

Appenders:

Console:

name: Console_Appender

target: SYSTEM_OUT

PatternLayout:

pattern: "[%-5level] %d{yyyy-MM-ddHH:mm:ss.SSS}[%t]%c{1}-%msg%n"

File:

name: File_Appender

fileName: ${log-path}/logfile.log

PatternLayout:

pattern: "[%-5level]%d{yyyy-MM-ddHH:mm:ss.SSS}[%t]%c{1}-%msg%n"

Loggers:

Root:

level: debug

AppenderRef:

- ref: Console_Appender

Logger:

- name: guru.springframework.blog.log4j2yaml

level: debug

AppenderRef:

- ref: File_Appender

level: error

In the configuration code above:

Line 4 – Line 7: We declared a
log-path property accessible to the other parts of the configuration code.

Line 9 – Line 21: We configured the
Console and
File appenders.

Line 23 – 35: We configured an application-specific logger for all the logger classes of the
guru.springframework.blog.log4j2yaml package. This logger writes error and higher level log messages to the file appender. We also configured the root logger to log debug and higher level messages to the configured console appender.

If we run the Log4J2YamlConfTest test class, Log4J 2 will generate log messages and send them both to the console and file, as shown in this figure.

Configuring a Rolling File Appender

In my earlier posts on configuring Log4J 2 using XML and JSON, I discussed about the benefits of the rolling file appender and how to configure one. Additionaly, you can refer the Log4J 2 manual to learn more about the rolling file. In YAML, you can configure a rolling file appender, like this.

YAML

1

2

3

4

5

6

7

8

9

10

11

12

13

...

RollingFile:

- name: RollingFile_Appender

fileName: ${log-path}/rollingfile.log

filePattern: "logs/archive/rollingfile.log.%d{yyyy-MM-dd-hh-mm}.gz"

PatternLayout:

pattern: "[%-5level]%d{yyyy-MM-ddHH:mm:ss.SSS}[%t]%c{1}-%msg%n"

Policies:

SizeBasedTriggeringPolicy:

size: 1 KB

DefaultRollOverStrategy:

max: 30

...

In the code above:

Line 3: We used the
name value of
RollingFile to define a name of this appender that loggers can use.

Line 4 – Line 5: We used the
filename and
filePattern values to define the name of the file to write to and the pattern of the file name of the archived log file respectively.

Line 8 -Line 10: We used the
Policies property to define a sized-based triggering policy. For testing purpose, we set the
size:1KB value to roll the log file once its size exceeds 1 KB.

Line 11 – Line 12: We used the
DefaultRolloverStrategy property with the
max:30 value. This instructs Log4J 2 to keep up to 30 rolling files before deleting them.

To use the rolling file appender, add the appender reference to the logger, like this.

YAML

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

...

Loggers:

Root:

level: debug

AppenderRef:

- ref: Console_Appender

Logger:

- name: guru.springframework.blog.log4j2yaml

level: debug

AppenderRef:

- ref: File_Appender

level: error

- ref: RollingFile_Appender

level: debug

...

In Line 15 – Line 16 of the configuration code above, we added a reference to the rolling file appender with the debug level.

On running the Log4J2YamlConfTest test class, a rollingfile.log file is generated in the logs folder with debug and higher level log messages. Now if you run the Log4J2YamlConfTest test class couple of more times till the size of the rollingfile.log file exceeds 1 KB, Log4J 2 creates a .gz archive of the generated rolling file in the archive directory.

Logging Additivity

So far in our example, we have been using additivity to send messages sent to the file appender also to the console appender. You can override this default behavior by setting the additivity property of a logger to false.

The complete code of the log4j2.yaml file without additivity is this.

log4j2.yaml

YAML

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

48

49

50

51

52

Configutation:

name: Default

Properties:

Property:

name: log-path

value: "logs"

Appenders:

Console:

name: Console_Appender

target: SYSTEM_OUT

PatternLayout:

pattern: "[%-5level] %d{yyyy-MM-ddHH:mm:ss.SSS}[%t]%c{1}-%msg%n"

File:

name: File_Appender

fileName: ${log-path}/logfile.log

PatternLayout:

pattern: "[%-5level]%d{yyyy-MM-ddHH:mm:ss.SSS}[%t]%c{1}-%msg%n"

RollingFile:

- name: RollingFile_Appender

fileName: ${log-path}/rollingfile.log

filePattern: "logs/archive/rollingfile.log.%d{yyyy-MM-dd-hh-mm}.gz"

PatternLayout:

pattern: "[%-5level]%d{yyyy-MM-ddHH:mm:ss.SSS}[%t]%c{1}-%msg%n"

Policies:

SizeBasedTriggeringPolicy:

size: 1 KB

DefaultRollOverStrategy:

max: 30

Loggers:

Root:

level: debug

AppenderRef:

- ref: Console_Appender

Logger:

- name: guru.springframework.blog.log4j2yaml

additivity: false

level: debug

AppenderRef:

- ref: Console_Appender

level: info

- ref: File_Appender

level: error

- ref: RollingFile_Appender

level: debug

In Line 47 – Line 48 of the code above, we configured a console appender with the level
info for our logger. We also disabled additivity in Line 44 by adding the
additivity:false value.

Now, when we run the test class, our logger will use the newly configured console appender instead of the one in the root logger. You can run the test class again to check that info and higher log messages are now getting sent to the console, as shown in this figure. You can see the debug level is no longer sent to the console.

For configuring additivity using XML and JSON, refer my earlier posts here, and also here. I also suggest reviewing the Log4J 2 documentation on the subject, where they have some good examples how this works.

Summary

Out of all the configuration options, YAML is the most compact and readable format. However, unlike properties and XML configurations, you need the additional Jackson JARs in your project to parse the YAML configuration file. If you are familiar with the XML or JSON format, you can use one of the several converters to convert your XML or JSON configurations to YAML, and also the other way around. One of the tools you can try is an online converter available here. However, as a note of caution, converters are not always 100% accurate. As a developer, use it for reference.

Maven Dependencies for JSON Configuration

To use Log4J2, you need to add the required Log4J 2 dependencies in your Maven POM, as described here. For JSON configuration, you also need Jackson, a suite of data-processing tools for Java. The JSON support for Log4J 2 uses three Jackson packages: Jackson core, Jackson databind, and Jackson annotations.

The following code snippet shows the Maven POM dependencies required to use JSON for configuring Log4J 2.

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

...

<dependency>

<groupId>org.springframework.boot</groupId>

<artifactId>spring-boot-starter</artifactId>

<exclusions>

<exclusion>

<groupId>org.springframework.boot</groupId>

<artifactId>spring-boot-starter-logging</artifactId>

</exclusion>

</exclusions>

</dependency>

<dependency>

<groupId>org.apache.logging.log4j</groupId>

<artifactId>log4j-api</artifactId>

<version>2.5</version>

</dependency>

<dependency>

<groupId>org.apache.logging.log4j</groupId>

<artifactId>log4j-core</artifactId>

<version>2.5</version>

</dependency>

<dependency>

<groupId>com.fasterxml.jackson.core</groupId>

<artifactId>jackson-core</artifactId>

<version>2.6.3</version>

</dependency>

<dependency>

<groupId>com.fasterxml.jackson.core</groupId>

<artifactId>jackson-databind</artifactId>

<version>2.6.3</version>

</dependency>

<dependency>

<groupId>com.fasterxml.jackson.core</groupId>

<artifactId>jackson-annotations</artifactId>

<version>2.6.3</version>

</dependency>

...

Creating a Logger

Before we start configuring Log4J 2 using JSON, lets create a logger class that uses the Log4J 2 API to log messages.

Log4J2JsonConf.java

Java

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

packageguru.springframework.blog.log4j2json;

importorg.apache.logging.log4j.LogManager;

importorg.apache.logging.log4j.Logger;

publicclassLog4J2JsonConf{

privatestaticLogger logger=LogManager.getLogger();

publicvoidperformSomeTask(){

logger.debug("This is a debug message");

logger.info("This is an info message");

logger.warn("This is a warn message");

logger.error("This is an error message");

logger.fatal("This is a fatal message");

}

}

We will use JUnit to test the preceding class.

Log4J2JsonConfTest.java

Java

1

2

3

4

5

6

7

8

9

10

11

12

13

packageguru.springframework.blog.log4j2json;

importorg.junit.Test;

importstaticorg.junit.Assert.*;

publicclassLog4J2JsonConfTest{

@Test

publicvoidtestPerformSomeTask()throwsException{

Log4J2JsonConf log4J2JsonConf=newLog4J2JsonConf();

log4J2JsonConf.performSomeTask();

}

}

Configuring Console and File Appenders Using JSON

In order to configure Log4J 2 using JSON, you need a JSON configuration file, named either log4j2.json or log4j2.jsn in the project classpath.

The following figure shows the skeleton of a JSON configuration file in the IntelliJ editor.

As shown in the preceding figure, a log4j2.json file is composed of nested JSON objects. At the top is the
configuration object that contains the following objects:

properties: Defines one or more properties as a JSON array of name-value pairs. The properties can be referred by their names from the different parts of the configuration file.

appenders: Configures one or more appenders, such as
Console,
File, and
RollingFile.

Loggers: Configures the root logger represented by
root along with zero or more application-specific loggers, each represented by
logger.

We will configure two appenders to write log messages to the console and a file. We’ll also configure an application-specific logger along with the root logger to use the appenders, like this:

Line 4 – Line 15: We declared two properties as name-value pairs in the
property JSON array.

Line 16 – Line 31: We configured the
Console and
File appenders.

Line 32 – Line 43: We configured an application-specific logger for all the logger classes of the
guru.springframework.blog.log4j2json package. This logger writes error and higher level log messages to the file appender. We also configured the root logger to log debug and higher level messages to the console appender.

If we run the Log4J2JsonConfTest test class, Log4J 2 will generate log messages and send them to both the console and a file, as shown in this figure.

Configuring a Rolling File Appender via JSON

If you use the file appender for writing logs, the size of log file will grow with time. This can have significant consequences in enterprise applications that typically have very large code bases containing a significant amount of logging code. A long running application can easily produce millions and millions of lines of logging information which can cause the files to become very large. Imagine scanning through a log file with 10s of thousands of lines of log statements to find a specific statement. In order to avoid that situation, you should use the rolling file appender.

A rolling file appender supports writing to a file and rolls the file over according to one of your pre-defined policies. For example, you can define a size-based triggering policy that causes a rollover once the file has reached a specified size. You can also define a time-based triggering policy that causes a rollover once the date/time pattern is no longer applies to the active log file. You can refer the Log4J 2 manual to learn more about the rolling file.

Line 3: We used the
name property of
RollingFile to define a name of this appender that loggers can use.

Line 4- Line 5: We used the
fileName and
filePattern properties to define the name of the file to write to and the pattern of the file name of the archived log file respectively.

Line 9 -Line 13: We used
Policies to define a sized-based triggering. For testing purpose, we set the
max property to roll the log file once its size exceeds
1KB for.

Line 14 – Line 16: We used
DefaultRolloverStrategy to instruct Log4J 2 to keep up to 30 rolling files before deleting them.

To use the rolling file appender, add the appender reference to the logger, like this.

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

...

"loggers":{

"logger":{

"name":"guru.springframework.blog.log4j2json",

"level":"debug",

"appender-ref":[

{

"ref":"File-Appender","level":"error"

},

{

"ref":"RollingFile-Appender","level":"debug"

}]

},

"root":{

"level":"debug",

"appender-ref":{"ref":"Console-Appender"}

}

}

...

In Line 11 of the configuration code above, we added a reference to the rolling file appender. Note that we used a JSON array to define the appender references. This is required because, without an array, Log4J 2 will only catch one appender – the last one.

On running the Log4J2JsonConfTest test class, a rollingfile.log file is generated in the logs folder with debug and higher level log messages. Now if you run the Log4J2JsonConfTest test class couple of more times till the size of the rollingfile.log file exceeds 1 KB, Log4J 2 creates a .gz archive of the generated rolling file in the
archive directory.

Logging Additivity

If you have noticed, till now we haven’t used the console appender in our application-specific logger, but log messages are still getting sent to the console. It’s due to additivity. Log messages are getting sent additively to the console by the root logger. You can override this default behavior by setting the
additivity property of a logger to
false.

The complete code of the log4j2.json file with additivity disabled is this:

In Line 54 of the code above, we configured a console appender with the level
info for our logger. We also disabled additivity in Line 52 by adding the
additivity property with a
false value.

Now, when we run the test class, our logger will use the newly configured console appender instead of the one in the root logger. You can run the test class again to check that info and higher log messages are now getting sent to the console, as shown in this figure.

Additivity can be somewhat confusing. I suggest reviewing the Log4J 2 documentation on the subject, where they have some good examples how this works.

Summary

JSON is the natural choice for data-interchange in enterprise applications, particularly Web 2.0 applications. There is no evident performance advantage or disadvantage from logging perspective between the various Log4J2 supported formats: properties file, XML, JSON, and YAML. Many argue from configuration perspective that presence of schemas and associated schema validation, which is undoubtedly huge for enterprises, gives XML the edge. Many others support JSON or YAML as they are not only more compact and readable, as compared to XML, but also faster in transmission because it does not come with the extra baggage of tags.

I suggest, as a developer, you should not get tied to a particular format. You may have your own preference, but every enterprise is different. Some may standardize on a format, some may not. Some development teams may prefer JSON over XML, others will like XML over JSON.

In addition to properties file, Log4J 2 supports configuration through XML, JSON, and YAML. In this post, I discuss how to configure Log4J 2 using XML and explain few advanced configuration options.

Setting up an XML Configuration File

Like any other configuration files, Log4J 2 expects your XML configuration file to be present in the classpath with the name log4j2.xml. Although, you can have your XML configuration file in a different location, some additional steps are required for that. You can either perform them programmatically or through the command line by setting the log4j.configuration system property to point to the file. The code to programmatically set a different configuration file location is below.

1

2

3

4

5

...

LoggerContext context=(LoggerContext)LogManager.getContext(false);

File file=newFile("path/to/log4j2.xml");

context.setConfigLocation(file.toURI());

...

The command to set the log4j.configuration system property through the command line is this.

1

java-Dlog4j.configurationFile=path/to/log4j2.xml MyApp

Note: In enterprise applications, logging configuration can differ drastically between development(staging) and production environments. Therefore, you will commonly find logger configuration in scripts, rather than being hardcoded into the source code. Generally, it would be considered a bad practice to hard code the file name in source code.

Creating a Logger

Before we start configuring Log4J 2, we will create a logger class that uses the Log4J 2 API to log messages. Here, I am assuming that you have added the required Log4J 2 dependencies for the project, as explained in my previous post here.

Configuring Console and File Appenders

Assuming that we have log4j2.xml is in the project classpath, let’s inspect how to use it for Log4J 2 configuration. A log4j2.xml file contains the
<Configuration> root tag. In this root tag, you can set the monitorInterval attribute with a time value in seconds. This attribute tells Log4J 2 to detect changes to the configuration file and reconfigure itself after the monitorInterval value has elapsed since the last check. The
<Configuration> root tag can include a
<Properties> tags to specify one or more properties that can be referred from the different parts of this file. The
<Configuration> root tag also encloses the
<Appenders> and
<Loggers> tags to configure one or more appenders and loggers respectively. The skeleton of a log4j2.xml file is like this.

1

2

3

4

5

6

7

8

9

10

11

12

<?xml version="1.0"encoding="UTF-8"?>

<Configuration monitorInterval="60">

<Properties>

<!—Configure properties here-->

</Properties>

<Appenders>

<!--Configure appenders here-->

</Appenders>

<Loggers>

<!--Configure loggers here-->

</Loggers>

</Configuration>

We will start by configuring two appenders to write log messages to the console and a file. We will also configure an application-specific logger along with the root logger to use the appenders, like this.

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

<?xml version="1.0"encoding="UTF-8"?>

<Configuration monitorInterval="60">

<Properties>

<Propertyname="log-path">logs</Property>

<Propertyname="archive">${log-path}/archive</Property>

</Properties>

<Appenders>

<Console name="Console-Appender"target="SYSTEM_OUT">

<PatternLayout>

<pattern>

[%-5level]%d{yyyy-MM-dd HH:mm:ss.SSS}[%t]%c{1}-%msg%n

</pattern>>

</PatternLayout>

</Console>

<File name="File-Appender"fileName="${log-path}/xmlfilelog.log">

<PatternLayout>

<pattern>

[%-5level]%d{yyyy-MM-dd HH:mm:ss.SSS}[%t]%c{1}-%msg%n

</pattern>

</PatternLayout>

</File>

</Appenders>

<Loggers>

<Logger name="guru.springframework.blog.log4j2xml"level="debug">

<AppenderRef ref="File-Appender"level="error"/>

</Logger>

<Root level="debug">

<AppenderRef ref="Console-Appender"/>

</Root>

</Loggers>

</Configuration>

In the configuration code above:

Line 4 – Line 5 : We declared two properties, named log-path and
archive that will be used by the appenders.

Line 8 – Line 23: We declared the
<Appenders> tag enclosing the
<Console> and
<File> tags for the console and file appenders respectively. The target attribute of the
<Console> tag specifies that log messages will be sent to the system console. The
filename attribute of the
<File> tag specifies the name and location of the log file to which log messages will be sent. Both the console and file appenders use pattern layout and specifies the pattern string using the
<PatternLayout> and
<Pattern> tags respectively. You can learn more about pattern layouts here.

Line 25 – Line 32: We defined two loggers within the
<Loggers> tag. The first logger defined by
<Logger> configures all loggers under the
guru.springframework.blog.log4j2xml package to use the file appender. The second one defined by
<Root> is the root logger configured to use the console appender.

If we run the Log4J2XmlConfTest test class, Log4J 2 will generate log messages and send them to both the console and the file, as shown in this figure.

In the figure above, observe that debug and higher log messages were sent to console because we specified debug level for the root appender. Also, as we specified error for the file appender to be used by all the loggers of the guru.springframework.blog.log4j2xml package, only error and fatal messages got logged to the file.

Configuring a Rolling File Appender

According to Apache, approximately 4% of a typical code base is logging. This can be a significant amount of code in enterprise applications which typically have very large code bases. Imagine scanning through a log file with 10s of thousands of lines of log statements to find a specific statement. A long running application can easily produce millions and millions of lines of logging information. Which can cause the files to become very large.

To address such concerns, Log4J 2 provides a rolling file appender. This appender supports writing to a file and rolls the file over according to one of your pre-defined policies. For example, you can define a size-based triggering policy that causes a rollover once the file has reached a specified size. You can also define a time-based triggering policy that causes a rollover once the date/time pattern is no longer applies to the active log file. You can refer the Log4J 2 manual to learn more about the rolling file appender and its triggering policies.

To configure a rolling file appender, use the
<RollingFile> tag, like this.

Line 3 – Line 4: We used the
filename and
filePattern attributes of
<RollingFile> to define the name of the log file to write to and the pattern of the file name of the archived log file respectively.

Line 5: We used the
pattern attribute of
<Patternlayout> to
define a pattern that specifies when the log file should get rolled for a time-based policy.

Line 6 – Line 9: We used the
<Policies> tag to define two triggering policies: Time-based and sized-based. The time-based policy is set to roll the log file daily while the sized-based policy is set to roll the log file once its size exceeds 30 MB.

Line 10: We used the
<DefaultRollover> tag to instruct Log4J 2 to keep up to 30 rolling files before deleting them.

To use the rolling file appender, add the appender reference to the logger declaration, like this.

1

2

3

4

5

6

...

<Logger name="guru.springframework.blog.log4j2xml"level="debug">

<AppenderRef ref="File-Appender"level="error"/>

<AppenderRef ref="RollingFile-Appender"level="debug"/>

</Logger>

...

To simulate how the rolling file appender works, update the size attribute of
<SizeBasedTriggeringPolicy> to 1 KB, like this.

1

2

3

...

<SizeBasedTriggeringPolicy size="1 KB"/>

...

On running the Log4J2XmlConfTest test class now, a rollingfile.log file gets generated with debug and higher level log messages, as shown in the following figure.

Now run the Log4J2XmlConfTest class couple of more times till the size of the rollingfile.log file exceeds 1 KB. Log4J 2 creates a .gz archive of the generated rolling file in the archive directory.

In Line 37 of the code above, I have configured a console appender with the level info to demonstrate the additivity feature of Log4J 2. I disabled logger additivity in Line 34 by adding the
additivity attribute of
<Logger> with a
false value. Now, when you run the test class, our logger will use the newly configured console appender instead of the one in the root logger. You can run the test class again to check that info and higher log messages are now getting sent to the console.

You can see that the Rolling File Appender has a number of configuration options. This appender is very popular for enterprise use. Often the operations team will have requirements for the log files. In some cases, the enterprise will need to archive the files for compliance reasons. Sometimes the files are consumed by Splunk for indexing. Sometimes, the files are simply deleted. Each application and company will have different requirements.

Summary

XML configuration of Log4J 2 opens a number of configuration options, many of which aren’t available through properties file configuration. Therefore, it’s recommended to use XML configuration in enterprise applications having advanced logging requirements. One particular aspect that is very useful in enterprise applications is the ability to divide a large and complex XML configuration file into multiple files. For example, you can have a main log4j2.xml file that includes two other files, say log4j-appenders.xml to configure appenders and log4j-loggers.xml to configure loggers.

Remember, much more time is spent reading code and configuration files than is spent writing it. At some point in the future, yourself, or someone else, will be reading the source code and configuration files. By logically dividing your Log4J 2 XML configuration files, you improve the long term maintainability of your application. Multiple XML configuration files are not supported out of the box by Log4J 2. You will need to use the XInclude library and use it as described here. It’s fairly easy to do, and large enterprise applications will likely find benefit in using multiple XML configuration files for Log4J 2 configuration properties.

Log4J 2 is a logging framework designed to address the logging requirements of enterprise applications. Its predecessor Log4J 1.x has been around for more than one and a half decade and is still one of the most widely used Java logging framework. Log4J has even been ported to the .NET world. Log4net is one of the most popular logging frameworks for Microsoft’s .NET environment.

Log4J 2 goes steps ahead by removing the inherent architectural flaws of Log4J 1.x. Since the initial release of Log4J 2 on August 2015, it’s quickly being adopted by the developer community. I wrote an introductory post on Log4J 2 here. If you have not read it, I recommend starting with the introductory post first. In this post, I will discuss how to configure Log4J 2 using a properties configuration file. This is just one of several ways you can configure Log4J 2.

What are Log4J 2 Configuration Files?

Log4J 2 provides various components, such as loggers, appenders, and layouts that work together to perform logging in an application. As different applications have different logging requirements, you’re able configure LogJ 2 accordingly. Also, you will often need to keep changing Log4J 2 configurations of an application across its deployment lifecycle. For example, it is common to set the logging level to DEBUG during development, and later switch it to ERROR to avoid filling your logs with excessive debug information. Similarly, during local development, you can work with the console appender to avoid file I/O overheads and in other deployment environments, set a file appender or some other persistent destination to preserve log messages.

You can configure Log4J 2 either programmatically in your application or through configuration files, such as properties, XML, JSON, and YAML residing on your project classpath. Through the use of configuration files, you have the flexibility of changing the various configuration options without modifying your application code. In this post we’re going to look at using properties file.

Setting up Log4J 2 to Use Properties File

Unlike its predecessor Log4J 1.x, Log4J 2 did not support configuration through properties file when it was initially released. It was from Log4J 2.4 that support for properties file was again added, but with a completely different syntax.

Log4J4 Maven Dependencies

To use Log4J 2 in your application, you need to ensure that the Log4J 2 jars are on your project classpath. If you intend to use properties file, give extra attention to ensure that you have the Log4J 2.4 or greater jars on the classpath. Otherwise, your properties file will not get picked.
When using Maven, specify the following Log4J 2 dependencies.

1

2

3

4

5

6

7

8

9

10

11

12

...

<dependency>

<groupId>org.apache.logging.log4j</groupId>

<artifactId>log4j-api</artifactId>

<version>2.5</version>

</dependency>

<dependency>

<groupId>org.apache.logging.log4j</groupId>

<artifactId>log4j-core</artifactId>

<version>2.5</version>

</dependency>

...

Log4J 2 Spring Boot Dependencies

If you want to use Log4J 2 in a Spring Boot project, things can be a bit tricky. Simply adding the dependencies above won’t work as Spring Boot will first find the default Logback classic on the classpath, and will use it. Therefore, you need to exclude the default dependency of the Spring Boot starter on Logback classic, and instead include the Spring Boot starter dependency on Log4J 2, like this.

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

...

<dependency>

<groupId>org.springframework.boot</groupId>

<artifactId>spring-boot-starter</artifactId>

<exclusions>

<exclusion>

<groupId>org.springframework.boot</groupId>

<artifactId>spring-boot-starter-logging</artifactId>

</exclusion>

</exclusions>

</dependency>

<dependency>

<groupId>org.springframework.boot</groupId>

<artifactId>spring-boot-starter-log4j2</artifactId>

</dependency>

...

This will configure Spring Boot to use Log4J 2, but with a catch – You still won’t be able to use properties file for configuration. As of Spring Boot 1.3.3 Release, Spring Boot starter dependency on Log4J 2 is for Log4J 2.1, and as I have already mentioned it is from Log4J 2.4 onward that properties file is supported. Therefore, you need to explicitly specify dependencies of Log4J 2.4 or above after excluding Spring Boot starter logging, like this.

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

...

<dependency>

<groupId>org.springframework.boot</groupId>

<artifactId>spring-boot-starter</artifactId>

<exclusions>

<exclusion>

<groupId>org.springframework.boot</groupId>

<artifactId>spring-boot-starter-logging</artifactId>

</exclusion>

</exclusions>

</dependency>

<dependency>

<groupId>org.apache.logging.log4j</groupId>

<artifactId>log4j-api</artifactId>

<version>2.5</version>

</dependency>

<dependency>

<groupId>org.apache.logging.log4j</groupId>

<artifactId>log4j-core</artifactId>

<version>2.5</version>

</dependency>

...

The above dependencies will set up Log4J 2 to use properties file in a Spring Boot application.

Check out my FREE Introduction to Spring course!

Configuring Log4J 2 using Properties File

By default, Log4J 2 looks for a properties file with the name log4j2.properties in the classpath. In a Spring Boot application, the log4j2.properties file will typically be in the resources folder.

Before we start configuring Log4J 2, we will write a Java class to generate log messages via Log4J 2.

Log4J2PropertiesConf.java

Java

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

packageguru.springframework.blog.log4j2properties;

importorg.apache.logging.log4j.LogManager;

importorg.apache.logging.log4j.Logger;

publicclassLog4J2PropertiesConf{

privatestaticLogger logger=LogManager.getLogger();

publicvoidperformSomeTask(){

logger.debug("This is a debug message");

logger.info("This is an info message");

logger.warn("This is a warn message");

logger.error("This is an error message");

logger.fatal("This is a fatal message");

}

}

To test the Log4J2PropertiesConf class above, we will write a JUnit test class.

Log4J2PropertiesConfTest.java

Java

1

2

3

4

5

6

7

8

9

10

11

12

13

14

packageguru.springframework.blog.log4j2properties;

importorg.junit.Test;

importstaticorg.junit.Assert.*;

publicclassLog4J2PropertiesConfTest{

@Test

publicvoidtestPerformSomeTask()throwsException{

Log4J2PropertiesConf log4J2PropertiesConf=newLog4J2PropertiesConf();

log4J2PropertiesConf.performSomeTask();

}

}

We will now configure Log4J 2 using a properties file. Like any other Java properties file, a log4j2.properties file are a set of key value pairs with options to configure the various components of Log4J 2, such as loggers, appenders, and layouts. A basic log4j2.properties file starts with a name, optional properties to be used in other parts of the file, and appender declarations.

1

2

3

4

name=PropertiesConfig

property.filename=logs

appenders=console,file

...

The preceding code declares two appenders, named console and file. Next, let’s configure both the appenders to write log messages to the console and a file. The configuration code for the appenders is this.

In the code above we configured two appenders: One to write log messages to the console and the other to a log file. Both the appenders use pattern layouts that are configurable with conversion pattern strings to format log messages. The appender.console.layout.pattern property specifies the pattern string. You can learn more about the pattern layout and conversion pattern strings here. For the file appender, we used the appender.file.fileName property to specify the name and location of the log file that Log4J 2 will generate. Here, notice the ${filename} declaration that we used as a substitution for the property.filename property we declared earlier.

Next we will configure the loggers, starting from the root logger.

1

2

3

4

5

...

rootLogger.level=debug

rootLogger.appenderRefs=stdout

rootLogger.appenderRef.stdout.ref=STDOUT

...

In the code above, we configured the root logger to log debug and its lower level messages to the console (stdout). When we run the Log4J2PropertiesConfTest test class, the output in the IntelliJ console will be similar to this.

When we run the Log4J2PropertiesConfTest test class now, log messages will be sent to the logs/propertieslogs.log by the file logger and additively to the console by the root logger. The following figure shows the log messages sent to the file and console in IntelliJ.

In the example above, it is due to logger additivity that caused log messages to be sent to the file by the logger and additively to the console by the root logger. You can override this default behavior by setting the additivity flag of a logger to false.

1

2

3

...

logger.file.additivity=false

...

The property above configures our file appender so that it is no longer additive, thus log messages will only be sent to the file.

Appender additivity can be somewhat confusing. I suggest reviewing the Log4J 2 documentation on the subject, where they have some good examples how this works.

Summary

Using properties file is one of the several options you have to configure Log4J 2. Log4J 2 is gradually moving to XML configuration and the new JSON and YAML configurations. Properties configuration cannot handle some advanced features, such as custom error handlers, time-based rolling policies, nested appenders, and special types of appenders, such as asynch appenders. However, properties configuration is still being extensively used. Often you don’t need many of the more advanced logging features of Log4J 2. So you’re fine using the simplicity of the properties file configuration. In future posts, I will cover using other configuration options for Log4J 2 to address logging configurations with more complex requirements.

If you are still using System.out to print debugging or diagnostic information in your application, it’s time to look for a more elegant and efficient solution in the form of a logging framework. Although there are lots of logging frameworks around for Java applications, Log4J is one of the most widely adopted one because of the simplicity and flexibility it provides.

Note: The Log4J version 1 was first released on 1999 and quickly became the most used logging framework ever. But due to some inherent architectural flaws, Apache announced End of Life of Log4j version 1 on August 2015 and encouraged users to upgrade to Log4j 2 – A framework that is far more reliable, faster, and much easier to develop and maintain. Log4J 2 is almost a compete changed framework with a different API and support for different configuration files having different syntax. Therefore, from here onward, I will refer the framework as Log4J 2

Log4J 2 is an open source logging package distributed under the Apache Software License. The advantages it provides over System.out is monumental. Log4J 2 allows you to define different levels of importance, such as ERROR, WARN, INFO, and DEBUG for log messages. Log4J 2 also allows you to define one or more destinations, such as console, file, database, and SMTP server to send log messages. And the great thing is that using Log4J 2, you can perform logging asynchronously.

Additionally, Log4J 2 allows you to control logging on a class-by-class basis. For example, one class of an application can redirect logs to the console while another to a file. By contrast, a programmer can only control System.out at the granularity of the entire application. If a programmer redirects System.out, the redirect happens for the whole application.

Another important feature of Log4J 2 is that it’s easy to enable or disable only some type of log message. For example, you can configure Log4J 2 to disable every debug message in production.

So how Log4J 2 do all this? It’s through the loggers, appenders, and layouts – the components of the Log4J 2 API. These components work together to provide developer full control on how messages are logged, formatted, and where they are reported.

Loggers

Loggers are the key objects in Log4J 2 that are responsible for capturing logging information. Loggers are stored in a namespace hierarchy and a root logger, an implementation of the Logger interface, sits at the top of the hierarchy. Logger names are case-sensitive and they follow the hierarchical naming rule.

You can retrieve the root logger by calling the LoggerManager.getRootLogger() method. For all other loggers, you can instantiate and retrieve them by calling LoggerManager.getLogger(String loggerName) passing the name of the desired logger as a parameter. Although, you can specify any name for a logger, its recommend to name the logger with the fully qualified name of the class. In a large application, with thousands of log statements, it is easy to identify the origin of a log message as the log output bears the name of the generating logger. Since it is a common practice to name loggers after their owning class, Log4J 2 provides the overloaded convenience method LogManager.getLogger(). This method, by default, uses the fully qualified class name of the owning class.

Loggers can be assigned levels in the following order.

As you can see in the figure above, TRACE is the lowest level and the level moves up through DEBUG, INFO, WARN, ERROR, till FATAL – the highest level. What this means is that if you set the logger level to ERROR then only the ERROR and FATAL level log messages will be displayed and the rest will be ignored.

In addition to the levels I mentioned, there are two special levels:

ALL: Turns on all levels.

OFF: Turns off all levels.

While developing in your local machine, it is common to set the log level to DEBUG. This will give you detailed log messages for your development use. While on production, its typical set the log level to ERROR. This is to avoid filling your logs with excessive debug information. And while logging is very efficient, there is still a cost.

In your application, once you have retrieved a logger, you call one of the printing methods debug(), info(), warn(), error(), fatal(), and log() on the logger to log messages. These messages are contained in the Logger interface and part of the root logger that all Log4J 2 loggers inherit.

Appenders

Once you capture logging information through a logger, you need to send it to an output destination. The output destination is called an appender, and it is attached to the logger. Log4J 2 provides appenders for console, files, GUI components, remote socket servers, JMS, NT Event Loggers, and remote UNIX Syslog daemons.

Appenders are inherited additively from the logger hierarchy. This means, if the console appender is attached to the root logger, all child loggers will inherently use the console appender. If you have a child logger named Foo attached with a file appender, then Foo will use both the console and file appenders, unless you explicitly ask it not to do so by setting the additivity attribute to false.

Layouts

In addition to specifying your preferred output destination, you can also specify the format of log messages. You can do so by associating a layout with the appender. Some key layouts that Log4J 2 provides are PatternLayout, Htmlayout, JsonLayout, and XmlLayout. If you want to format logging data in an application-specific way, you can create your own layout class extending from the abstract AbstractStringLayout class – the base class for all Log4J 2 layouts that result in a String.

Using Log4J 2

Let’s jumpstart to create a trivial application to use Log4J 2 and start logging. For the application, I have used Spring Boot and started out with a Spring Boot starter POM. If you are new to Spring Boot, you can start with my introductory post on Spring Boot here.

As the Spring Boot starter POM uses Logback for logging, you need to exclude it and add the Log4J 2 dependencies.

Here is the code of the POM file to use Log4J 2 in a Spring Boot application.

1

2

3

4

5

6

7

8

9

10

11

12

...

<dependency>

<groupId>org.springframework.boot</groupId>

<artifactId>spring-boot-starter</artifactId>

<exclusions>

<exclusion>

<groupId>org.springframework.boot</groupId>

<artifactId>spring-boot-starter-logging</artifactId>

</exclusion>

</exclusions>

</dependency>

...

We can now start logging messages in the classes of the program. Let’s write a class for that.

MyApp.java

Java

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

packageguru.springframework.blog.log4joverview;

importorg.apache.logging.log4j.LogManager;

importorg.apache.logging.log4j.Logger;

publicclassMyApp{

privatestaticLogger logger=LogManager.getLogger("MyApp.class");

publicvoidperformSomeTask(){

logger.debug("This is a debug message");

logger.info("This is an info message");

logger.warn("This is a warn message");

logger.error("This is an error message");

logger.fatal("This is a fatal message");

}

}

In the class we wrote above, we retrieved a logger through a call to getLogger(). We then called the various printing methods on the logger.

Now let’s write a test class.

MyAppTest.java

Java

1

2

3

4

5

6

7

8

9

10

11

12

13

packageguru.springframework.blog.log4joverview;

importorg.junit.Test;

importstaticorg.junit.Assert.*;

publicclassMyAppTest{

@Test

publicvoidtestPerformSomeTask()throwsException{

MyApp app=newMyApp();

app.performSomeTask();

}

}

On running the test class, the output in the IntelliJ console is this.

You may have noticed that I did not specify an appender or layout. I did not configure either one, and Log4J 2 rightfully pointed that out with an error message, as shown in the figure above. Rather, I relied upon defaults inherited from the Log4J 2 root logger. The Log4J 2 root logger is associated with the console appender (ConsoleAppender class) by default, and our logger inherited it. Therefore the log messages were sent to the IntelliJ console. If you notice, only the error and fatal messages got logged. This happened because by default Log4j 2 assigns the root logger level to ERROR and without external configuration, messages of the lower levels (WARN, INFO, and DEBUG) were not send to the destination. Also, the root logger by default uses PatternLayout, which our logger inherited and used.

Summary

In this post, I have only scratched the surface of Log4J 2. You will realize the power of Log4J 2 when you start working with external configuration files. These configuration files can be .properties, XML, YAML, and JSON files containing Log4J 2 configuration options. This means, you can set and change the configuration options without having to modify and recompile the application. In upcoming posts, I will discuss using external configuration files to help you explore what a powerful logging tool Log4J 2 is.

There are a number of logging solutions available for Java. Each has its own merits and drawbacks. Yet all of them are far better options than using System.out.println()! Printing out to the console, simply is not an enterprise class solution. Often in the enterprise, your log files need to be secured, and are often indexed by monitoring tools such as Splunk. Professional Java developers will use a logging framework such as Log4J 2.