How To – Structured Logging with Spring Boot

Last Updated on November 14, 2024 by jt

Support for Structured Logging is a new Spring Boot feature introduced with the Spring Boot 3.4.0 release. Structured Logging is fairly common to use in distributed environments where you are using some type of consolidated logging.

Typically, you will use a JSON log format to allow for easy consolidated log searches with indexing tools such as Elasticsearch. Setting up Structured logging prior to Spring Boot 3.4.0 is fairly easy to do, as I explain in this blog post. While easy to do, this does not offer the flexibility nor configuration ease that we now have with the new Spring Boot feature supporting Structured Logging.

In this post, we’ll explore the new Structured Logging features now available in Spring Boot.

Structured Logging Quickstart

First, create a new Spring Boot project at start.spring.io with Spring Boot version 3.4.x. You don’t need anything further, but for my demo project I’m adding Spring MVC and Lombok support.

To demo log output, I’ve created a simple class that will generate some log chatter for our testing.

@Slf4j
@Component
public class LogGenerator implements CommandLineRunner {

    @Override
    public void run(String... args) throws Exception {
        log.trace("Trace log");
        log.debug("Debug log");
        log.info("Info log");
        log.warn("Hey, This is a warning!");
        log.error("Oops! We have an Error. OK");
    }
}

Without changing anything, this class will generate the following log output:

The Spring Boot Logback defaults are configured to use ANSI coloring, and log levels of info and above. If you’re new to Logback, check out this post where I explore configuring Logback with Spring Boot.

To enable structured logging using the Logstash format, add the following line to your applications.properties file.

logging.structured.format.console=logstash

This simple configuration line, effectively does everything I showed in my previous post on JSON Logging with Spring Boot.

Restarting the application, you will see the following console log output.

{"@timestamp":"2024-10-29T09:55:36.917469-04:00","@version":"1","message":"Info log","logger_name":"guru.springframework.structuredlogging.bootstrap.LogGenerator","thread_name":"main","level":"INFO","level_value":20000}
{"@timestamp":"2024-10-29T09:55:36.91769-04:00","@version":"1","message":"Hey, This is a warning!","logger_name":"guru.springframework.structuredlogging.bootstrap.LogGenerator","thread_name":"main","level":"WARN","level_value":30000}
{"@timestamp":"2024-10-29T09:55:36.917762-04:00","@version":"1","message":"Oops! We have an Error. OK","logger_name":"guru.springframework.structuredlogging.bootstrap.LogGenerator","thread_name":"main","level":"ERROR","level_value":40000}

In this case, we are using the logstash format. Here is the last line, in the pretty print format:

{
  "@timestamp": "2024-10-29T09:55:36.917762-04:00",
  "@version": "1",
  "message": "Oops! We have an Error. OK",
  "logger_name": "guru.springframework.structuredlogging.bootstrap.LogGenerator",
  "thread_name": "main",
  "level": "ERROR",
  "level_value": 40000
}

Supported Logging Formats

Out of the box, Spring Boot supports the following Structured Logging formats:

Using the different formats is easy to do. Just set logging.structured.format.console property to logstash, ecs or gelf.

Following is an example of the ecs format:

{
  "@timestamp": "2024-10-29T14:09:01.935015Z",
  "log.level": "ERROR",
  "process.pid": 29431,
  "process.thread.name": "main",
  "service.name": "structured-logging",
  "log.logger": "guru.springframework.structuredlogging.bootstrap.LogGenerator",
  "message": "Oops! We have an Error. OK",
  "ecs.version": "8.11"
}

Here is an example of the gelf format:

{
  "version": "1.1",
  "short_message": "Oops! We have an Error. OK",
  "timestamp": 1730211029.871,
  "level": 3,
  "_level_name": "ERROR",
  "_process_pid": 29483,
  "_process_thread_name": "main",
  "host": "structured-logging",
  "_log_logger": "guru.springframework.structuredlogging.bootstrap.LogGenerator"
}

Output Structured Output to File

Spring Boot is also pre-configured to support sending structured logging to a file. The same formats are support for file logging. Simply set the format property to logging.structured.format.file in application.properties.

Example

logging.structured.format.file=ecs

Spring Framework 6: Beginner to Guru

Checkout my best selling course on Spring Framework 6. This is the most comprehensive course you will find on Udemy. All things Spring!

Custom Log Formats

Custom Key Value Log Format

Structured Logging in Spring Boot can support custom formats as well. You can create a custom formatter for anything you may need.

For example, if you wished your log format to be output in key value pairs, you can define a new Java class which implements the StructuredLogFormatter interface.

Here is an example of a custom log formatter.

public class KeyValueLogger implements StructuredLogFormatter<ILoggingEvent> {

    @Override
    public String format(ILoggingEvent event) {
        return "level=" + event.getLevel() +
                " time=" + event.getTimeStamp() +
                " message=" + event.getMessage() +
                " timestamp=" + event.getTimeStamp() + "\n";

    }
}

To configure this for use, just set the path and classname in your application.properties file as follows:

logging.structured.format.console=guru.springframework.structuredlogging.log.KeyValueLogger

Example Output:

level=INFO time=1731264561320 message=Info log timestamp=1731264561320
level=WARN time=1731264561320 message=Hey, This is a warning! timestamp=1731264561320
level=ERROR time=1731264561320 message=Oops! We have an Error. OK timestamp=1731264561320

Custom JSON Log Format

We can also easily create a custom JSON log format by using the Spring Boot JsonWriter. Following is an example using the JsonWriter.

public class JsonLogger implements StructuredLogFormatter<ILoggingEvent> {
    private final JsonWriter<ILoggingEvent> writer = JsonWriter.<ILoggingEvent>of((members) -> {
        members.add("level", ILoggingEvent::getLevel);
        members.add("time", ILoggingEvent::getTimeStamp);
        members.add("message", ILoggingEvent::getFormattedMessage);
        members.add("applicationInfo").usingMembers((applicationInfo) -> {
            applicationInfo.add("name", "StructuredLoggingApplication");
            applicationInfo.add("version", "1.0.0-SNAPSHOT");
        });

    }).withNewLineAtEnd();

    @Override
    public String format(ILoggingEvent event) {
        return this.writer.writeToString(event);
    }
}

After updating our application.properties to use our JsonLogger implementation, the output is now:

{"level":"INFO","time":1731266745964,"message":"Info log","applicationInfo":{"name":"StructuredLoggingApplication","version":"1.0.0-SNAPSHOT"}}
{"level":"WARN","time":1731266745964,"message":"Hey, This is a warning!","applicationInfo":{"name":"StructuredLoggingApplication","version":"1.0.0-SNAPSHOT"}}
{"level":"ERROR","time":1731266745964,"message":"Oops! We have an Error. OK","applicationInfo":{"name":"StructuredLoggingApplication","version":"1.0.0-SNAPSHOT"}}

Conclusion

As you can see, Spring Boot has a lot of flexibility to generate structured log output. You have 3 common formats out of the box which should cover most use cases. However, if those are not sufficient for your needs, you have full control over the log format generated.

The source code used in this post is available on GitHub here.