Send Spring Boot Logs Directly to Logstash With No File

Sending Spring Boot logs directly to Logstash without writing them to a file can be achieved using a logging library like Logback or Log4j2 configured to send logs over a network protocol such as TCP or UDP. Below is a detailed guide on how to configure your Spring Boot application to achieve this using Logback.

Step 1: Add Required Dependencies

Ensure your Spring Boot application includes the necessary dependencies for Logback and a logging client that can send logs to Logstash. In your pom.xml (for Maven), add the following dependencies:

<dependencies>
    <!-- Spring Boot Starter for Logging -->
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-logging</artifactId>
    </dependency>

    <!-- Logback Encoder for JSON output (optional, for structured logging) -->
    <dependency>
        <groupId>net.logstash.logback</groupId>
        <artifactId>logstash-logback-encoder</artifactId>
        <version>7.2</version>
    </dependency>
</dependencies>

If you're using Gradle, your build.gradle would look something like this:

dependencies {
    implementation 'org.springframework.boot:spring-boot-starter-logging'
    implementation 'net.logstash.logback:logstash-logback-encoder:7.2'
}

Step 2: Configure Logback

Create or modify your logback-spring.xml file in the src/main/resources directory to configure Logback to send logs to Logstash. Below is an example configuration:

<configuration>

    <appender name="LOGSTASH" class="net.logstash.logback.appender.LogstashTcpSocketAppender">
        <remoteHost>localhost</remoteHost> <!-- Logstash host -->
        <port>5044</port>                   <!-- Logstash port -->
        <encoder class="net.logstash.logback.encoder.LoggingEventCompositeJsonEncoder">
            <providers>
                <timestamp />
                <logger />
                <thread />
                <level />
                <message />
                <logstashMarkers />
                <arguments />
                <MDC />
                <context />
            </providers>
        </encoder>
    </appender>

    <root level="INFO">
        <appender-ref ref="LOGSTASH" />
    </root>

</configuration>

Step 3: Configure Logstash

Set up Logstash to receive logs from your Spring Boot application. Create or modify your Logstash configuration file (e.g., logstash.conf):

input {
  tcp {
    port => 5044
    codec => json_lines  # Use json_lines codec to decode JSON logs
  }
}

filter {
  # Optional: Additional filters can be added here
}

output {
  elasticsearch {
    hosts => ["<http://localhost:9200>"]  # Update with your Elasticsearch host
    index => "spring-boot-logs-%{+YYYY.MM.dd}"  # Index pattern
  }
}

Step 4: Run Logstash

Start Logstash with your configuration:

bin/logstash -f /path/to/your/logstash.conf

Step 5: Run Your Spring Boot Application

Run your Spring Boot application normally. The logs should now be sent directly to Logstash without being written to a file.

Step 6: Verify Logs in Elasticsearch

To verify that the logs are being sent to Elasticsearch, you can query Elasticsearch directly:

curl -X GET "<http://localhost:9200/spring-boot-logs-*/_search?pretty>"

Conclusion

By following these steps, you can send Spring Boot logs directly to Logstash without writing them to a file. This approach allows for real-time logging and monitoring of your applications, facilitating better observability and faster troubleshooting. Make sure to adjust the configurations to fit your environment and requirements.

Got an article suggestion? Let us know

Explore more

Logstash

This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.