To direct logs matching a specific tag to multiple outputs in Fluentd, the @type copy directive can be utilized. Here's an example configuration:
<match pattern>
@type copy
<store>
@type file
path /var/log/myapp1
...
</store>
<store>
...
</store>
<store>
...
</store>
</match>
</label>
The copy plugin in Fluentd is designed to duplicate log events and send them to multiple destinations. The <store> section within the <match> block is where you define and configure the storage output for each duplicated log entry.
Suppose you have a source generating logs:
<source>
@type dummy
dummy '{"timestamp":"2024-01-27T08:33:17","severity":"ERROR"}'
format json
tag mylogs
</source>
To send these logs to both a file and standard output, you can configure it as follows:
...
<match mylogs>
@type copy
<store>
@type file
path /var/log/mytestapp
format json
</store>
<store>
@type stdout
</store>
</match>
</label>
When running Fluentd, you will see the logs in the console:
2024-01-28 19:26:29 +0000 [info]: #0 fluentd worker is now running worker=0
2024-01-28 19:26:30.071500553 +0000 mylogs: {"timestamp":"2024-01-27T08:33:17","severity":"ERROR"}
Simultaneously, a /var/log/mytestapp directory will be created, containing two files:
buffer.<b6100691ab4b1f59fcafccef634b0b085>.log buffer.b6100691ab4b1f59fcafccef634b0b085.log.meta
Viewing the file ending with .log will display the log contents:
{"timestamp":"2024-01-27T08:33:17","severity":"ERROR"}
🔠Want to centralize and monitor your logs?
Go to Logtail and start your log management in 5 minutes. [/summary]
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
-
How to Rotate Log Files in Fluentd?
Fluentd offers the capability to set up log rotation, particularly useful when managing large volumes of logs that consume significant disk space. This feature can be configured through Fluentd's s...
Questions -
How to Use Conditional Statements In Fluentd
Conditional statements can be seamlessly integrated into Fluentd's record_transformer plugin. Here’s how you can do it: <filter mylogs> @type recordtransformer enableruby true <record> is...
Questions -
How to Add Tags to My Fluentd Events
Here is how you can add tags to Fluentd events. Let's assume you configured Fluentd to process Nginx access logs from the access.log file with a source configuration like this: <source> @type ta...
Questions -
Fluentd
Learn how to use Fluentd to collect, process, and ship log data at scale, and improve your observability and troubleshooting capabilities.
Guides
Make your mark
Join the writer's program
Are you a developer and love writing and sharing your knowledge with the world? Join our guest writing program and get paid for writing amazing technical guides. We'll get them to the right readers that will appreciate them.
Write for us
Build on top of Better Stack
Write a script, app or project on top of Better Stack and share it with the world. Make a public repository and share it with us at our email.
community@betterstack.comor submit a pull request and help us build better products for everyone.
See the full list of amazing projects on github