6 Factors to Consider When Choosing a Logging Framework

A logging framework is a tool that helps you standardize the process of logging in your application. Some languages provide a logging module in their standard library for this purpose, but most logging frameworks come in the form of third-party libraries such as log4j (Java), Zap (Go), or Winston (Node.js). Some organizations also choose to develop a custom in-house logging framework, but this is often restricted to large companies with highly specialized needs.

For a tool to qualify as a logging framework, it must solve all the most common concerns when it comes to logging. This includes capturing the information to be recorded (with a timestamp and log level), formatting the logs preferably using some structured format like JSON, and transporting them to one or more destinations (the console, a file, some cloud service, etc). It also needs to be adaptable to different deployment environments and situations, and be as efficient as possible so that the performance of your application isn't affected even when many logs are being produced.

The concerns above disqualify packages like the log package in Go or the Console module in Node.js as serious logging solutions for production applications. While they address some concerns like recording log data, other essential features like log levels and support for structured formats are missing (at least without extra work to extend the API by app developers). So in most situations, you will need to forgo the logging solution built into the language and adopt a third-party library to handle your logging needs.

Therefore, this article will discuss some of the most critical factors to look out for when choosing a logging framework, and also give examples of specific frameworks to consider for some of the most popular languages.

1. Support for capturing a wide variety of data types

All logging frameworks will capture and record log messages, but support for data types may vary. For example, some frameworks require you to convert everything you feed to them into a string, while others provide the ability to capture a rich variety of data types. The latter style is usually much faster with less allocations compared to string formatting so it should be prioritized when deciding what logging framework to use.

2. First-class support for structured logging formats

Currently, most logging frameworks default to outputting unstructured log data primarily intended to be read directly by humans. These logs are made up of strings containing one or more embedded variables that often need to be parsed and extracted later on. The problem with unstructured records is that processing them cannot be automated easily due to the unstructured nature of the data. Often, finding an event involves writing a custom regular expression to match the relevant logs, and these expressions likely need to be modified if the log data changes.

On the other hand, structured logs consist of objects instead of strings, and each property in the object can be automatically extracted and processed by log processing tools in order to perform searches, provide alerts, and used produce a variety output formats that is more easily digestible by a human audience. The good news is many logging frameworks also support structured logging. Some default to structured logging, while others do not support unstructured logs in the first place.

Before selecting a logging framework, you should consider how the log data will be processed so that you can choose something that supports the required structured log format. Most frameworks utilize JSON for structured logging because it integrates well with most logging tools and systems, but if you have different formatting requirements, ensure that your logging framework supports it satisfactorily. Some frameworks also provide the ability to use a completely custom format, so be sure to investigate your options carefully here before making a decision.

Since structured logs are generally harder to read by humans, you might also consider choosing a framework that makes it easy to prettify the log output in development so that you can get a human-friendly, and possibly colorized console output while production systems use the more efficient structured format.

3. Error handling behavior

Errors are often the most common target for logging, so knowledge of how a library handles errors is crucial before adopting it. At a minimum, your framework should capture all the relevant details about the error (including its stack trace) by default. Some popular frameworks get this one wrong, so ensure to test and read the documentation thoroughly to verify the error handling behavior of your chosen solution.

Some frameworks output the entire stack trace as a string in an object property like this:

{"level":"error","time":1643706943924,"pid":13185,"hostname":"Kreig","err":{"type":"Error","message":"ValidationError: email address in invalid","stack":"Error: ValidationError: email address in invalid\n    at Object.<anonymous> (/home/ayo/dev/betterstack/demo/snippets/main.js:3:14)\n    at Module._compile (node:internal/modules/cjs/loader:1097:14)\n    at Object.Module._extensions..js (node:internal/modules/cjs/loader:1149:10)\n    at Module.load (node:internal/modules/cjs/loader:975:32)\n    at Function.Module._load (node:internal/modules/cjs/loader:822:12)\n    at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:81:12)\n    at node:internal/main/run_main_module:17:47"},"msg":"ValidationError: email address in invalid"}

Others may provide an option to format the stack trace into objects such that they can be automatically parsed:

{"level":"error","stack":[{"func":"inner","line":"20","source":"main.go"},{"func":"middle","line":"24","source":"main.go"},{"func":"outer","line":"32","source":"main.go"},{"func":"main","line":"15","source":"main.go"},{"func":"main","line":"250","source":"proc.go"},{"func":"goexit","line":"1571","source":"asm_amd64.s"}],"error":"seems we have an error here","time":1658700227}

If the language uses exceptions, you should also investigate if the framework can catch and log uncaught exceptions before the program exits so that you're not left clueless about how such a crash occurred during a debugging session.

4. Impact on application performance and maintenance

Since logging calls will be prevalent in your codebase, you need to ensure that the chosen framework supports abstractions that will help you minimize coupling in case it becomes necessary to migrate to a different framework later. A common approach involves using a logging interface to isolate the logging framework from the application code such that if the current one goes stale or a better one comes along, you can easily swap out the implementation in one place, and leave the logging calls untouched.

Options for standardizing your logging APIs include using an already established logging facade for your language of choice (like SLF4J for Java applications), or going the DIY approach by rolling out a custom interface. The latter offers more flexibility and customization, at the cost of increased maintenance workload.

You should also investigate the performance characteristics of each library under consideration before investing in one. Ideally, you should find existing and up-to-date benchmarks or even roll out your own to compare the different options. Some frameworks offer different tiers of performance depending on the enabled features, so you can also use that information to decide if certain features are worth the costs imposed on your application's runtime.

5. Log transportation solutions

Transporting log entries to one or more locations is another important consideration when choosing a logging framework. Ideally, your framework of choice should be able to directly append logs to the most common destinations (console and file), coupled with the ability to configure custom destinations for the log output (e.g. some database or cloud service). In such cases, you may also be able to take advantage of custom transports that already exist in the community.

If you have complete control over your environment, you can output all logs to the console or a file, and use a dedicated log shipping tool (such as Rsyslog, Vector, or similar tools) to collect, transform, and forward them to other destinations as needed. In other situations where the deployment platform provides limited access to the application environment, the ability to ship the logs directly to their final destination may be pivotal.

6. Reputation and community support

Another important consideration when choosing a logging framework is its reputation in the community. Most of the established logging frameworks have been around for many years, and they've been deployed and tested in diverse environments, so it should be easy to find reports of people's experiences which can be a tie-breaker if there are multiple good options.

You also want to find out if the framework is being actively maintained, what documentation is available, and if it has significant activity on Q&A sites like Stack Overflow or the project's issue tracker such that if you run into issues, you have an outlet for asking questions and fixing them.

Some logging framework recommendations

To make choosing a logging framework a bit easier, we have prepared a list of recommended frameworks for a few of the most popular languages out there. If you want to dig deeper into these libraries, you can check out the linked guides below.

Pino (Node.js)

Pino is a Node.js structured logging framework that offers great performance and a wealth of useful features. Although it logs in JSON format by default, it provides a pino-pretty module for making the log output more human-friendly in development, and it can also produce logs in other formats with the help of transport modules.

It handles errors satisfactorily and integrates well with all the most popular Node.js web frameworks. It also offers a unique log redaction feature that helps you keep sensitive data out of your logs. We generally recommend Pino over Winston because it's lighter and easier to use with saner defaults and better performance to boot.

Zap or Zerolog (Go)

Zap is a structured logging framework made by Uber that offers impressive performance, features and a customizable API, making it one of the go-to options for logging in the Go community. It provides two distinct APIs for logging: a base Logger that avoids reflection when encoding JSON to prevent allocations and serialization overhead where possible, and a SugaredLogger that is less verbose and does not insist on structured logging.

Alternatively, you can check out Zerolog. It is another Go logging library dedicated to structured JSON (or CBOR) logging only using a similar zero-allocation approach as Zap's Logger API, but with a simpler API and better performance. It also offers a human-friendly, colorized console output through zerolog.ConsoleWriter, but this is significantly less efficient than its structured logging API so it's intended for development environments only.

To avoid coupling your application's logging implementation to any single logging framework (as discussed earlier in tip #4), you can wrap it in a library like logr which defines an interface for structured, leveled logging. Henceforth, you can write logging code in terms of logr.Logger while the actual implementation of the API (through Zap, Zerolog, or other frameworks) is managed in one place. This architecture makes it easy to swap out one logging framework for another if future needs demand it.

Monolog (PHP)

Monolog is the most popular and recommended logging framework for PHP applications. In fact, popular PHP application frameworks like Laravel and Symphony integrate with it to provide a comprehensive logging solution. It supports structured logs (via its JSON Formatter) and several other structured and unstructured formats, and a variety of handlers for transporting your logs to various destinations. It also implements the PSR-3 logger interface (a common PHP interface for logger objects) which makes it easy to switch to another compatible library at a later time.

SLF4J with Log4J2 or Logback (Java)

Simple Logging Facade for Java (SLF4J) is a logging facade for various Java logging frameworks such as Log4j2, Logback, and java.util.logging. It introduces a generic API layer independent of the actual implementation so that you can easily migrate from one framework to the other with minimial impact if you feel your needs/requirements aren't being met. Regarding what SLF4J implementation to use, Log4j2 and Logback are two of the most popular ones out there. They are both mature and stable, and support all the necessary features you're likely to need. The former appears to be more actively maintained and the better performer of the two.

Logging module or Loguru (Python)

The built-in logging module in Python's standard library is quite robust, so it's a good place to start your logging journey. It boasts many features that one might need a third-party logging framework for in other languages, and it adequately satisfies our criteria for a good logging framework. It is also widely used in the Python ecosystem, making it easy to find help when needed.

The main criticism of the logging module is that its setup and configuration can be convoluted even for simple tasks, so if you find its API cumbersome, you can checkout Loguru, the most prominent third-party logging framework for Python. It positions itself as a simpler alternative that's pre-configured (but customizable), with tons of features and better performance. It supports structured and unstructured logging, fully descriptive exception stack traces, and it also retains compatibility with the standard logging module.

Final thoughts

Choosing a logging framework is an important decision for your application as it will be used heavily throughout its codebase to record various events which will subsequently be read by humans or processed through a log management tool. The good news is you don't have to reinvent the wheel as there are tons of options out there that will solve the problem for you adequately.

We hope the guidelines above help you decide on what logging framework is best for you. Once you've incorporated a logging API into your project, you need to get the recorded data to where it can do some good and this is where a cloud log management service like Logtail can come in handy to make your logs more accessible when they are needed and simplify its processing and consumption.

Thanks for reading, and happy logging!

Got an article suggestion? Let us know

This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.