Docker Compose Logs: A Comprehensive Guide

Docker Compose is a tool that streamlines the management of multi-container Docker applications by allowing you to define and configure all your services, networks, and volumes within a single YAML file.

Each service declared in this file runs in an isolated container, producing logs that record events, activities, and any errors encountered.

When you use the docker compose logs command, these individual container logs are aggregated and presented together. This consolidated output is what is referred to as Docker Compose logs.

This guide explores the key concepts of Docker Compose logs and explains its customization options to help meet your monitoring and debugging needs.

Let's get started!

Prerequisites

To follow along with this guide, ensure that you meet the following prerequisites:

• Basic command-line skills.
• A recent version of Docker installed on your system.
• A Docker Compose setup that's already generating logs such as the following:

[docker-compose.yml]
services:
  collector:
    container_name: otel-collector
    image: otel/opentelemetry-collector:latest

  vector:
    image: timberio/vector:latest-alpine
    container_name: vector-demo

These two services are configured to automatically generate logs continuously so ensure to start them with the command below before proceeding:

docker compose up -d

Getting started with Docker Compose logs

The primary command for viewing logs is docker compose logs, and it must be run from the directory containing the relevant docker-compose.yml file or a subdirectory.

It has the following syntax:

docker compose logs [<options>] [<service>...]

Executing the command without options will display the logs for all services in the Compose file:

docker compose logs

This presents a consolidated and chronological view of all logs generated by the containers in the Compose application stack.

Each log entry is prefixed with the container name to help identify the source. For example:

vector-demo     | 2024-12-27T10:33:42.745754Z  INFO vector::signal: Signal received. signal="SIGTERM"
vector-demo     | 2024-12-27T10:33:42.745800Z  INFO vector: Vector has stopped.
vector-demo     | 2024-12-27T10:33:42.746980Z  INFO vector::topology::running: Shutting down... Waiting on running components. remaining_components="print, parse_logs, dummy_logs" time_remaining="59 seconds left"
vector-demo     | {
vector-demo     |   "appname": "AmbientTech",
vector-demo     |   "facility": "local0",
vector-demo     |   "hostname": "some.attorney",
vector-demo     |   "message": "Maybe we just shouldn't use computers",
vector-demo     |   "msgid": "ID374",
vector-demo     |   "procid": 5312,
vector-demo     |   "severity": "warning",
vector-demo     |   "timestamp": "2024-12-27T10:33:42.935Z",
vector-demo     |   "version": 1
vector-demo     | }
otel-collector  | StartTimestamp: 1970-01-01 00:00:00 +0000 UTC
otel-collector  | Timestamp: 2024-12-27 10:33:27.937 +0000 UTC
otel-collector  | Value: 8.000000
otel-collector  | Metric #11
otel-collector  | Descriptor:
otel-collector  |      -> Name: scrape_samples_post_metric_relabeling
otel-collector  |      -> Description: The number of samples remaining after metric relabeling was applied
otel-collector  |      -> Unit:
otel-collector  |      -> DataType: Gauge
otel-collector  | NumberDataPoints #0
otel-collector  | StartTimestamp: 1970-01-01 00:00:00 +0000 UTC
otel-collector  | Timestamp: 2024-12-27 10:33:27.937 +0000 UTC
otel-collector  | Value: 8.000000
otel-collector  |       {"kind": "exporter", "data_type": "metrics", "name": "debug"}

If scaling is applied to a service (e.g., docker-compose up --scale vector=3), each instance will be uniquely identified (e.g. vector-1, vector-2, etc).

You'll also notice that the container names are color-coded to visually distinguish logs from different containers:

If you get the following error from docker compose logs, it means that Docker cannot locate a docker-compose.yml (or compose.yml) file in the working directory.

no configuration file provided: not found

Therefore, you'll need to navigate to the directory containing the file before re-running the command.

Viewing logs for specific services

In most cases, you only want to view logs from a specific service or a subset of services within your Docker Compose stack. You can achieve this by specifying one or more service names when running the docker compose logs command:

docker compose logs <service_1> <service_2> . . .

For example, to view logs for the vector service alone, run:

docker compose logs vector

vector-demo  | {
vector-demo  |   "appname": "shaneIxD",
vector-demo  |   "facility": "ntp",
vector-demo  |   "hostname": "up.lc",
vector-demo  |   "message": "You're not gonna believe what just happened",
vector-demo  |   "msgid": "ID628",
vector-demo  |   "procid": 6363,
vector-demo  |   "severity": "notice",
vector-demo  |   "timestamp": "2024-12-27T11:25:08.084Z",
vector-demo  |   "version": 2
vector-demo  | }
vector-demo  | {
vector-demo  |   "appname": "CrucifiX",
vector-demo  |   "facility": "lpr",
vector-demo  |   "hostname": "make.ltd",
vector-demo  |   "message": "We're gonna need a bigger boat",
vector-demo  |   "msgid": "ID562",
vector-demo  |   "procid": 9048,
vector-demo  |   "severity": "info",
vector-demo  |   "timestamp": "2024-12-27T11:25:09.084Z",
vector-demo  |   "version": 2
vector-demo  | }

You must provide the service name as defined in the Compose file rather than the container name (vector-demo in this case) to ensure that logs from all instances of the service are displayed together.

If you'd like to isolate the logs for a specific service instance, you can use the --index flag as shown below:

docker compose logs <service> --index 1 # show logs from instance 1 of <service>

The docker compose logs command provides various other options to customize its output. Here's a brief overview:

Let's look at each one of them in turn.

Tailing Docker Compose logs

The default docker compose logs command outputs all available logs from all running services, which is often impractical, especially for long-running services or applications generating high volumes of logs.

In most cases, you’re only interested in:

• Viewing the most recent log entries, or
• Actively monitoring new logs in real-time for a specific service or group of services.

To limit the log output to the most recent entries, use the -n or --tail flag:

docker compose logs --tail 50

docker compose logs --tail 50 <service>

This above commands will display the last 50 lines of logs from all services or from a specific service.

You can also monitor the live activity of running containers with the -f or --follow flag:

docker compose logs --follow

For the best of both worlds, combine --tail with --follow to narrow the initial output to recent logs while streaming new log entries:

docker compose logs --follow --tail 20

This displays the last 20 lines as the initial output and continues to stream new log entries in real-time.

Filtering Docker Compose logs by timestamp

Docker Compose logs can be filtered by specific time ranges using the --since and --until flags. These options allow you to narrow down log output to a particular period, making it easier to pinpoint issues or monitor activity within defined time frames.

Both flags accept the following time formats:

• RFC 3339 date: e.g., 2024-12-17T09:00:00.
• UNIX timestamp: e.g., 1702819200.
• Go duration string: e.g., 1m30s, 3h.

Here are some examples:

• Show logs produced after a specific timestamp:

docker compose logs --since 2024-12-27T09:00:00

• Display logs from the last 15 minutes:

docker compose logs --since 15m

• Show logs produced before a specific timestamp:

docker compose logs --until 2024-12-27T12:00:00

• Present logs from up to 1 hour ago:

docker compose logs --until 1h

You can also filter logs within a specific time range by combine both options, such as:

docker compose logs --since 2024-12-17T12:45:00 --until 2024-12-17T13:00:00

This command displays logs generated between 12:45 PM and 1:00 PM on December 17, 2024.

Formatting the Docker Compose log output

There are a few ways to customize the Docker Compose log output. The most useful option is often --no-log-prefix which removes the log prefix from the output when copying log lines for further processing or sharing:

docker compose logs --no-log-prefix

If the service logs do not include timestamps, you can enable them with the -t or --timestamps option. This prepends ISO 8601 timestamps to each log entry.

docker compose logs --no-log-prefix --timestamps vector

2024-12-27T15:30:15.447550836Z {
2024-12-27T15:30:15.447597685Z   "appname": "AmbientTech",
2024-12-27T15:30:15.447604453Z   "facility": "daemon",
2024-12-27T15:30:15.447610353Z   "hostname": "we.fast",
2024-12-27T15:30:15.447615603Z   "message": "Great Scott! We're never gonna reach 88 mph with the flux capacitor in its current state!",
2024-12-27T15:30:15.447621239Z   "msgid": "ID775",
2024-12-27T15:30:15.447626122Z   "procid": 3492,
2024-12-27T15:30:15.447631152Z   "severity": "info",
2024-12-27T15:30:15.447636136Z   "timestamp": "2024-12-27T15:30:15.446Z",
2024-12-27T15:30:15.447641455Z   "version": 1
2024-12-27T15:30:15.447646420Z }

A limitation of Docker's logging driver is that it treats multiline logs as a series of separate log entries.

Each line in the output is handled as an independent log entry, which can make it challenging to work with logs that span multiple lines, such as stack traces, formatted logs, or verbose application outputs.

To avoid this in your services, configure them to always emit single-line logs. For example:

{"appname":"AmbientTech","message":"Great Scott! We're never gonna reach 88 mph...","severity":"info","timestamp":"2024-12-27T15:30:15.446Z"}

You can then pipe the logs through a JSON processor like jq or similar tools to make them more human-readable.

Finally, you can disable colored logs with the --no-color option:

docker compose logs --no-color

Viewing Docker Compose Logs with Dozzle

An easier way to monitor logs from multiple services in a Compose stack is by using a log viewer like Dozzle.

With features like Pin as column, Dozzle makes it simple to display logs from multiple containers side by side, making it particularly useful for debugging issues involving interactions between multiple services.

You can set it up by running Dozzle as a standalone container:

docker run --name dozzle -d --volume=/var/run/docker.sock:/var/run/docker.sock -p 8888:8080 amir20/dozzle:latest

Or you can add it as a service in your Compose file:

services:
  dozzle:
    image: amir20/dozzle:latest
    container_name: dozzle
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
    ports:
      - 8888:8080

Once the Dozzle container is running, open your browser and navigate to http://localhost:8888.

From there:

• Select the container whose logs you want to view.

• Use the Pin as column feature to display logs from multiple containers side by side.

Centralizing your Docker Compose logs

While Docker provides basic tools for viewing multi-container application logs, they will fall short when dealing with large-scale deployments or containers spread across multiple servers. Managing logs efficiently in such environments requires a centralized solution.

Centralized logging is when all your container logs are gather and stored centrally, allowing you to monitor and analyze them in one location. This helps you find problems faster and keeps your logs safe and usable, even if servers crash or are shut down.

Better Stack is a comprehensive observability platform that simplifies log management and monitoring for multi-container applications.

For example, it can take your container logs and transform them into metrics, allowing you to view high-level trends and drill down into detailed insights when needed.

Coupled with high-performance querying, intuitive dashboards, collaborative workflows, and incident management tools, Better Stack empowers you to swiftly identify and resolve issues, minimizing downtime and go beyond basic monitoring to achieve observability.

Give it a try by signing up for a free account here.

Final thoughts

I hope this article has given you a solid grounding in how Docker Compose handles logs and how to use its features to monitor and troubleshoot your multi-container applications.

To learn more, ensure to dive deeper with the official Docker Compose logs documentation, and don't miss our guide on Docker logging best practices to optimize the performance and reliability of your logging setup.

Thanks for reading, and happy logging!

Got an article suggestion? Let us know

Next article

A Complete Guide to Logging in Heroku

Learn how to start logging in Heroku and go from basics to best practices in no time.

→

This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.