Better Stack collector for Docker

Better Stack collector is the easiest and the recommended way to send logs, metrics and traces from your Docker hosts to Better Stack. Install it with one command on a single host or across a Docker Swarm cluster.

Why should you use the collector?

  • One install command. Start simple and use the advanced options later on.
  • Open source, no lock-in. The best open source tools, preconfigured for you: OpenTelemetry, Vector, OBI, the eBPF sensor maintained by CNCF, and Prometheus exporters for popular databases.
  • Traces without code changes. eBPF instruments your services and builds a service map from their network flows. It complements your existing OpenTelemetry SDK traces, it doesn't replace them.
  • Remote configuration. Adjust sampling, compression and batching from Better Stack without redeploying. Lower the sampling of a legacy service to save on ingesting and egress costs.
  • Databases out of the box. The collector recognizes PostgreSQL, MySQL, Redis, Memcached, MongoDB, Apache, Nginx, Elasticsearch and Kafka, and each comes with a ready-made dashboard.
  • Transform wide events with VRL. Transform logs, spans and other wide events to redact personally identifiable information, or discard useless events so that you don't get billed.

Resource requirements

The collector runs with Docker Compose 1.25.0 or later.

eBPF tracing is enabled by default. For this full tracing setup, keep 4 GiB of memory and 4 vCPUs available on each host. If you disable eBPF tracing, keep 2 GiB of memory and 2 vCPUs available per host.

eBPF tracing needs Linux kernel 5.14 or newer. Check that your host supports it:

Check eBPF support
docker run --rm --privileged alpine:latest sh -c "apk add --no-cache bash wget -q && \
  wget -qO- https://telemetry.betterstack.com/api/collector/public/ebpf.sh | bash"

Check failed? See eBPF requirements.

Install via Docker

Deploy collector with the provided install script:

Install using Docker Compose
curl -sSL https://raw.githubusercontent.com/BetterStackHQ/collector/main/install.sh | \
  COLLECTOR_SECRET="$COLLECTOR_SECRET" bash


Then restart your services so the eBPF instrumenters can attach correctly.

Upgrading collector

Rerun the install command to upgrade collector:

Upgrade collector
curl -sSL https://raw.githubusercontent.com/BetterStackHQ/collector/main/install.sh | \
  COLLECTOR_SECRET="$COLLECTOR_SECRET" bash


Uninstalling collector

Something wrong? Let us know at hello@betterstack.com.

Uninstall collector
curl -sSL https://raw.githubusercontent.com/BetterStackHQ/collector/main/uninstall.sh | bash

Install to Docker Swarm

Deploy collector to each node in your Swarm cluster using the following script:

Deploy to all Swarm nodes
curl -sSL https://raw.githubusercontent.com/BetterStackHQ/collector/refs/heads/main/deploy-to-swarm.sh | \
    MANAGER_NODE=root@swarm-manager COLLECTOR_SECRET="$COLLECTOR_SECRET" bash


Then restart your services so the eBPF instrumenters can attach correctly.

Deploying to specific Docker Swarm nodes

To restrict the Collector to selected Swarm nodes, add the better-stack.collector=true label to those nodes from a swarm manager. Nodes without the label will stop running the Collector.

Managing Collector placement in Docker Swarm
# Enable Collector on a specific node
docker node update --label-add better-stack.collector=true <node-name>

# Disable Collector on a node
docker node update --label-rm better-stack.collector <node-name>

# If needed, remove the collector host container from the node directly
docker stop better-stack-collector-host && docker rm better-stack-collector-host

Upgrading collector in Docker Swarm

Rerun the deploy command to upgrade collector:

Upgrade collector in Docker Swarm
curl -sSL https://raw.githubusercontent.com/BetterStackHQ/collector/refs/heads/main/deploy-to-swarm.sh | \
    MANAGER_NODE=root@swarm-manager COLLECTOR_SECRET="$COLLECTOR_SECRET" bash


Getting container name is already in use? Force the upgrade. It removes the existing collector containers before upgrading collector:

Force upgrade in Docker Swarm
curl -sSL https://raw.githubusercontent.com/BetterStackHQ/collector/refs/heads/main/deploy-to-swarm.sh | \
    ACTION=force_upgrade \
    MANAGER_NODE=root@swarm-manager COLLECTOR_SECRET="$COLLECTOR_SECRET" bash


Uninstalling collector from Docker Swarm

Uninstall collector from Docker Swarm
curl -sSL https://raw.githubusercontent.com/BetterStackHQ/collector/refs/heads/main/deploy-to-swarm.sh | \
    MANAGER_NODE=root@swarm-manager COLLECTOR_SECRET="$COLLECTOR_SECRET" ACTION=uninstall bash


Using labels with Docker logs

Attach container labels to your logs with Docker's json-file log driver. List the label keys to attach in /etc/docker/daemon.json:

/etc/docker/daemon.json
{
  "log-driver": "json-file",
  "log-opts": {
    "labels": "com.example.team,environment"
  }
}

Then apply the change:

  1. Restart the Docker daemon at a time your services can be briefly unavailable:

    Restart Docker
    systemctl restart docker
    
  2. Recreate your containers. Docker applies log options only to containers created after the restart.

The labels show up on logs in the attrs object.

Collect additional logs

Send any log files from the host filesystem to Better Stack. For example, add the following configuration to your manual.vector.yaml file to send the logs from /var/www/custom.log and any file matching /var/www/**/service.*.log:

Sending custom logs
sources:
  better_stack_logs_custom_file:
    type: file
    read_from: beginning
    ignore_older_secs: 259200 # send only log lines newer than 3 days
    include:
      - "/host/var/www/custom.log"
      - "/host/var/www/**/service.*.log"

Any source with a name starting with better_stack_logs_ is forwarded to the logs sink.

Mount fewer paths into the collector container

Collector mounts the root filesystem read-only into the collector container by default. You can mount only select paths into the container:

Docker Compose Docker Swarm
curl -sSL https://raw.githubusercontent.com/BetterStackHQ/collector/main/install.sh | \
  COLLECTOR_SECRET="$COLLECTOR_SECRET" \
  MOUNT_HOST_PATHS="/var/www/custom,/var/lib/something" bash
curl -sSL https://raw.githubusercontent.com/BetterStackHQ/collector/refs/heads/main/deploy-to-swarm.sh | \
    MANAGER_NODE=root@swarm-manager COLLECTOR_SECRET="$COLLECTOR_SECRET" \
    MOUNT_HOST_PATHS="/var/www/custom,/var/lib/something" bash


Selected paths will be mounted read-only under the /host prefix, e.g. /host/var/www/custom.

Required permissions

The ebpf container requires privileged access as it runs the eBPF code responsible for generating traces. The collector container doesn't request privileged access.

Need help?

Please let us know at hello@betterstack.com.
We're happy to help! 🙏