# How to Run Fluentd in Docker Within the Internal Network

Running Fluentd in Docker within an internal network is a common setup for aggregating and processing logs in containerized environments. To achieve this, you need to configure Docker networking so Fluentd can communicate with other containers and services within your internal network.

### **Step-by-Step Guide to Running Fluentd in Docker Within an Internal Network**

### **1. Create a Docker Network**

First, create a Docker network for internal communication:

```bash
docker network create fluentd-network
```

This creates a network named `fluentd-network` that you can use to connect containers.

### **2. Create a Fluentd Configuration File**

Prepare a Fluentd configuration file (`fluentd.conf`) with your desired settings. For example:

```
<source>
  @type forward
  port 24224
  tag docker.*
</source>

<filter docker.**>
  @type record_transformer
  <record>
    hostname ${hostname}
    environment production
  </record>
</filter>

<match docker.**>
  @type stdout
</match>
```

This configuration listens for logs on port `24224` and prints them to the standard output.

### **3. Create a Dockerfile for Fluentd**

If you want to build a custom Docker image for Fluentd, create a `Dockerfile`:

```
FROM fluent/fluentd:v1.14-1

USER root
COPY fluentd.conf /fluentd/etc/fluentd.conf

RUN gem install fluent-plugin-record-transformer
RUN gem install fluent-plugin-rewrite-tag-filter

USER fluent
```

This Dockerfile uses the Fluentd base image, copies your configuration file into the container, and installs necessary plugins.

### **4. Build the Docker Image**

Build the Fluentd Docker image:

```bash
docker build -t my-fluentd-image .
```

### **5. Run Fluentd Container**

Run the Fluentd container on the internal network:

```bash
docker run -d --name fluentd \\
  --network fluentd-network \\
  -p 24224:24224 \\
  -v /path/to/fluentd.conf:/fluentd/etc/fluentd.conf \\
  my-fluentd-image
```

### **6. Run Other Containers on the Same Network**

Ensure other containers that need to send logs to Fluentd are also connected to the same network.

For example, to run a container with Fluentd as the logging driver:

```bash
docker run -d --name my-app \\
  --network fluentd-network \\
  --log-driver=fluentd \\
  --log-opt fluentd-address=fluentd:24224 \\
  my-app-image
```

### **7. Verify Connectivity**

To ensure that Fluentd and other containers can communicate:

1. **Check Fluentd Logs**: Verify that Fluentd is receiving logs:
    
    ```bash
    docker logs fluentd
    ```
    
2. **Check Container Logs**: Ensure the application container is sending logs:
    
    ```bash
    docker logs my-app
    ```
    
3. **Network Inspection**: Inspect the Docker network to verify connected containers:
    
    ```bash
    docker network inspect fluentd-network
    ```
    

### **8. Access Fluentd from Other Services**

If you have services outside of Docker containers that need to send logs to Fluentd, ensure they are configured to send logs to the host's IP address or domain name on the Fluentd port (e.g., `localhost:24224`).

### **9. Example of Using Fluentd with Docker Compose**

For a more integrated setup, you can use Docker Compose to manage Fluentd and other services.

**docker-compose.yml**

```yaml
version: '3'
services:
  fluentd:
    image: my-fluentd-image
    networks:
      - fluentd-network
    ports:
      - "24224:24224"
    volumes:
      - ./fluentd.conf:/fluentd/etc/fluentd.conf

  my-app:
    image: my-app-image
    networks:
      - fluentd-network
    logging:
      driver: "fluentd"
      options:
        fluentd-address: "fluentd:24224"

networks:
  fluentd-network:
    driver: bridge

```

In this setup, Docker Compose handles network creation and container management.

### **Summary**

- **Create a Docker Network**: Establish a network for internal communication.
- **Configure Fluentd**: Set up Fluentd configuration and Dockerfile.
- **Run Containers**: Start Fluentd and other containers on the same network.
- **Verify**: Ensure containers can communicate and logs are processed correctly.
- **Optional**: Use Docker Compose for a simplified setup.