# What Is the Format of Logstash Config File

The Logstash configuration file (`.conf`) is structured to define how Logstash processes and transforms data. It consists of three main sections: **input**, **filter**, and **output**. Each section is responsible for a different stage of the data pipeline.

### Basic Structure of Logstash Configuration File

```
input {
  # input plugins configuration
}

filter {
  # filter plugins configuration (optional)
}

output {
  # output plugins configuration
}
```

### 1. **Input Section**

The **input** section specifies the source(s) from which Logstash will receive the data. You can use multiple input plugins like `file`, `tcp`, `http`, etc.

**Example (input from a file)**:

```
input {
  file {
    path => "/var/log/syslog"
    start_position => "beginning"
  }
}
```

- **`path`**: Specifies the file path to read data from.
- **`start_position`**: Defines whether to start reading the file from the beginning or from the end (for newly appended logs).

### 2. **Filter Section**

The **filter** section is used for data transformation and parsing. Common filters include `grok`, `mutate`, `date`, and `geoip`. Filters allow you to manipulate, parse, and enrich the data before sending it to the output.

**Example (filter using Grok)**:

```
filter {
  grok {
    match => { "message" => "%{SYSLOGTIMESTAMP:timestamp} %{WORD:logsource} %{GREEDYDATA:logmessage}" }
  }

  date {
    match => [ "timestamp", "MMM  d HH:mm:ss", "MMM dd HH:mm:ss" ]
    target => "@timestamp"
  }

  mutate {
    remove_field => [ "timestamp" ]
  }
}
```

- **`grok`**: Used for pattern matching to parse unstructured log data into structured data.
- **`date`**: Used to parse date fields and set the `@timestamp` field to match the parsed time.
- **`mutate`**: Can be used to modify fields, such as renaming, removing, or converting field types.

### 3. **Output Section**

The **output** section defines where the data should be sent. Outputs can include destinations like Elasticsearch, a file, a database, or another system.

**Example (output to Elasticsearch)**:

```
output {
  elasticsearch {
    hosts => ["<http://localhost:9200>"]
    index => "logstash-%{+YYYY.MM.dd}"
  }

  stdout { codec => rubydebug }  # Output to console for debugging
}
```

- **`hosts`**: Specifies the address of your Elasticsearch instance.
- **`index`**: Defines the index pattern for storing the logs in Elasticsearch (with dynamic date-based indices).
- **`stdout`**: Sends output to the console, using the `rubydebug` codec to make it human-readable (useful for debugging).

### Example of a Complete Logstash Configuration File

```
input {
  file {
    path => "/var/log/syslog"
    start_position => "beginning"
  }
}

filter {
  grok {
    match => { "message" => "%{SYSLOGTIMESTAMP:timestamp} %{WORD:logsource} %{GREEDYDATA:logmessage}" }
  }

  date {
    match => [ "timestamp", "MMM  d HH:mm:ss", "MMM dd HH:mm:ss" ]
    target => "@timestamp"
  }

  mutate {
    remove_field => [ "timestamp" ]
  }
}

output {
  elasticsearch {
    hosts => ["<http://localhost:9200>"]
    index => "logstash-%{+YYYY.MM.dd}"
  }

  stdout { codec => rubydebug }
}
```

### Additional Notes:

- **Multiple Inputs, Filters, and Outputs**: Logstash allows multiple input, filter, and output blocks within a single configuration file. Each block will be processed independently.
- **Conditionals**: You can use conditionals to apply specific filters or outputs based on certain criteria:
    
    ```
    filter {
      if [logsource] == "apache" {
        grok { ... }
      }
    }
    ```
    

### Conclusion

The Logstash configuration file is highly flexible and modular, allowing you to specify multiple data sources, apply complex transformations, and send data to various outputs. The `input`, `filter`, and `output` sections are the core elements that define how Logstash processes data from start to finish.