# Logstash: How to Include Input File Line Number

To include the line number of the input file in Logstash, you need to use a combination of Logstash filters and plugins. Logstash itself does not natively provide a line number for each log entry, but you can achieve this functionality with the help of the `logstash-input-file` plugin and custom configuration.

Here’s a step-by-step guide on how to include the line number of the input file in Logstash:

### **1. Use the `logstash-input-file` Plugin**

The `logstash-input-file` plugin is used to read from log files. This plugin can be configured to track the line numbers.

**Example Logstash Configuration:**

```yaml
input {
  file {
    path => "/path/to/your/logfile.log"
    start_position => "beginning"
    codec => multiline {
      pattern => "^%{TIMESTAMP_ISO8601}"
      negate => true
      what => "previous"
    }
    sincedb_path => "/dev/null"
    # Enable this to include the line number
    file_chunk_size => "1mb"
    # or
    # file_sorter => "name"  # Optional, if sorting files
  }
}

filter {
  # Add a field to capture the line number if the log has it
  grok {
    match => { "message" => "%{GREEDYDATA:log_line}" }
  }

  ruby {
    code => '
      # Extract line number if available
      if event.get("message") =~ /^(?<line_number>\\d+)/
        event.set("line_number", $LAST_MATCH_INFO["line_number"])
      end
    '
  }
}

output {
  stdout { codec => rubydebug }
}

```

### **2. Track Line Numbers with `file` Input Plugin**

The `file` input plugin does not provide a line number directly, but you can approximate this by including the line content as a field and then processing it further.

1. **Configure the File Input:**
    - Configure the `file` input plugin to read the log file.
2. **Use a Filter to Add Line Numbers:**
    - Use the `ruby` filter to add line numbers to each event. This involves custom code to parse the log content and add a line number.

**Example with Ruby Filter:**

```yaml
filter {
  ruby {
    code => '
      # Initialize the counter if not already set
      if !defined?(@@line_counter)
        @@line_counter = 0
      end

      # Increment the counter and set the line number
      @@line_counter += 1
      event.set("line_number", @@line_counter)
    '
  }
}

```

### **3. Alternative Approach:**

If you are processing files where line numbers are critical and need to be accurate, consider preprocessing your logs to include line numbers before Logstash reads them. For example, use a script or tool to preprocess the logs and add line numbers.

**Example Preprocessing Script (Python):**

```python
# preprocess_logs.py
import sys

def add_line_numbers(input_file, output_file):
    with open(input_file, 'r') as infile, open(output_file, 'w') as outfile:
        for line_number, line in enumerate(infile, start=1):
            outfile.write(f"{line_number} {line}")

if __name__ == "__main__":
    add_line_numbers(sys.argv[1], sys.argv[2])

```

Run the preprocessing script:

```bash
python preprocess_logs.py input.log output_with_line_numbers.log
```

Then configure Logstash to read from `output_with_line_numbers.log`.

### **Summary**

- **Directly in Logstash:** Use the `file` input plugin and custom filters to approximate line numbers. The `ruby` filter can be used to manually add line numbers if necessary.
- **Preprocess Logs:** Consider preprocessing logs to add line numbers before Logstash reads them for more accuracy.

Choose the method that best fits your needs and environment for including line numbers in your log data.