How to Stop Logstash From Creating a Default Mapping in Elasticsearch

When Logstash sends data to Elasticsearch, it can automatically create indices and mappings for new data types. However, this can lead to issues, especially if the default mapping doesn’t fit your data model. To prevent Logstash from creating a default mapping, you can use the following strategies:

1. Use Explicit Mappings in Elasticsearch

Before sending data from Logstash to Elasticsearch, you can predefine your index and its mappings. This way, when Logstash sends data to Elasticsearch, it will use the predefined mapping instead of creating a default one.

Steps to Create Explicit Mappings:

1. Create an Index with Mappings: Use the Elasticsearch API to create an index with the desired mappings.

    

   Copied!

   PUT /your_index_name
   {
     "mappings": {
       "properties": {
         "field1": { "type": "text" },
         "field2": { "type": "keyword" },
         "field3": { "type": "date" }
       }
     }
   }
   
   

2. Configure Logstash to Use the Existing Index: In your Logstash configuration, specify the existing index in the output section.

    

   Copied!

   output {
     elasticsearch {
       hosts => ["<http://localhost:9200>"]
       index => "your_index_name"  # Use the predefined index
     }
   }
   
   

2. Disable Dynamic Mapping

If you prefer to avoid creating a default mapping altogether, you can disable dynamic mapping for your indices. This requires you to set the dynamic property to false in your index settings.

Steps to Disable Dynamic Mapping:

1. Create an Index with Dynamic Mapping Disabled:

    

   Copied!

   PUT /your_index_name
   {
     "mappings": {
       "dynamic": "false",  # Disable dynamic mapping
       "properties": {
         "field1": { "type": "text" },
         "field2": { "type": "keyword" },
         "field3": { "type": "date" }
       }
     }
   }
   
   

2. Modify Logstash Configuration: Ensure that your Logstash output points to this index.

    

   Copied!

   output {
     elasticsearch {
       hosts => ["<http://localhost:9200>"]
       index => "your_index_name"
     }
   }
   
   

3. Use Index Templates

Index templates allow you to define settings and mappings that will be applied to indices that match a specified pattern. This is particularly useful when you have multiple indices and want to maintain consistent mappings.

Steps to Create an Index Template:

1. Create an Index Template:

    

   Copied!

   PUT _template/your_template_name
   {
     "index_patterns": ["your_index_name*"],
     "mappings": {
       "properties": {
         "field1": { "type": "text" },
         "field2": { "type": "keyword" },
         "field3": { "type": "date" }
       }
     }
   }
   
   

2. Configure Logstash to Use Matching Indices:

   Make sure your Logstash configuration aligns with the index pattern specified in the template.

    

   Copied!

   output {
     elasticsearch {
       hosts => ["<http://localhost:9200>"]
       index => "your_index_name-%{+YYYY.MM.dd}"  # Pattern matching template
     }
   }
   
   

Conclusion

By following these strategies, you can effectively prevent Logstash from creating default mappings in Elasticsearch. Predefining your index and its mappings, disabling dynamic mapping, and using index templates are all effective methods to maintain control over your data structure and ensure that Elasticsearch indexes your data according to your specifications. This can help improve performance and prevent issues with incorrect data types.

Got an article suggestion? Let us know

Explore more

Elasticsearch Logstash

This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.