# Kibana Logstash Elasticsearch | Unindexed Fields Cannot Be Searched

The error **"Unindexed fields cannot be searched"** in Kibana typically occurs when you try to search or filter on fields that are not indexed in Elasticsearch. This happens because Elasticsearch, by default, only indexes certain fields for search and aggregation operations. Fields that are not indexed cannot be used for querying or filtering, which is why you encounter this error.

Here are the steps to understand and resolve the issue:

### 1. **Check Index Mapping**

The first step is to check whether the fields you are trying to query are indexed. Fields in Elasticsearch can have different mappings, which define how they are stored and whether they are indexed or not.

You can view the mapping for a specific index using the following API request:

```bash
curl -X GET "localhost:9200/my_index/_mapping"
```

In the response, check the `index` property for each field. If a field is set to `"index": false`, it means that the field is **not indexed**, and therefore it cannot be searched or used in filters.

Example:

```json
"my_field": {
  "type": "text",
  "index": false
}
```

### 2. **Reindex the Data with Proper Mapping**

If you find that the fields you need are not indexed, you'll need to reindex your data with the correct mappings.

### Step 1: Create a New Index with Updated Mapping

Create a new index with the correct mapping for the fields you want to be searchable:

```bash
curl -X PUT "localhost:9200/my_new_index" -H 'Content-Type: application/json' -d'
{
  "mappings": {
    "properties": {
      "my_field": {
        "type": "text",
        "index": true
      }
    }
  }
}
'

```

In this example, `my_field` is explicitly set to `"index": true`, which ensures that it will be indexed and searchable.

### Step 2: Reindex the Old Data

Once the new index is created with the updated mapping, you can reindex your old data into the new index:

```bash
curl -X POST "localhost:9200/_reindex" -H 'Content-Type: application/json' -d'
{
  "source": {
    "index": "my_old_index"
  },
  "dest": {
    "index": "my_new_index"
  }
}
'

```

After reindexing, the fields in the new index will be searchable.

### 3. **Update Field Mapping Dynamically (Elasticsearch 7.x and 8.x)**

Elasticsearch does not allow you to change the mapping of an existing field once the index is created. You either need to reindex the data with the correct mapping (as explained above), or in some cases, you can update dynamic field settings to allow for indexing in the future.

To make sure new fields are indexed, update the dynamic templates (if applicable):

```bash
curl -X PUT "localhost:9200/my_index/_mapping" -H 'Content-Type: application/json' -d'
{
  "dynamic": "strict"
}
'
```

This approach ensures that new fields are indexed automatically.

### 4. **Use Keyword Fields for Text Searching**

If you are trying to search on a `text` field and getting this error, Elasticsearch doesn't index the full-text fields for search by default. Instead, it provides a `.keyword` subfield, which you can use for exact matching.

Example query in Kibana:

```json
{
  "query": {
    "term": {
      "my_field.keyword": "some_value"
    }
  }
}
```

Use `.keyword` if you need to perform an exact match search on text fields.

### 5. **Check Field Capabilities in Kibana**

In Kibana, you can also check whether a field is indexed by going to **Management > Index Patterns**, selecting the relevant index pattern, and seeing the **searchable** property for each field. If the field is not searchable, it means it’s not indexed.

You can also refresh the index pattern in Kibana by clicking on the **refresh** button, which will reload the field list and properties.

### 6. **Set Fields to be Indexed When Ingesting via Logstash**

If you're using Logstash to send data to Elasticsearch, you can control field indexing directly in Logstash by adjusting your output configuration.

Example Logstash output configuration:

```
output {
  elasticsearch {
    hosts => ["localhost:9200"]
    index => "my_index"
    document_type => "_doc"
    template_overwrite => true
    template => "/path/to/template.json"
  }
}
```

In your `template.json`, ensure the fields are set to be indexed:

```json
{
  "mappings": {
    "properties": {
      "my_field": {
        "type": "text",
        "index": true
      }
    }
  }
}
```

### 7. **Use `dynamic_templates` for Auto-indexing**

To make sure that new fields are indexed automatically when they are encountered, you can configure a dynamic template that tells Elasticsearch how to index new fields:

```json
{
  "mappings": {
    "dynamic_templates": [
      {
        "strings_as_keywords": {
          "match_mapping_type": "string",
          "mapping": {
            "type": "keyword",
            "index": true
          }
        }
      }
    ]
  }
}
```

This template ensures that any new string field is indexed as `keyword` and is searchable.

### Summary

If you encounter the **"unindexed fields cannot be searched"** error:

1. **Check the field mapping** using the `_mapping` API to verify whether the field is indexed.
2. **Reindex the data** with updated mappings if necessary.
3. For **exact text searches**, use `.keyword` fields.
4. Use **Logstash templates** or **dynamic templates** to ensure fields are properly indexed when ingesting data.

These steps should resolve the issue and allow you to search on previously unindexed fields in Kibana.