How to Retrieve Unique Count of a Field Using Kibana + Elastic Search

To retrieve the unique count of a field using Kibana and Elasticsearch, you can use the "Cardinality Aggregation" in Kibana's interface. This allows you to calculate the unique values of a specified field from the documents in your Elasticsearch index.

Here’s a step-by-step guide:

1. Using Kibana Visualizations

Step 1: Open the Kibana Dashboard

• Navigate to Kibana and open the "Visualize" tab in the left-hand menu.

Step 2: Create a New Visualization

• Click on the "Create Visualization" button and select a visualization type that supports metrics, such as:
  • Data Table
  • Metric
  • Bar Chart

Step 3: Configure the Aggregation

• Once the visualization type is selected, you’ll configure the metric aggregation.
• In the "Metrics" section of the visualization editor:
  1. Select "Metric Aggregation" or similar.
  2. Choose "Unique Count" (Cardinality).
  3. In the "Field" dropdown, select the field for which you want to calculate the unique count.
  4. Adjust any additional options like time ranges or filters as necessary.

Example: Creating a Metric Visualization

• Metric Aggregation: Select "Cardinality" (which provides the unique count).
• Field: Choose the field you want to retrieve the unique count for (e.g., user_id, ip_address, etc.).

Step 4: View the Results

• Once you’ve configured the aggregation, Kibana will show the unique count for the selected field.
• You can also save the visualization and add it to your Kibana dashboard for future use.

2. Using Kibana Dev Tools (Elasticsearch Query)

If you prefer using Elasticsearch queries directly, you can retrieve the unique count of a field using the Cardinality Aggregation in the Dev Tools console in Kibana.

Step 1: Open Dev Tools

• In the Kibana sidebar, click on "Dev Tools".

Step 2: Execute the Query

• You can run the following Elasticsearch query to retrieve the unique count of a specific field using the Cardinality aggregation.

GET /your_index/_search
{
  "size": 0,
  "aggs": {
    "unique_count_of_field": {
      "cardinality": {
        "field": "your_field_name"
      }
    }
  }
}

• Replace:
  • /your_index/ with the name of your Elasticsearch index.
  • "your_field_name" with the field for which you want to calculate the unique count.

Step 3: View the Results

• The query returns the unique count under the aggregations section. For example, the response will look like:

{
  "took": 12,
  "timed_out": false,
  "_shards": {
    "total": 5,
    "successful": 5,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": {
      "value": 1000,
      "relation": "eq"
    },
    "max_score": null,
    "hits": []
  },
  "aggregations": {
    "unique_count_of_field": {
      "value": 250  // This is the unique count of the field
    }
  }
}

3. Filters and Time Ranges

In both Kibana Visualizations and Dev Tools queries, you can add filters to focus on specific time periods or criteria. For example:

• If you are working in Kibana Visualizations, select a time range from the time picker.
• In Dev Tools, you can add a query filter to your Elasticsearch query:

GET /your_index/_search
{
  "query": {
    "range": {
      "timestamp": {
        "gte": "now-7d/d",
        "lt": "now/d"
      }
    }
  },
  "size": 0,
  "aggs": {
    "unique_count_of_field": {
      "cardinality": {
        "field": "your_field_name"
      }
    }
  }
}

This query filters the data for the last 7 days (now-7d/d to now/d) and calculates the unique count of the specified field.

Conclusion:

You can retrieve the unique count of a field in Kibana using:

1. Visualizations: By configuring a visualization with a Cardinality Aggregation.
2. Dev Tools: By running an Elasticsearch query that uses the Cardinality Aggregation directly.

Both methods provide flexible ways to analyze your data and get insights into unique values within your Elasticsearch indices.

Got an article suggestion? Let us know

Explore more

Logstash

This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.