What is a Prometheus target?

In Prometheus, a target is an endpoint or service that Prometheus monitors by scraping metrics. These metrics are exposed in a Prometheus-compatible format, typically over an HTTP or HTTPS endpoint. Targets are configured in the prometheus.yml file and identified by two labels:

• instance: Specifies the target’s address (e.g., 192.168.1.101:9090).
  
• job: Groups targets logically (e.g., web-servers).
  

Configuring targets

Targets can be defined either statically or dynamically:

1. Static configuration
   Targets are explicitly listed in the prometheus.yml file:
   yaml scrape_configs: - job_name: 'web-servers' static_configs: - targets: - '192.168.1.101:8080' - '192.168.1.102:8080'

2. Dynamic service discovery
   Prometheus integrates with platforms like Kubernetes, AWS, or Consul to discover targets automatically.
   Example for Kubernetes:
   yaml scrape_configs: - job_name: 'kubernetes-pods' kubernetes_sd_configs: - role: pod relabel_configs: - source_labels: [__meta_kubernetes_pod_label_app] action: keep regex: 'web'

Inspecting targets

To view the status of targets:
- Open the Prometheus web UI and navigate to Status > Targets.
- Details include:
- State: Whether the target is UP or DOWN.
- Labels: Assigned labels such as job and instance.
- Last Scrape: Timestamp of the last scrape.
- Scrape Duration: Time taken for the scrape.

Troubleshooting

• If a target is DOWN, check its /metrics endpoint and verify it exposes metrics in Prometheus format.
  
• For missing targets:
  
  • Static targets: Confirm the correct IP and port in prometheus.yml.
    
  • Dynamic targets: Verify service discovery and relabeling configurations.
    
• Address label conflicts by using relabeling to modify or filter labels.
  

Best practices

• Use meaningful job_name values (e.g., web-servers or db-cluster) to organize targets logically.
  
• Set appropriate scrape intervals to balance resource usage and data freshness:
  yaml scrape_interval: 30s scrape_timeout: 10s
• Use relabeling to exclude unnecessary targets and optimize resource usage.
  
• Monitor target states and configure alerts for targets that are consistently DOWN.
  

Targets are the foundation of Prometheus monitoring. Proper configuration and management ensure reliable and efficient metric collection from your systems.

Got an article suggestion? Let us know

Explore more

Limitations of Prometheus Labels

Questions

How To Manage Prometheus Counters

Questions

What is the Difference Between a Gauge and a Counter?

Questions

What are Prometheus Labels?

Questions

This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.