Is the stub_status module available?
It is included in most nginx packages by default. Verify it with nginx -V 2>&1 | grep -o with-http_stub_status_module.
Better Stack collector automatically recognizes databases and web servers running on your hosts and starts collecting their metrics: PostgreSQL, MySQL, Redis, Memcached, MongoDB, Elasticsearch, Kafka, Apache, Nginx and Traefik. Each recognized database comes with a pre-built dashboard covering database health, slow queries, connections, and queries per second.
Most services need no setup. MySQL and PostgreSQL need a monitoring user, and nginx, Apache and Traefik a status endpoint, before all their metrics arrive.
Create a database user with the following permissions to collect MySQL database metrics:
If the default postgres database does not exist, create it:
Create a database user with the pg_monitor role and enable the pg_stat_statements extension:
On hardened or RDS databases the default public schema grants are often revoked, so the GRANT USAGE ON SCHEMA public statement above is required.
Finally, load pg_stat_statements at startup and restart PostgreSQL. Amazon RDS already loads it, so you're done there.
Enable a status endpoint on nginx, Apache or Traefik so the collector can scrape its metrics.
The collector scrapes nginx metrics from the stub_status module at /nginx_status. Add the following to your nginx configuration:
Reload nginx to apply the change:
stub_status module available?It is included in most nginx packages by default. Verify it with nginx -V 2>&1 | grep -o with-http_stub_status_module.
The collector scrapes Apache metrics from mod_status at /server-status?auto. Enable the module and add the following configuration:
Enable the module and reload Apache:
Set ExtendedStatus On to get the full set of metrics. On Apache 2.3.6 and up, this is already enabled by default.
Enable the Prometheus exporter on Traefik:
After enabling the status endpoint, verify it's reachable from the host:
The process status in Better Stack will automatically change from Configuration required to Active within a few minutes.
The collector automatically discovers your Kafka brokers and collects cluster metadata with no Kafka configuration: broker count, partitions per topic, in-sync replicas (ISR), leader status and under-replication status per partition. These metrics power the Overview and Partitions & replication sections of the Kafka dashboard
Brokers are discovered and reached automatically through the cluster network. Advertise each broker with a fully qualified domain name the cluster can resolve, such as the broker's Kubernetes Service DNS name:
A bare hostname or localhost only resolves inside the broker's own pod, so the collector and other clients elsewhere in the cluster can't reach it.
The collector connects to brokers from the host network. Publish the broker port and advertise a listener host clients can follow:
Containers on the compose network keep connecting to kafka:19092. Then add the broker in Better Stack:
kafka.localhost and Port to 9092. Use the Kafka protocol port, not the JMX exporter port.Kafka keeps broker-level performance metrics in JMX (Java Management Extensions) and doesn't expose them by default. Deploy the Prometheus JMX exporter as a Java agent on each broker to light up the rest of the Kafka dashboard:
The setup is the same everywhere: put the agent JAR and a rules file on each broker, add the -javaagent flag to Kafka's JVM options, and let the collector scrape port 7071.
Save the following configuration next to where the agent JAR will live:
Rules are applied first-match-wins. Keep the specific patterns above the generic catch-alls, otherwise per-topic metric names get mangled.
Fetch the agent in an init container and attach it to the broker container. Mount the rules file to /jmx-exporter/jmx-kafka-config.yaml, for example from a ConfigMap:
Annotate the Kafka pod template and the collector discovers and scrapes the endpoint automatically:
Download the agent JAR next to the rules file:
Mount the folder, attach the agent, and publish the metrics port:
Restart Kafka and verify the endpoint:
Then add the endpoint as a scrape target in Better Stack:
kafka. The shared name groups both metric sources under one service.http://localhost:7071/metrics.The collector scrapes the endpoint from the host network every 30 seconds. That's why port 7071 is published.
Kafka CLI tools, including Docker healthchecks and Kubernetes probes calling scripts like kafka-broker-api-versions.sh, inherit KAFKA_OPTS and crash trying to attach a second agent to the busy port. Clear the variable for CLI invocations:
Within a few minutes, metrics appear in your collector source, and the scrape target shows as Active in your collector's Configure → Collect metrics. Check out the Kafka dashboard to see your broker throughput, controller state, partition health, and storage in one place.
Please let us know at hello@betterstack.com.
We're happy to help! 🙏
We use cookies to authenticate users, improve the product user experience, and for personalized ads. Learn more.