Heroku Logging Tools in 2024

Heroku is a container-based cloud Platform-as-a-Service used to deploy, manage and scale modern applications. It was originally meant for the Ruby on Rails web application framework. Now it offers support for the most populars programming languages such as Java, Python, or Node.js.

Basic Terms

Dynos - Heroku runs applications in dynos, virtual machines that are scaled depending on the needs of your application. If you need to process more data or run more complex applications, you simply get more dynos. Heroku charges you a monthly fee, based on the number of dynos you use and their sizes.

Heroku Buildpack - Buildpacks are sets of open-source scripts that help you to compile apps. They allow you to code in languages and frameworks that suit the needs of your project the most. Buildpacks determine the process for each app, which assets and runtimes should be made available etc. You can either create a custom buildpack or choose from over 7 thousand third-party buildpacks available.

Heroku add-ons - Add-ons are components supporting your application. It could be data storage and processing, monitoring, analytics, and many more. They are maintained either by Heroku or a third-party developer and help you to focus on your project instead of spending focus and capacity on supporting services. Most add-ones offer multiple plans, different features and options, and pricing models.

Heroku CLI - CLI stands for Command Line Interface. It allows you to create and manage your apps directly from the terminal.

Logging on Heroku

)

Logs are aggregated from the output streams of all of its running processes, system components, and backing services. Heroku uses its own Logplex to collate and distribute log entries and makes these entries available via the Logplex API, and the command-line tool.

Types of Logs

You can collect either Runtime or Build Logs.

Runtime logs come from the deployed app and are split into these categories:

• App logs
• System logs
• API logs
• Add-on logs.

Build logs are generated while building and deploying your app and are separate from the app's runtime logs.

Why should you get a Logging Add-On?

Heroku’s Logplex router collates and distributes the log entries generated within your Heroku platform. Logplex tries to bring order to your logs in an accessible way. However, it is designed to keep only the most recent 1500 lines of your consolidated logs, which expire after 7 days. This seriously limits the benefits you can get from logging since you can’t overview your logs in time.

You can always learn more from Heroku's official documentation.

Heroku Logging Add-ons in 2023

Having the best practices and the importance of having a Logging solution for Heroku, let’s take a look at the add-ons available for Heroku in 2023.

1. Better Stack

Better Stack allows you to query your logs the same way you'd query your database with SQL-compatible structured log management. Thanks to a custom-built pipeline and ClickHouse you can search & filter petabytes of logs in a moment and set an anomaly detection alert to receive alerts when your logs become out of the ordinary. On top of that, thanks to Better Stack's efficiency you save funds at the same time.

Presence/Absence alerts help you to overview your system, and a custom alert can be triggered on most of your favorite platforms. Better Stack also offers state-of-the-art security thanks to secure transit and storage in ISO/IEC 270001 certified data centers.

Logs can be commented on, fragmented & archived, and shared with your teammates for seamless cooperation.

Better Stack supports CLOJURE, GO, Java, Node, Scala, Ruby, Python, and PHP.

Main benefits of Better Stack:

• Integration with Better Stack Uptime add-on for complex Heroku logging and monitoring solution
• Free subscription available
• Custom pipeline and data management are both cost and resource-efficient

2. LogDNA

LogDNA parses major log line types on ingestion and offers Custom Parsing Templates. You can filter your logs based on app, host, or cluster, browse logs from any source instantly, and search through them with simple keywords, exclusion terms, chained expressions, and data ranges. Alerts are set off based on either Presence or Absence, or generate an alert from a saved View and report on them in PagerDuty, Slack, or with a custom Webhook. LogDNA also allows you to save views to access common Filters and Searches and share them.

LogDNA is built on Elasticsearch, providing you with relatively fast and reliable indexing and filtering of your logs. A web-based GUI handles filtering, logs grouping by source, and more. Visualization and custom dashboards are also available, and you can work with user-specific logs. LogDNA offers a Heroku Template. This template allows you to auto-populate Custom Parsing, Views, Boards, and Screens curated for Heroku logs.

Main benefits of LogDNA:

• A well-designed web app makes it easy to navigate through your logs
• Free subscription available
• LogDNA's Live Tail offers a real-time stream into your project's logs

3. Sumo Logic

Sumo Logic offers a log management solution available on Cloud, on-premises, or in hybrid deployments. Thanks to log centralization, you can create multiple dashboards and overview your monitors in one place, allowing you to correlate issues and perform benefit from a machine learning-boosed root-cause analysis.

Sumo Logic is also a security and compliance management. Thanks to its log analysis capabilities, it offers fast threat detection and also helps you detect breaches, and leverage other threat intelligence tools.

Thanks to anomaly detection, outlier detection, and predictive analytics, you get deep and comprehensive insights into your architecture's performance. Sumo logic offers real-time visibility into AWS, Azure, and GCP cloud applications and infrastructure. Alongside that, you get access to over 150 apps and native integrations to get full out-of-the-box visibility into third-party technologies.

Sumo Logic provides you with two dashboards - a live dashboard and an interactive one. The live dashboard offers numerous real-time data in the order as they come. However, it doesn't provide an option to look back at the older data. That's where the interactive dashboard comes in. In the interactive dashboard, you can view a complete overview of events and trends, focus on the graphs and identify rare events. You can filter for the specific errors and exceptions to be able to focus on them in the future.

Main benefits of Sumo Logic:

• Machine Learning powered Anomaly Detection
• Free subscription available
• easy, search engine-like syntax

4. Papertrail (SolarWinds)

Papertrail is quite popular. It allows for faster diagnostic and monitoring. Logs are collected, visualized, and analyzed. You can identify patterns or anomalies, spot potentially dangerous trends, or resolve occurring errors.

Papertrail also allows you to tail and search logs in real-time. Papertrail comes from a wide portfolio of services. For example, with AppOptics, you can reach absolute end-to-end visibility and analyze traces, metrics, and logs without losing context.

In this case, integration with Papertrail is rather seamless, thanks to the add-on installation. Overall, Papertrail is verified by time and its users. However, for some, it seems old-fashioned and not really user-friendly. Also, one of the main downsides is certainly an unfriendly subscription model.

Main benefits of Papertrail:

• Papertrail comes from a portfolio of a huge software enterprise
• Papertrail supports multiple approaches to log management

5. Logentries

Logentries add-on streams logs in real-time with pre-configured search queries, tags, alerts, and dashboards, meaning that you can start working with your logs instantly. Logentries query builder allows you to query your logs and calculate values including COUNT, SUM, AVERAGE, and more.

Logentries' pre-built dashboards allow for the visualization and better overview of your logs. If pre-built dashboards do not suit your needs, you can always create your own.

You can create alerts based on either presence or absence and look for any anomalies. You can receive alerts directly in the email, webhooks, or your favorite platforms such as Slack, Pagerduty, and more.

Main Benefits of Logentries:

• Thanks to Logentries and New Relic integration, you can combine log data and application performance metrics
• Logentries requires zero setup
• Their free subscription offers up to 5GBs of data per month

6. Coralogix Logging

Coralogix enables you to centralize logs, metrics, and security data from all over your stack. After ingestion, data are analyzed, and Coralogix provides you with common trends and patterns that make a further prediction or issues analysis easier. Coralogix automatically parses your logs and enriches the data using their parsing wizard, an automatic parsing for JSON logs. You can enrich logs using a pre-defined, custom data source and add important business, operation, or security information.

Coralogix benefits from Machine Learning and Anomaly Detection. ML algorithms help discover any abnormalities without the need for pre-setting thresholds or other rules. Visualization is made easier thanks to their UI, Kibana, Grafana, SQL clients, Tableau or CLI, and APIs. Coralogix also offers support for multiple syntaxes, including ELK syntax.

Coralogix is an enterprise-ready solution thanks to GDPR, SOC2, PCI, and HIPAA certifications.

Coralogix’s pricing is based on a per GB price and starts at $0.60 for Monitoring Data, $1.80/GB of Frequently Searched Data, and $0.22/GB for compliance data.

Main Benefits of Coralogix:

• Enteprise-ready solution thank to compliance with multiple security standards

Heroku Logging Best Practices

Heroku’s Devcenter defines the following best practices for Application Logs:

1. Define which events to log

Each log brings a different piece of context. You should always log the following:

• App errors
• Input and output validation failures
• Authorization failures
• Session management failures
• Privilege elevation successes and failures
• Higher-risk events such as Data imports and exports
• Logging initialization
• Opt-ins such as terms of service

You should also consider logging other events, that can come in handy when:

• Troubleshooting
• Monitoring and Improving Performance
• Testing
• Monitoring User Behavior
• Security and auditing

2. Include Pertinent Details

It is great to include details to get a broader context. Heroku automatically adds to your log output a timestamp and the source (dyno) from where a log message came. Adding additional context such as:

• What action was performed
• Who performed the action
• Why an issue occurred
• If available, remediation information for WARN and ERROR messages
• HTTP request ID

3. Exclude sensitive information

Some things seem like a no-brainer, but one can never be careful enough. In order to prevent unnecessary risks in the security of your app:

• Do not emit sensitive Personal Identifiable Information (PII)
• Do not emit encryption keys and other confidential information to your logs
• Ensure that your company’s privacy policy includes your logs
• Ensure that your logging add-on provider meets your security standards
• Ensure that you meet data residency requirements

4. Choose a structure for your logs

Structured logs are easier to parse, query, and process for analytics. Heroku’s default structure of log entries is called logfmt, which is a human-readable and easy to parse format. Always review the supported formats that your add-on provider can automatically parse.

5. Attribute correct levels to your logs

You need to give your logs a level of importance in order to benefit from your logging add-on the most.

Log levels are:

• INFO - Messages containing information, these messages do not indicate any fault or error.
• WARN - These logs indicate, that there is a potential problem, which does not affect users.
• ERROR - Logs containing information about a serious problem, often interfering with user experience
• FATAL - Logs inform about fatal errors, often significantly influencing the performance of your application
• DEBUG - Used for debugging.

Conclusion

We've briefly introduced Heroku, what it is and how it helps developers on a daily basis. Then we've focused on the basics of logging and log collecting, followed by a list of Heroku Logging Add-ons. It is worth mentioning that there are other ways of collecting logs from Heroku, for example, using the HTTPS Log Drain app. Thanks to the fact that Heroku is very well documented, picking a logging solution best matching your needs should be a piece of cake.