Logging is an important part of every application life cycle. Having a good logging system becomes a key feature that helps developers, sysadmins, and support teams to understand and solve appearing problems.
The Python standard library and Django already comes with an integrated logging module that provides basic as well as advanced logging features. Log messages can give helpful information about various events happening behind the scenes.
In this tutorial, you will learn how to:
- Configure and use handlers, loggers, formatters, and filters.
- Use the default logger.
- Log to a Console and a File.
- Use predefined loggers.
Prerequisites
- Python 3.x installed.
- IDE set up for Python.
- Django installed and added to your project.
- Django project created.
Step 1 — Using loggers
Logger is the entry point to the logging system. Each logger has a log level. This log level indicates the severity of the messages the logger will handle. Python defines the following log levels:
- Debug: used for detailed information, typically of interest only when diagnosing problems.
- Info: used to get a confirmation that things are working as expected.
- Warning: used as an indication that something unexpected happened, or indicative of some problem in the near future (e.g. 'disk space low'). The software is still working as expected.
- Error: used in cases of more serious problems the software has not been able to perform some function.
- Critical: used to indicate serious errors which might cause the program to stop running.
Each of the recorded messages is a Log record, which contains valuable data and descriptions about the logged event. This can include things like stack trace or an error code.
Every time the message is given to a logger, the log levels are compared. If the logger determines that the message is to be processed by it, it will pass the message to the handler.
Creating a logger
You need to create a logger. For that, import the particular module and use the now available function:
import logging
logger = logging.getLogger(__name__)
The getLogger()
function obtains or creates an instance of a logger. The
logger instance is identified by name, which is usually __name__
. This refers
to the name of the Python module that contains the logger.
Now you can use the newly created logger to create event messages:
logger.error('Error.')
The message will appear in the console. But because you didn't configure the logging yet, it can be easily overlooked.
On the logger, you can call the following functions for each of the default log levels:
logger.debug()
logger.info()
logger.warning()
logger.error()
logger.critical()
You can also use:
logger.log()
- The log level is set in a parameter as a number. (10, 20, 30, 40, 50)logger.exception()
- Creates anERROR
level logging message and wraps the exception stack frame.
Root logger
You do the configuration in the settings.py
file. Let us edit the default
root
logger:
LOGGING = {
'version': 1,
'disable_existing_loggers': False,
'root': {
'level': 'WARNING',
},
}
You are modifying the default root
logger, assigning your newly created
handler to it, and setting a log level.
By default, you must set a version, let's write '1'. On the following line, you forbid the program to disable any other loggers that exist outside of this configuration.
Creating a logger
You can also create your own logger. Name it django
:
LOGGING = {
'version': 1,
'disable_existing_loggers': False,
'loggers': {
'django': {
'level': os.getenv('DJANGO_LOG_LEVEL', 'INFO'),
'propagate': False,
},
},
}
Under loggers
, you created your logger with the name django
. This logger has
the handler console
assigned to him.
Notice that the level
attribute has been set differently. You changed the
environment variable from default level DEBUG
and now can see all debug
logging, including all database queries.
Attribute propagate
determines if the logged message should also be passed to
the handlers of a higher level (ancestor) loggers.
Default logger root
only displays log records when the DEBUG
is set to
TRUE
.
Step 2 — Using handlers
The handler is the thing that determines what happens to each message. It can be configured to some particular behaviour, like recording an event to a console or a file.
Handlers like loggers also have a log level. If the message's log level doesn't meet the requirements of the handler, the message will be ignored. Loggers can have more than one handler. Because of that, the messages can be logged to multiple places.
We can create our handler the following way:
LOGGING = {
'version': 1,
'disable_existing_loggers': False,
'handlers': {
'console': {
'class': 'logging.StreamHandler',
},
},
'loggers': {
'django': {
'handlers': ['console'],
'level': os.getenv('DJANGO_LOG_LEVEL', 'INFO'),
'propagate': False,
},
},
}
With this you created a new handler named console
. The only attribute you need
is class
. Two of the basic classes you can set are StreamHandler
and
FileHandler
. Let us use the first one to log the message to the console. After
that, assign the handler to your logger.
Logging to a file
You can log into a file in a similar fashion. Let us modify our handler:
LOGGING = {
'version': 1,
'disable_existing_loggers': False,
'handlers': {
'file': {
'level': 'DEBUG',
'class': 'logging.FileHandler',
'filename': './your/path/debug.log',
},
},
'loggers': {
'django': {
'handlers': ['file'],
'level': 'DEBUG',
'propagate': True,
},
},
}
You changed the class attribute from the StreamHandler
to the FileHandler
,
indicating that you want to log to a file. Now you need to add a filename where
the logs should be stored. Be sure to add your own. You might need to create the
directory manually. After that change the log level of your logger from INFO
to DEBUG
.
Now you can look at the content of the log file yourself. Because we assigned
the logger name django
, which is a special and reserved name for the logger,
we can see all info in the file, which will look like this:
File C:\\Users\\User\\AppData\\Local\\Packages\\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\\LocalCache\\local-packages\\Python39\\site-packages\\django\\contrib\\sessions\\apps.py first seen with mtime 1618946321.2354982
Step 3 — Formatting
In the end, the log record needs to be projected as text. For this, you will use formatters that, as the name suggests, describes the format of the rendered text.
Create a new formatter and assign it to a handler:
LOGGING = {
'formatters': {
'verbose': {
'format': '{levelname} {asctime} {module} {process:d} {thread:d} {message}',
'style': '{',
},
},
'version': 1,
'disable_existing_loggers': False,
'handlers': {
'file': {
'level': 'DEBUG',
'class': 'logging.FileHandler',
'filename': './your/path/debug.log',
'formatter': 'verbose',
},
},
'loggers': {
'django': {
'handlers': ['file'],
'level': 'DEBUG',
'propagate': True,
},
},
}
Have a look at the file, and you can see that the format of the logged messages changed:
DEBUG 2021-04-21 18:25:31,286 autoreload 4532 6260 File C:\\Users\\User\\AppData\\Local\\Packages\\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\\LocalCache\\local-packages\\Python39\\site-packages\\django\\db\\migrations\\state.py first seen with mtime 1618946321.7684982
Step 4 — Filtering
Using filters, you can add additional requirements when the message will be passed to a handler from a logger. By default, any message passing criteria will be given to a handler. Using filters, you can allow only some messages meeting additional requirements to be handled.
You can also choose to downgrade the log level of a message if the requirements are met.
You can use filters to choose which messages you want to see additionally. Set a
filter that requires attribute settings.DEBUG
to be set to FALSE
:
'filters': {
'require_debug_false': {
'()': 'django.utils.log.RequireDebugFalse',
},
},
And then add it to your handler:
'handlers': {
'file': {
'level': 'DEBUG',
'filters': ['require_debug_false'],
'class': 'logging.FileHandler',
'formatter': 'allInfo',
'filename': './your/path/debug.log',
},
},
Now the handler will ignore messages when the DEBUG
attribute is set to
FALSE
.
You can similarly ask to require the attribute DEBUG
to be set to TRUE
:
'filters': {
'require_debug_true': {
'()': 'django.utils.log.RequireDebugTrue',
},
},
Step 5 — Using logging extensions
Django provides several built-in loggers.
django
Logger named django you used in the article is the catch-all logger. No messages are posted using this name.
django.request
Log messages that are related to the handling of requests. 5XX responses are
raised as ERROR
messages, while 4XX responses are projected as WARNING
messages.
Messages that are assigned to this logger have additional content:
status_code
: the HTTP response code.request
: the request object that generated the logging message.
django.server
Log messages related to the handling of requests received by the server and
invoked by the runserver
command. 5XX responses are raised as ERROR
messages, while 4XX responses are projected as WARNING
messages. Messages that
are assigned to this logger have additional content:
status_code
: the HTTP response code.request
: the request object that generated the logging message.
django.template
Log messages that are related to the rendering of templates.
django.db.backends
Log messages that are related to the interaction of code with a database.
Messages that are assigned to this logger have additional content:
duration
: time it took to execute an SQL statement.sql
: SQL statement executed.params
: parameters used in an SQL call.
This logging is only enabled when the DEBUG
attribute is set to TRUE
.
django.db.backends.schema
Logs the SQL queries that are processed during schema changes to the database using the migrations framework.
django.security.*
This logger will receive security-related errors. There is a sub-logger for each type of security error.
Logs assigned to the django.security.*
are not logged to the django.request
.
Conclusion
You have just learned the basics of logging in Python using Django. The logging module included with Python and Django from the start is considered to be good and sufficient. You can use it in a small project or go into an advanced category by creating your classes and levels.
If you have not been using logging into your project, now you have good basics to start. Logging can speed up your development in a big way when done right.
dashboards with Grafana.
dashboards with Grafana.