How to Control and Manage Systemd Services with Systemctl

Every operating system uses background processes to maintain the system's various operations. In Linux-based systems, these processes are known as daemons, and they are started and maintained by the initialization daemon that is launched immediately after system boot.

In many of the most popular Linux distributions, this initialization daemon is Systemd, so understanding how it works is essential for effective Linux server administration.

This article will teach you the basics of Systemd and the systemctl command, which is used to manage Systemd services.

Prerequisites

To follow through with this tutorial, you will need:

• One Ubuntu 20.04 server that includes a non-root user with sudo access.

Step 1 — Viewing Systemd unit files

Systemd's most important abstraction is the unit, which represents any resource maintained by operating systems such as daemons, sockets, devices, and many others. We will target only a service unit in this section.

Each specific unit of Systemd has its configuration and, typically, multiple configurations with various priorities and scope. These configurations are saved in special files called unit files. Generally, unit files are found in the following three directories (listed in ascending order of priority):

1. /lib/systemd/system: the most general unit files with the lowest priority. You shouldn't modify the files in this directory.
2. /run/systemd/system: the runtime unit configuration. These files change unit behavior at runtime. They are created dynamically and exist only for the actual boot session.
3. /etc/systemd/system: unit files with the highest priority. If you need to change a unit configuration, you will typically edit the files in this directory.

Unit-Specific Configuration

In cases where you only need to edit specific options in a unit file, you will typically create a subdirectory whose name is the unit service with the .d suffix (for example, subdirectory example.d belongs to example unit). This subdirectory will hold one or more files with a .conf suffix that modify and extend the base unit configuration.

You can execute the ls command to list all units in /etc directory that utilize unit-specific configuration:

ls -d /etc/*.d

The -d option restricts the output to only directories, while /etc/*.d specifies that only directories in /etc that end with .d suffix should be presented.

/etc/apparmor.d         /etc/ld.so.conf.d    /etc/rc1.d      /etc/sensors.d
/etc/bash_completion.d  /etc/libibverbs.d    /etc/rc2.d      /etc/sudoers.d
/etc/binfmt.d           /etc/libpaper.d      /etc/rc3.d      /etc/sysctl.d
/etc/cron.d             /etc/logrotate.d     /etc/rc4.d      /etc/tmpfiles.d
/etc/depmod.d           /etc/modprobe.d      /etc/rc5.d      /etc/update-motd.d
/etc/environment.d      /etc/modules-load.d  /etc/rc6.d      /etc/usb_modeswitch.d
/etc/grub.d             /etc/pam.d           /etc/rcS.d
/etc/init.d             /etc/profile.d       /etc/rsyslog.d
/etc/insserv.conf.d     /etc/rc0.d           /etc/sane.d

The above output shows all the unit configuration directories present for each service. For example, notice the /logrotate.d directory for the logrotate service. This is where you can edit, or create a new log rotation setup (see How to Manage Logs with Logrotate on Ubuntu 20.04 for more information).

Structure of Systemd unit files

The structure of a unit file is precisely defined. Each one is divided into sections, and each section consists of directives.

Let's view the contents of a unit file for the rsyslog service in /etc/systemd/system by executing the cat command:

cat /etc/systemd/system/syslog.service

You'll see the contents of the file appear on the screen:

[Unit]
Description=System Logging Service
Requires=syslog.socket
Documentation=man:rsyslogd(8)
Documentation=https://www.rsyslog.com/doc/

[Service]
Type=notify
ExecStart=/usr/sbin/rsyslogd -n -iNONE
StandardOutput=null
Restart=on-failure

# Increase the default a bit in order to allow many simultaneous
# files to be monitored, we might need a lot of fds.
LimitNOFILE=16384

[Install]
WantedBy=multi-user.target
Alias=syslog.service

The output shows definitions of the different sections and their directives for the rsyslog service. Each directive starts at a new line with a keyword followed by a value (for example, Description=System Logging Service). There are three sections in the file above:

• [Unit]: defines the metadata of unit and its relations to the other units.
• [Service]: used for the configuration of the service unit. The most common directive in this section is Type, which tells Systemd how to manage the service and find out its state. Type=notify indicates that the service will notify systemd once it has finished staring up. The ExecStart directive defines the full command that should be executed to start the service.
• [Install]: defines the behavior of a unit after it is enabled. The WantedBy directive specifies the relationship between the unit and other ones, while Alias defines alternative names for the unit (in our case the rsyslog, and syslog are the same Systemd units).

For more information about all other unit sections and their directives, see the Systemd documentation.

Step 2 — Checking the service status with systemctl

The systemctl command is a utility that controls Systemd and its units. This tool allows us to check a unit's status, and disable or enable them as needed. Go ahead and view the list of all available units on your system through the command below:

systemctl list-units --type=service --no-pager

The --type=service option shows only units of type service, and --no-pager prevents the output from being piped to the less pager. The command's output should appear on the screen:

UNIT                        LOAD   ACTIVE SUB     DESCRIPTION
  accounts-daemon.service     loaded active running Accounts Service
  acpid.service               loaded active running ACPI event daemon
  alsa-restore.service        loaded active exited  Save/Restore Sound Card Sta…
  apparmor.service            loaded active exited  Load AppArmor profiles
  apport.service              loaded active exited  LSB: automatic crash report…
  avahi-daemon.service        loaded active running Avahi mDNS/DNS-SD Stack
  bluetooth.service           loaded active running Bluetooth service
● certbot.service             loaded failed failed  Certbot
  clean-mount-point@media-ju… loaded active exited  Clean the /media/juraj/Exte…
  rsyslog.service             loaded active running System Logging Service
  colord.service              loaded active running Manage, Install and Generat…
  console-setup.service       loaded active exited  Set console font and keymap
  cron.service                loaded active running Regular background program …
  cups-browsed.service        loaded active running Make remote CUPS printers a…
  cups.service                loaded active running CUPS Scheduler
  dbus.service                loaded active running D-Bus System Message Bus

. . .

You can also view detailed information about a specific unit. Go ahead and verify the status of rsyslog service by executing systemctl with the status subcommand:

systemctl status rsyslog.service --no-pager

You'll see the program's output appear on the screen:

● rsyslog.service - System Logging Service
     Loaded: loaded (/lib/systemd/system/rsyslog.service; enabled; vendor preset: enabled)
     Active: active (running) since Thu 2021-03-25 19:20:26 CET; 2 weeks 5 days ago
TriggeredBy: ● syslog.socket
       Docs: man:rsyslogd(8)
             https://www.rsyslog.com/doc/
   Main PID: 8099 (rsyslogd)
      Tasks: 4 (limit: 19026)
     Memory: 3.6M
     CGroup: /system.slice/rsyslog.service
             └─8099 /usr/sbin/rsyslogd -n -iNONE

mar 25 19:20:26 alice systemd[1]: rsyslog.service: Succeeded.
mar 25 19:20:26 alice systemd[1]: Stopped System Logging Service.
mar 25 19:20:26 alice systemd[1]: Starting System Logging Service...
mar 25 19:20:26 alice rsyslogd[8099]: imuxsock: Acquired UNIX socket '/…1.0]
mar 25 19:20:26 alice rsyslogd[8099]: rsyslogd's groupid changed to 110
mar 25 19:20:26 alice rsyslogd[8099]: rsyslogd's userid changed to 104
mar 25 19:20:26 alice rsyslogd[8099]: [origin software="rsyslogd" swVer…tart
mar 25 19:20:26 alice systemd[1]: Started System Logging Service.

The output above describes the rsyslog service in detail. You can see that the service is enabled and that it has been active for almost three weeks. A few of the most recent log entries from the journal are displayed at the end of the output.

If you are not interested in such a detailed output, you can use the systemctl command with the is-active or is-enabled subcommand.

systemctl is-active rsyslog.service

This shows a simple boolean status indicating that the rsyslog service is active:

active

Execute systemctl with the is-enabled subcommand for rsyslog:

systemctl is-enabled rsyslog.service

Once again, a simple status output is displayed:

enabled

Step 3 — Starting and stopping services with Systemctl

In this section, you will modify the status of a service with systemctl. In the following examples, we will use systemctl with sudo because modifying Systemd unit files requires root privileges. The start subcommand is used to start a Systemd unit service while the stop subcommand does the opposite operation. Both operations do not persist after a reboot.

Your Ubuntu distribution already activates the rsyslog service. Let's stop it by executing the systemctl stop command:

sudo systemctl stop rsyslog.service

You should see the following output on your screen:

Warning: Stopping rsyslog.service, but it can still be activated by:
  syslog.socket

The output shows that the rsyslog.service is required by the syslog.socket unit. Therefore, if you stop only the rsyslog service, it will start immediately on demand of syslog.

You can check that both units are still active:

systemctl is-active rsyslog.service syslog.socket

The command shows output for both units in order:

active
active

Let's try to stop both units:

sudo systemctl stop rsyslog.service syslog.socket

If systemctl doesn't show any output, it means that the operation was completed successfully. You can check that both services are now inactive with the is-active subcommand.

Both units are essential for system operation, so let's start them again with start subcommand:

sudo systemctl start rsyslog.service syslog.socket

Once again, you can confirm the status of both services with is-active.

Step 4 — Enabling and disabling services with Systemctl

The stop and start subcommands are useful for stopping or starting a service unit during an active session. However, if you want the service to start or stop without your intervention on system boot, you must use the enable or disable subcommands.

For example, you can disable the rsyslog service with systemctl disable as shown below:

sudo systemctl disable rsyslog.service

You'll see the following output on the screen:

Synchronizing state of rsyslog.service with SysV service script with /lib/systemd/systemd-sysv-install.
Executing: /lib/systemd/systemd-sysv-install disable rsyslog
Removed /etc/systemd/system/syslog.service.
Removed /etc/systemd/system/multi-user.target.wants/rsyslog.service.

The output shows that systemctl removed the unit files related to rsyslog. Once the system reboots, Systemd won't find a configuration for this service. As a result, it won't start.

Let's check the new rsyslog status:

systemctl status rsyslog.service

You'll see the program's output appear on the screen:

● rsyslog.service - System Logging Service
     Loaded: loaded (/lib/systemd/system/rsyslog.service; disabled; vendor preset: enabled)
     Active: active (running) since Thu 2021-04-15 08:45:46 CEST; 38min ago
       Docs: man:rsyslogd(8)
             https://www.rsyslog.com/doc/
   Main PID: 23838 (rsyslogd)
      Tasks: 4 (limit: 19026)
     Memory: 1.2M
     CGroup: /system.slice/rsyslog.service
             └─23838 /usr/sbin/rsyslogd -n -iNONE

The output shows that rsyslog is active but also disabled. This means that on the next boot, it won't start again. But it will still work normally till the end of the current session.

Since rsyslog is an important system service, you should enable it back through the command below:

sudo systemctl enable rsyslog.service

You'll see the following output on the screen:

Synchronizing state of rsyslog.service with SysV service script with /lib/systemd/systemd-sysv-install.
Executing: /lib/systemd/systemd-sysv-install enable rsyslog
Created symlink /etc/systemd/system/syslog.service → /lib/systemd/system/rsyslog.service.
Created symlink /etc/systemd/system/multi-user.target.wants/rsyslog.service → /lib/systemd/system/rsyslog.service.

The output indicates that Systemctl created symbolic links to the unit files for the rsyslog unit. On the next boot, the system will find its configuration file, and it will start this service normally. Go ahead and check the status of the rsyslog service. It should be enabled once again.

Step 5 — Reloading and restarting services with Systemctl

Another typical use case for systemctl is to immediately reflect the changes made to a unit configuration file. If the application can reload configuration without restarting then you can use systemctl reload, otherwise, you must restart the entire service with systemctl restart.

You can try to reload the rsyslog service with systemctl:

sudo systemctl reload rsyslog.service

You'll see the following output appear on the screen:

Failed to reload rsyslog.service: Job type reload is not applicable for unit rsyslog.service.

The text explains that you cannot reload this type of service. In such cases, you must restart it. Let's try to restart the rsyslog service and immediately check the status of this service:

sudo systemctl restart rsyslog.service && systemctl status rsyslog.service

If the service was restarted successfully, you'll see the status output appear on the screen:

● rsyslog.service - System Logging Service
     Loaded: loaded (/lib/systemd/system/rsyslog.service; enabled; vendor preset: enabled)
     Active: active (running) since Thu 2021-04-15 09:56:45 CEST; 18ms ago
TriggeredBy: ● syslog.socket
       Docs: man:rsyslogd(8)
             https://www.rsyslog.com/doc/
   Main PID: 28793 (rsyslogd)
      Tasks: 4 (limit: 19026)
     Memory: 1.2M
     CGroup: /system.slice/rsyslog.service
             └─28793 /usr/sbin/rsyslogd -n -iNONE

The shows that the rsyslog service was activated 18ms ago. In other words, it was just restarted.

Step 6 — Masking and unmasking Systemd units

We've already discussed how to start and stop a service, and how to enable or disable it. But we can go one step further and mark a Systemd service as completely unstartable (either manually or automatically) by masking it through the mask subcommand. This links the unit file to /dev/null.

Go ahead and mask both the rsyslog and syslog services with the command below:

sudo systemctl mask rsyslog.service syslog.socket

You should observe the following output:

Created symlink /etc/systemd/system/rsyslog.service → /dev/null.
Created symlink /etc/systemd/system/syslog.socket → /dev/null.

When you try to start a masked service, you will receive a message informing you that the service is masked:

sudo systemctl start rsyslog.service syslog.socket

This yields:

Failed to start rsyslog.service: Unit rsyslog.service is masked.
Failed to start syslog.socket: Unit syslog.socket is masked.

You can also use the list-unit-files subcommand to view the status of a service:

systemctl list-unit-files

This produces the following output:

. . .

quotaon.service                        static          enabled
rc-local.service                       static          enabled
rc.service                             masked          enabled
rcS.service                            masked          enabled
rescue.service                         static          enabled
rsync.service                          enabled         enabled
rsyslog.service                        masked          enabled
screen-cleanup.service                 masked          enabled
secureboot-db.service                  enabled         enabled
serial-getty@.service                  indirect        enabled
setvtrgb.service                       enabled         enabled
ssh.service                            enabled         enabled
ssh@.service                           static          enabled
sshd.service                           enabled         enabled
stellar.service                        enabled         enabled
sudo.service                           masked          enabled
syslog.service                         masked          enabled
system-update-cleanup.service          static          enabled

. . .

To make a masked service available for use again, use the unmask subcommand:

sudo systemctl unmask rsyslog.service syslog.socket

The following output will be presented on your screen:

Removed /etc/systemd/system/rsyslog.service.
Removed /etc/systemd/system/syslog.socket.

You can use the status subcommand or list-unit-files to confirm the status of the services afterwards.

Conclusion

In this tutorial, you've learned how to view Systemd unit files and how to control its services with the systemctl utility. We started by examining the location and structure of unit files, and then we looked at how to check the status of a service. Afterwards, we discussed how to start, stop, enable or disable a Systemd service, and how to mask or unmask one. Finally we considered how you can reload or restart a service and when it may be necessary to do so.

Thanks for reading, and ensure to check out the systemd docs for more information.

Got an article suggestion? Let us know

This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.