# How to Control and Manage Systemd Services with Systemctl Every operating system uses background processes to maintain the system's various operations. In Linux-based systems, these processes are known as _daemons_, and they are started and maintained by the initialization daemon that is launched immediately after system boot. In many of the most popular Linux distributions, this initialization daemon is [Systemd](https://wiki.archlinux.org/title/systemd), so understanding how it works is essential for effective Linux server administration. This article will teach you the basics of Systemd and the `systemctl` command, which is used to manage Systemd services. ## Prerequisites To follow through with this tutorial, you will need: - One Ubuntu 20.04 server that includes a non-root user with `sudo` access. [ad-logs-small] ## Step 1 — Viewing Systemd unit files Systemd's most important abstraction is the **unit**, which represents any resource maintained by operating systems such as daemons, sockets, devices, and many others. We will target only a _service_ unit in this section. Each specific unit of Systemd has its configuration and, typically, multiple configurations with various priorities and scope. These configurations are saved in special files called **unit files**. Generally, unit files are found in the following three directories (listed in ascending order of priority): 1. `/lib/systemd/system`: the most general unit files with the lowest priority. You shouldn't modify the files in this directory. 2. `/run/systemd/system`: the runtime unit configuration. These files change unit behavior at runtime. They are created dynamically and exist only for the actual boot session. 3. `/etc/systemd/system`: unit files with the highest priority. If you need to change a unit configuration, you will typically edit the files in this directory. ### Unit-Specific Configuration In cases where you only need to edit specific options in a unit file, you will typically create a subdirectory whose name is the unit service with the `.d` suffix (for example, subdirectory `example.d` belongs to `example` unit). This subdirectory will hold one or more files with a `.conf` suffix that modify and extend the base unit configuration. You can execute the `ls` command to list all units in `/etc` directory that utilize unit-specific configuration: ```command ls -d /etc/*.d ``` The `-d` option restricts the output to only directories, while `/etc/*.d` specifies that only directories in `/etc` that end with `.d` suffix should be presented. ``` [output] /etc/apparmor.d /etc/ld.so.conf.d /etc/rc1.d /etc/sensors.d /etc/bash_completion.d /etc/libibverbs.d /etc/rc2.d /etc/sudoers.d /etc/binfmt.d /etc/libpaper.d /etc/rc3.d /etc/sysctl.d /etc/cron.d /etc/logrotate.d /etc/rc4.d /etc/tmpfiles.d /etc/depmod.d /etc/modprobe.d /etc/rc5.d /etc/update-motd.d /etc/environment.d /etc/modules-load.d /etc/rc6.d /etc/usb_modeswitch.d /etc/grub.d /etc/pam.d /etc/rcS.d /etc/init.d /etc/profile.d /etc/rsyslog.d /etc/insserv.conf.d /etc/rc0.d /etc/sane.d ``` The above output shows all the unit configuration directories present for each service. For example, notice the `/logrotate.d` directory for the `logrotate` service. This is where you can edit, or create a new log rotation setup (see [How to Manage Logs with Logrotate on Ubuntu 20.04](https://betterstack.com/community/guides/logging/how-to-manage-log-files-with-logrotate-on-ubuntu-20-04/) for more information). ### Structure of Systemd unit files The structure of a unit file is precisely defined. Each one is divided into _sections_, and each section consists of _directives_. Let's view the contents of a unit file for the `rsyslog` service in `/etc/systemd/system` by executing the `cat` command: ```command cat /etc/systemd/system/syslog.service ``` You'll see the contents of the file appear on the screen: ``` [output] [Unit] Description=System Logging Service Requires=syslog.socket Documentation=man:rsyslogd(8) Documentation=https://www.rsyslog.com/doc/ [Service] Type=notify ExecStart=/usr/sbin/rsyslogd -n -iNONE StandardOutput=null Restart=on-failure # Increase the default a bit in order to allow many simultaneous # files to be monitored, we might need a lot of fds. LimitNOFILE=16384 [Install] WantedBy=multi-user.target Alias=syslog.service ``` The output shows definitions of the different sections and their directives for the `rsyslog` service. Each directive starts at a new line with a keyword followed by a value (for example, `Description=System Logging Service`). There are three sections in the file above: - `[Unit]`: defines the metadata of unit and its relations to the other units. - `[Service]`: used for the configuration of the service unit. The most common directive in this section is `Type`, which tells Systemd how to manage the service and find out its state. `Type=notify` indicates that the service will notify `systemd` once it has finished staring up. The `ExecStart` directive defines the full command that should be executed to start the service. - `[Install]`: defines the behavior of a unit after it is enabled. The `WantedBy` directive specifies the relationship between the unit and other ones, while `Alias` defines alternative names for the unit (in our case the `rsyslog`, and `syslog` are the same Systemd units). For more information about all other unit sections and their directives, see the [Systemd documentation](https://www.freedesktop.org/software/systemd/man/systemd.unit.html). ## Step 2 — Checking the service status with systemctl The `systemctl` command is a utility that controls Systemd and its units. This tool allows us to check a unit's status, and disable or enable them as needed. Go ahead and view the list of all available units on your system through the command below: ```command systemctl list-units --type=service --no-pager ``` The `--type=service` option shows only units of type `service`, and `--no-pager` prevents the output from being piped to the `less` pager. The command's output should appear on the screen: ``` [output] UNIT LOAD ACTIVE SUB DESCRIPTION accounts-daemon.service loaded active running Accounts Service acpid.service loaded active running ACPI event daemon alsa-restore.service loaded active exited Save/Restore Sound Card Sta… apparmor.service loaded active exited Load AppArmor profiles apport.service loaded active exited LSB: automatic crash report… avahi-daemon.service loaded active running Avahi mDNS/DNS-SD Stack bluetooth.service loaded active running Bluetooth service ● certbot.service loaded failed failed Certbot clean-mount-point@media-ju… loaded active exited Clean the /media/juraj/Exte… rsyslog.service loaded active running System Logging Service colord.service loaded active running Manage, Install and Generat… console-setup.service loaded active exited Set console font and keymap cron.service loaded active running Regular background program … cups-browsed.service loaded active running Make remote CUPS printers a… cups.service loaded active running CUPS Scheduler dbus.service loaded active running D-Bus System Message Bus . . . ``` You can also view detailed information about a specific unit. Go ahead and verify the status of `rsyslog` service by executing `systemctl` with the `status` subcommand: ```command systemctl status rsyslog.service --no-pager ``` You'll see the program's output appear on the screen: ``` [output] ● rsyslog.service - System Logging Service Loaded: loaded (/lib/systemd/system/rsyslog.service; enabled; vendor preset: enabled) Active: active (running) since Thu 2021-03-25 19:20:26 CET; 2 weeks 5 days ago TriggeredBy: ● syslog.socket Docs: man:rsyslogd(8) https://www.rsyslog.com/doc/ Main PID: 8099 (rsyslogd) Tasks: 4 (limit: 19026) Memory: 3.6M CGroup: /system.slice/rsyslog.service └─8099 /usr/sbin/rsyslogd -n -iNONE mar 25 19:20:26 alice systemd[1]: rsyslog.service: Succeeded. mar 25 19:20:26 alice systemd[1]: Stopped System Logging Service. mar 25 19:20:26 alice systemd[1]: Starting System Logging Service... mar 25 19:20:26 alice rsyslogd[8099]: imuxsock: Acquired UNIX socket '/…1.0] mar 25 19:20:26 alice rsyslogd[8099]: rsyslogd's groupid changed to 110 mar 25 19:20:26 alice rsyslogd[8099]: rsyslogd's userid changed to 104 mar 25 19:20:26 alice rsyslogd[8099]: [origin software="rsyslogd" swVer…tart mar 25 19:20:26 alice systemd[1]: Started System Logging Service. ``` The output above describes the `rsyslog` service in detail. You can see that the service is enabled and that it has been active for almost three weeks. A few of the most recent log entries from the journal are displayed at the end of the output. If you are not interested in such a detailed output, you can use the `systemctl` command with the `is-active` or `is-enabled` subcommand. ```command systemctl is-active rsyslog.service ``` This shows a simple boolean status indicating that the `rsyslog` service is active: ``` [output] active ``` Execute `systemctl` with the `is-enabled` subcommand for `rsyslog`: ```command systemctl is-enabled rsyslog.service ``` Once again, a simple status output is displayed: ``` [output] enabled ``` ## Step 3 — Starting and stopping services with Systemctl In this section, you will modify the status of a service with `systemctl`. In the following examples, we will use `systemctl` with `sudo` because modifying Systemd unit files requires root privileges. The `start` subcommand is used to start a Systemd unit service while the `stop` subcommand does the opposite operation. Both operations do not persist after a reboot. Your Ubuntu distribution already activates the `rsyslog` service. Let's stop it by executing the `systemctl stop` command: ```command sudo systemctl stop rsyslog.service ``` You should see the following output on your screen: ``` [output] Warning: Stopping rsyslog.service, but it can still be activated by: syslog.socket ``` The output shows that the `rsyslog.service` is required by the `syslog.socket` unit. Therefore, if you stop only the `rsyslog` service, it will start immediately on demand of `syslog`. You can check that both units are still active: ```command systemctl is-active rsyslog.service syslog.socket ``` The command shows output for both units in order: ``` [output] active active ``` Let's try to stop both units: ```command sudo systemctl stop rsyslog.service syslog.socket ``` If `systemctl` doesn't show any output, it means that the operation was completed successfully. You can check that both services are now _inactive_ with the `is-active` subcommand. Both units are essential for system operation, so let's start them again with `start` subcommand: ```command sudo systemctl start rsyslog.service syslog.socket ``` Once again, you can confirm the status of both services with `is-active`. ## Step 4 — Enabling and disabling services with Systemctl The `stop` and `start` subcommands are useful for stopping or starting a service unit during an active session. However, if you want the service to start or stop without your intervention on system boot, you must use the `enable` or `disable` subcommands. For example, you can disable the `rsyslog` service with `systemctl disable` as shown below: ```command sudo systemctl disable rsyslog.service ``` You'll see the following output on the screen: ``` [output] Synchronizing state of rsyslog.service with SysV service script with /lib/systemd/systemd-sysv-install. Executing: /lib/systemd/systemd-sysv-install disable rsyslog Removed /etc/systemd/system/syslog.service. Removed /etc/systemd/system/multi-user.target.wants/rsyslog.service. ``` The output shows that `systemctl` removed the unit files related to `rsyslog`. Once the system reboots, Systemd won't find a configuration for this service. As a result, it won't start. Let's check the new `rsyslog` status: ```command systemctl status rsyslog.service ``` You'll see the program's output appear on the screen: ``` [output] ● rsyslog.service - System Logging Service Loaded: loaded (/lib/systemd/system/rsyslog.service; disabled; vendor preset: enabled) Active: active (running) since Thu 2021-04-15 08:45:46 CEST; 38min ago Docs: man:rsyslogd(8) https://www.rsyslog.com/doc/ Main PID: 23838 (rsyslogd) Tasks: 4 (limit: 19026) Memory: 1.2M CGroup: /system.slice/rsyslog.service └─23838 /usr/sbin/rsyslogd -n -iNONE ``` The output shows that `rsyslog` is active but also disabled. This means that on the next boot, it won't start again. But it will still work normally till the end of the current session. Since `rsyslog` is an important system service, you should enable it back through the command below: ```command sudo systemctl enable rsyslog.service ``` You'll see the following output on the screen: ``` [output] Synchronizing state of rsyslog.service with SysV service script with /lib/systemd/systemd-sysv-install. Executing: /lib/systemd/systemd-sysv-install enable rsyslog Created symlink /etc/systemd/system/syslog.service → /lib/systemd/system/rsyslog.service. Created symlink /etc/systemd/system/multi-user.target.wants/rsyslog.service → /lib/systemd/system/rsyslog.service. ``` The output indicates that Systemctl created symbolic links to the unit files for the `rsyslog` unit. On the next boot, the system will find its configuration file, and it will start this service normally. Go ahead and check the status of the `rsyslog` service. It should be enabled once again. ## Step 5 — Reloading and restarting services with Systemctl Another typical use case for `systemctl` is to immediately reflect the changes made to a unit configuration file. If the application can reload configuration without restarting then you can use `systemctl reload`, otherwise, you must restart the entire service with `systemctl restart`. You can try to reload the `rsyslog` service with `systemctl`: ```command sudo systemctl reload rsyslog.service ``` You'll see the following output appear on the screen: ``` [output] Failed to reload rsyslog.service: Job type reload is not applicable for unit rsyslog.service. ``` The text explains that you cannot reload this type of service. In such cases, you must restart it. Let's try to restart the `rsyslog` service and immediately check the status of this service: ```command sudo systemctl restart rsyslog.service && systemctl status rsyslog.service ``` If the service was restarted successfully, you'll see the status output appear on the screen: ``` [output] ● rsyslog.service - System Logging Service Loaded: loaded (/lib/systemd/system/rsyslog.service; enabled; vendor preset: enabled) Active: active (running) since Thu 2021-04-15 09:56:45 CEST; 18ms ago TriggeredBy: ● syslog.socket Docs: man:rsyslogd(8) https://www.rsyslog.com/doc/ Main PID: 28793 (rsyslogd) Tasks: 4 (limit: 19026) Memory: 1.2M CGroup: /system.slice/rsyslog.service └─28793 /usr/sbin/rsyslogd -n -iNONE ``` The shows that the `rsyslog` service was activated `18ms` ago. In other words, it was just restarted. ## Step 6 — Masking and unmasking Systemd units We've already discussed how to start and stop a service, and how to enable or disable it. But we can go one step further and mark a Systemd service as completely unstartable (either manually or automatically) by masking it through the `mask` subcommand. This links the unit file to `/dev/null`. Go ahead and mask both the `rsyslog` and `syslog` services with the command below: ```command sudo systemctl mask rsyslog.service syslog.socket ``` You should observe the following output: ``` [output] Created symlink /etc/systemd/system/rsyslog.service → /dev/null. Created symlink /etc/systemd/system/syslog.socket → /dev/null. ``` When you try to start a masked service, you will receive a message informing you that the service is masked: ```command sudo systemctl start rsyslog.service syslog.socket ``` This yields: ``` [output] Failed to start rsyslog.service: Unit rsyslog.service is masked. Failed to start syslog.socket: Unit syslog.socket is masked. ``` You can also use the `list-unit-files` subcommand to view the status of a service: ```command systemctl list-unit-files ``` This produces the following output: ``` [output] . . . quotaon.service static enabled rc-local.service static enabled rc.service masked enabled rcS.service masked enabled rescue.service static enabled rsync.service enabled enabled rsyslog.service masked enabled screen-cleanup.service masked enabled secureboot-db.service enabled enabled serial-getty@.service indirect enabled setvtrgb.service enabled enabled ssh.service enabled enabled ssh@.service static enabled sshd.service enabled enabled stellar.service enabled enabled sudo.service masked enabled syslog.service masked enabled system-update-cleanup.service static enabled . . . ``` To make a masked service available for use again, use the `unmask` subcommand: ```command sudo systemctl unmask rsyslog.service syslog.socket ``` The following output will be presented on your screen: ``` [output] Removed /etc/systemd/system/rsyslog.service. Removed /etc/systemd/system/syslog.socket. ``` You can use the `status` subcommand or `list-unit-files` to confirm the status of the services afterwards. ## Conclusion In this tutorial, you've learned how to view Systemd unit files and how to control its services with the `systemctl` utility. We started by examining the location and structure of unit files, and then we looked at how to check the status of a service. Afterwards, we discussed how to start, stop, enable or disable a Systemd service, and how to mask or unmask one. Finally we considered how you can reload or restart a service and when it may be necessary to do so. Thanks for reading, and ensure to check out the [systemd docs](https://www.freedesktop.org/software/systemd/man/systemd.unit.html) for more information. [ad-logs]