Tag: systemd

Moving towards the firewalld software and especially the systemd some good old init scripts got missing! For example, one of those good scripts is the init script for iptables firewall, which allows saving iptables rules and during boot, it loads them again. With the init iptables script we have persistence of the iptables rules. Meanwhile, we can always call the init script with “save” argument to update the currently saved rules. Many different Linux distributions have this init script – “/etc/init.d/iptables”, but in systemd world, it has been removed and replaced with nothing (probably, because you are encouraged to use firewalld, which is not a bad thing!).

There are two packages “iptables-persistent” and “netfilter-persistent”, which work together to have iptables persistence over reboots. The rules are saved and restored automatically during system startup.

First, install “iptables-persistent” and “netfilter-persistent” with

sudo apt install netfilter-persistent iptables-persistent

During the iptables–persistent installation the setup asks the user to save the current iptables rules. Hit “Yes” if you want to save the current iptables rules, which will be automatically loaded the next time the system starts up.Configuring iptables-persistent setup
So it is safe to install it on a live system – the current iptables rules won’t be deleted.
Second, ensure the boot script to restore the iptables rules is enabled

And you can always edit them manually or save/restore with iptables-save and iptables-restore redirecting the output to the above files.

It’s normal the state of the “active (exited)”. The service is “enabled” as you can see (by default the setup automatically enables the service on Ubuntu, but always check it to be sure, it’s the firewall!).

On some Linux distributions, systemd log files are not saved on your disk, but only temporary in the memory and when you reboot all logs are discarded. So the systemd logs are not persistent, which could lead to missing important information if you want to check them when you are booted in a rescue disk or even if you just reboot your server. for exmaple,

if some important service failed to boot and your server is unreachable and you boot in rescue CD you do not have logs to check why the service failed and the (error) output of the process of starting the services!

Here is how you can enable the systemd logs to be persistent i.e. save them on the disk. This is tested on CentOS 7, which by default saves the systemd logs on memory!

Here we offer you a relatively new way of keeping your server’s time (or your computer and laptop) synchronized with a reliable time service on the Internet.

systemd has a built-in feature – a small daemon (systemd-timesyncd) to periodically to contact NTP servers and keep the server’s clock synchronized with them!

Of course, you must use systemd in your Linux distribution. This article is for those Linux systems using systemd, not for upstart (sysvinit, openrc, upstart, runit and so on). Most of the modern Linux distributions use the systemd like Fedora, Ubuntu, CentOS, RedHat, Gentoo, SuSe and many more.

Once there were not many options to keep your server’s clock synced with NTP servers. Now we have simpler programs (some of which by the way could act as clients only!!!) – chrony, openntpd, systemd-timesyncd and more.
This time synchronization service is not going to open server port 123, it does not have the server capabilities of an NTP server. So you won’t need any firewall rules (like for ntpd). It is a simple client service to sync your time and keep it synchronized all the time with accuracy not more than 100ms.

Do not expect complex clock discipline like training or compensating. It just sets the time according to a selected time server from the configuration file in “/etc/systemd/timesyncd.conf”. The polling interval is automatically adjusted in minimal and maximal values from the configuration file and the daemon decides which is the actual interval based on the near-term drift it thinks. Possible back running clock if it needs to set in the past. The quality of the clock source could not be checked, so

in any case, you may not expect more than 100ms accuracy.

Of course, this service is actively developed and it has already many changes from the base client once it was!

We’ve experienced this with the MySQL server under CentOS 7, but you can have this state with other services!
After updating our MySQL we tried to start it up, but the service got this strange state after “systemctl start” returned:

[root@mysql2 ~]# systemctl start mysqld
Job for mysqld.service failed because a timeout was exceeded. See "systemctl status mysqld.service" and "journalctl -xe" for details.

The timeout is big it’s something like 5 to 10 minutes and so it is typical (do not do it!) to type “ctrl+c” and you end up without this message and a strange state of the mysql:

So as you can see no mysqld process! Apparently systemctl had tried to start MySQL server process and it failed.
So the first thing to do was to check the MySQL logs. In our case it was a obsolete option in my.cnf:

the service got “Active: activating (start-post)” and when you fix the problem you cannot “systemctl start mysqld” it just start to wait for the current timeout.

In fact this state means “I’m trying to start the service…” and it is in an endless loop to start the service and if you the service has a big start timeout like 5-10 minutes you must wait for the next iteration of the loop to start the service successfully (if you fixed the problem!). And if you want not to wait you must execute first stop to the service and then start – you’ll not wait for any timeout and you can check immediately if the service was started successfully:

As you can see we even received error again that the service cannot be started and immediately after that the service status is in normal “active (running)” state! And we waited for around 10 minutes! You can see the times in the logs above.
So to summarize it up:

If you have a service in “activating (start-post)” the service cannot be started because of an error, check and fix the problem and then issue “stop and start”:

Systemd linux distros use Journald service to collect and store logs in the system. Here are a couple of tips if you have problems with the space they occupy. It is good that all systemd linux distros support it – CentOS 7, Ubuntu 16+, Fedora, OpenSuse and so on.

TIP 1) Remove the old archive logs older than 10 days with

Time based removal of old logs. It will remove the old files. This command won’t change the configuration, so it just has a temporary effect.

journalctl --vacuum-time=10d

TIP 2) Remove the old archive logs greater than 1G

Size based removal of old logs. It will reduce the size of the logs to this specified size. This command won’t change the configuration, so it just has a temporary effect.

journalctl --vacuum-size=1024M

TIP 3) Show disk usage

[root@srv ~]# journalctl --disk-usage
Archived and active journals take up 785.5M on disk.

TIP 4) Show all logs and information for them

Where are the log files, the size they occupy, the time period of the entries in them and more:

A nice feature of systemd is the scripts could be daemonized without any modifications of the scripts like standard stdin, stdout, stderr to be redirected to null (and even in some languages is not that simple!).
So if you want to run your ordinary script as daemon under a systemd linux distro, create a systemd service file in

We use cookies to ensure that we give you the best experience on our website. Unfortunately, the site is unable to function properly without using session, functional and third-party cookies. If you continue to use this site we will assume that you are happy with it or just close the page if you do not accept any cookies when viewing our site!Ok