After some waiting the initial issuing of certificates worked well, see here. I did not change anything knowingly since then. Do you have any idea what could be wrong?
I can reach the domains in question by webbrowser without problem.
Thanks!

I would try to update to 3.1.10, then go to the settings of that website, disable the letsencrypt checkbox, press save, then enable the letsencypt checkbox again. Then wait at least one minute before you check in a web browser if the LE cert is up to date now.

Thanks Till for this hint - it worked. The update to 3.1.10 went smooth and disabling and reenabling Let's Encrypt brought me two new certificates for the domains I tested it with.
The rest of the affected domains I did not change up until now for testing purposes. For them unfortunately the regular update process failed in the night, again. I tested it now with another one of these domains: doing it manually (disabling and reenabling) worked there, too.
Do you have any idea what could cause this? Automatic renewal fails but manually ordering a new cert works...

Yes, there are no new certificates. I had a look in the logfiles in /var/log/letsencrypt where I found the corresponding error messages and in /etc/letsencrypt the relevant file dates are much too old, too.

As I got to know now I have this problem with another machine, too. Manual reissuing with deactivating and reactivating the Let's Encrypt option works, but the automatic process fails.
As the certificates were nearly expired I corrected this by hand for now. But for the future a real solution would be great.
In the meantime I created a script to check the used certificates for validity: Checking validity of Let's Encrypt certificates in ISPConfig.