Bug Description

[Impact]

* Some systems fail to upgrade due to conflicts between systemd and the (now removed from the archive) systemd-shim / upstart.

* Instead of trying to work out what's the problem in ordering / removal of diverts, ensure that systemd is never unpacked whilst systemd-shim/upstart are still on disk. Thus declare conflicts against systemd-shim/upstart packages in systemd package.

[Test Case]

* monitor drop-off of upgrades with below reported problem

* Check that it is possible to upgrade to bionic's libpam-systemd from xenial with systemd-shim installed on xenial, ie.

this currently passes, however, systemd-shim remains installed. It should be removed instead. Apt install systemd should have lines like this:

The following packages will be REMOVED:
systemd-shim
...
Removing 'diversion of /usr/share/dbus-1/system-services/org.freedesktop.systemd1.service to /usr/share/dbus-1/system-services/org.freedesktop.systemd1.service.systemd by systemd-shim'
...

[Regression Potential]

* systemd-shim/upstart are both removed and not supported in bionic, thus forcing their removal via conflicts should bring the system into an expected state.

The error complains that files will be overwritten, but this code runs in the systemd-shim postrm script, which is after systemd-shim's files have been removed from disk. What does 'dpkg -S /usr/share/dbus-1/system-services/org.freedesktop.systemd1.service' report in this situation?

On Sun, Jun 10, 2018 at 07:38:26PM -0000, Shane Synan wrote:
> I just ran into this situation myself. After I received the error
> report and let the upgrade recover, running the dpkg command gives me...

Thanks for the information. And does the file
/usr/share/dbus-1/system-services/org.freedesktop.systemd1.service exist on
your system? If so, please attach it to the bug report so we can see its
contents and try to figure out where it came from.

I ran into this running a `do-release-upgrade -d` on a xenial machine at $work. The upgrade failed partway through leaving the system in a partial upgrade state afaict; lsb-release -a reports 18.04, but the apt sources are still xenial.

I just ran into this during do-release-upgrade from 16.04.5 to 18.04.1 as well among other issues. The suggested mv solution worked.

I ended up in a situation where some to-be-upgraded packages like network-manager-gnome were removed at first but never reinstalled, which turned out to be problematic, although I'm not certain it was this interruption's fault that apt somehow "forgot".

I have tried with mine: bug #1787367.
Okay, the usual update / upgrade runs through now, though what got was just the removal of the systemd-shim, and nothing else. feh and flowblade have been kept back. OpenOffice 6.0 (just as example) remains uninstalled. How can I know what else remains missing? The awk command lists a dozen of rather irrelevant stuff.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Similar experience on a DO instance. Initial do-release-upgrade failure due to bug #1793932: exim4 deprecated conffile issue. Now recovery is systemd-shim (9-1bzr4ubuntu1) blocked. @digitalcircuit's 2018-06-10 suggestion above did not work for me. Will restore 16.04 and try again later.

> Hello Kees, or anyone else affected,
>
> Accepted systemd into bionic-proposed. The package will build now and be
> available at
> https://launchpad.net/ubuntu/+source/systemd/237-3ubuntu10.8 in a few
> hours, and then in the -proposed repository.
>
> Please help us by testing this new package. See
> https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how
> to enable and use -proposed. Your feedback will aid us getting this
> update out to other Ubuntu users.
>
> If this package fixes the bug for you, please add a comment to this bug,
> mentioning the version of the package you tested and change the tag from
> verification-needed-bionic to verification-done-bionic. If it does not
> fix the bug for you, please add a comment stating that, and change the
> tag to verification-failed-bionic. In either case, without details of
> your testing we will not be able to proceed.
>
> Further information regarding the verification process can be found at
> https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in
> advance for helping!
>
> N.B. The updated package will be released to -updates after the bug(s)
> fixed by this package have been verified and the package has been in
> -proposed for a minimum of 7 days.
>
> ** Changed in: systemd (Ubuntu Bionic)
> Status: In Progress => Fix Committed
>
> ** Tags added: verification-needed verification-needed-bionic
>
> --
> You received this bug notification because you are subscribed to a
> duplicate bug report (1792241).
> https://bugs.launchpad.net/bugs/1773859
>
> Title:
> upgrades to 18.04 fail
>
> Status in systemd package in Ubuntu:
> Fix Released
> Status in systemd-shim package in Ubuntu:
> Won't Fix
> Status in systemd source package in Bionic:
> Fix Committed
> Status in systemd-shim source package in Bionic:
> Won't Fix
> Status in systemd source package in Cosmic:
> Fix Released
> Status in systemd-shim source package in Cosmic:
> Won't Fix
>
> Bug description:
> [Impact]
>
> * Some systems fail to upgrade due to conflicts between systemd and
> the (now removed from the archive) systemd-shim / upstart.
>
> * Instead of trying to work out what's the problem in ordering /
> removal of diverts, ensure that systemd is never unpacked whilst
> systemd-shim/upstart are still on disk. Thus declare conflicts against
> systemd-shim/upstart packages in systemd package.
>
> [Test Case]
>
> * monitor drop-off of upgrades with below reported problem
>
> * Check that it is possible to upgrade to bionic's libpam-systemd
> from xenial with systemd-shim installed on xenial.
>
> [Regression Potential]
>
> * systemd-shim/upstart are both removed and not supported in bionic,
> thus forcing their removal via conflicts should bring the system into
> an expected state.
>
> [Other Info]
>
> * original bug report
>
>
> $ sudo apt upgrade
> Reading package lists... Done
> Building dependency tree
> Reading...

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

If this package fixes the bug for you, please add a comment to this bug,
mentioning the version of the package you tested and change the tag from
verification-needed-bionic to verification-done-bionic. If it does not
fix the bug for you, please add a comment stating that, and change the
tag to verification-failed-bionic. In either case, without details of
your testing we will not be able to proceed.

* Some systems fail to upgrade due to conflicts between systemd and
the (now removed from the archive) systemd-shim / upstart.

* Instead of trying to work out what's the problem in ordering /
removal of diverts, ensure that systemd is never unpacked whilst
systemd-shim/upstart are still on disk. Thus declare conflicts against
systemd-shim/upstart packages in systemd package.

[Test Case]

* monitor drop-off of upgrades with below reported problem

* Check that it is possible to upgrade to bionic's libpam-systemd
from xenial with systemd-shim installed on xenial, ie.

> Dear Łukasz ZemczakThank you for all your replies.Sorry I couldn't make
> this test in the last weeks I will make them as soon as
> possibleRegardsAlejandro
>
> From: Łukasz Zemczak <email address hidden>
> To: <email address hidden>
> Sent: Tuesday, November 20, 2018 9:21 AM
> Subject: [Bug 1773859] Please test proposed package
>
> Hello Kees, or anyone else affected,
>
> Accepted systemd into bionic-proposed. The package will build now and be
> available at
> https://launchpad.net/ubuntu/+source/systemd/237-3ubuntu10.10 in a few
> hours, and then in the -proposed repository.
>
> Please help us by testing this new package. See
> https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how
> to enable and use -proposed. Your feedback will aid us getting this
> update out to other Ubuntu users.
>
> If this package fixes the bug for you, please add a comment to this bug,
> mentioning the version of the package you tested and change the tag from
> verification-needed-bionic to verification-done-bionic. If it does not
> fix the bug for you, please add a comment stating that, and change the
> tag to verification-failed-bionic. In either case, without details of
> your testing we will not be able to proceed.
>
> Further information regarding the verification process can be found at
> https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in
> advance for helping!
>
> N.B. The updated package will be released to -updates after the bug(s)
> fixed by this package have been verified and the package has been in
> -proposed for a minimum of 7 days.
>
> --
> You received this bug notification because you are subscribed to a
> duplicate bug report (1796221).
> https://bugs.launchpad.net/bugs/1773859
>
> Title:
> upgrades to 18.04 fail
>
> Status in systemd package in Ubuntu:
> Fix Released
> Status in systemd-shim package in Ubuntu:
> Won't Fix
> Status in systemd source package in Bionic:
> Fix Committed
> Status in systemd-shim source package in Bionic:
> Won't Fix
> Status in systemd source package in Cosmic:
> Fix Released
> Status in systemd-shim source package in Cosmic:
> Won't Fix
>
> Bug description:
> [Impact]
>
> * Some systems fail to upgrade due to conflicts between systemd and
> the (now removed from the archive) systemd-shim / upstart.
>
> * Instead of trying to work out what's the problem in ordering /
> removal of diverts, ensure that systemd is never unpacked whilst
> systemd-shim/upstart are still on disk. Thus declare conflicts against
> systemd-shim/upstart packages in systemd package.
>
> [Test Case]
>
> * monitor drop-off of upgrades with below reported problem
>
> * Check that it is possible to upgrade to bionic's libpam-systemd
> from xenial with systemd-shim installed on xenial, ie.
>
> lxc launch ubuntu-daily:xenial test-shim-upgrade
> lxc exec test-shim-upgrade
> apt update
> apt install systemd-shim
> wget
> https://deb.debian.org/debian/pool/main/s/systemd-shim/systemd-shim_10-3_amd64.deb
> apt install ./syste...

> Dear Łukasz ZemczakThank you for all your replies.Sorry I couldn't make
> this test in the last weeks I will make them as soon as
> possibleRegardsAlejandro
>
> From: Łukasz Zemczak <email address hidden>
> To: <email address hidden>
> Sent: Tuesday, November 20, 2018 9:21 AM
> Subject: [Bug 1773859] Please test proposed package
>
> Hello Kees, or anyone else affected,
>
> Accepted systemd into bionic-proposed. The package will build now and be
> available at
> https://launchpad.net/ubuntu/+source/systemd/237-3ubuntu10.10 in a few
> hours, and then in the -proposed repository.
>
> Please help us by testing this new package. See
> https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how
> to enable and use -proposed. Your feedback will aid us getting this
> update out to other Ubuntu users.
>
> If this package fixes the bug for you, please add a comment to this bug,
> mentioning the version of the package you tested and change the tag from
> verification-needed-bionic to verification-done-bionic. If it does not
> fix the bug for you, please add a comment stating that, and change the
> tag to verification-failed-bionic. In either case, without details of
> your testing we will not be able to proceed.
>
> Further information regarding the verification process can be found at
> https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in
> advance for helping!
>
> N.B. The updated package will be released to -updates after the bug(s)
> fixed by this package have been verified and the package has been in
> -proposed for a minimum of 7 days.
>
> --
> You received this bug notification because you are subscribed to a
> duplicate bug report (1796221).
> https://bugs.launchpad.net/bugs/1773859
>
> Title:
> upgrades to 18.04 fail
>
> Status in systemd package in Ubuntu:
> Fix Released
> Status in systemd-shim package in Ubuntu:
> Won't Fix
> Status in systemd source package in Bionic:
> Fix Committed
> Status in systemd-shim source package in Bionic:
> Won't Fix
> Status in systemd source package in Cosmic:
> Fix Released
> Status in systemd-shim source package in Cosmic:
> Won't Fix
>
> Bug description:
> [Impact]
>
> * Some systems fail to upgrade due to conflicts between systemd and
> the (now removed from the archive) systemd-...

systemd now correctly conflicts with systemd-shim, and installing both simultaniously is now even harder:

$ sudo apt install systemd-shim
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following package was automatically installed and is no longer required:
python3-netifaces
Use 'sudo apt autoremove' to remove it.
The following additional packages will be installed:
cgmanager cgroup-lite libcgmanager0 libnih-dbus1
Suggested packages:
pm-utils
The following packages will be REMOVED:
cloud-init friendly-recovery init libnss-systemd libpam-systemd netplan.io nplan plymouth plymouth-theme-ubuntu-text policykit-1 snapd
systemd systemd-sysv ubuntu-minimal ubuntu-standard
The following NEW packages will be installed:
cgmanager cgroup-lite libcgmanager0 libnih-dbus1 systemd-shim
WARNING: The following essential packages will be removed.
This should NOT be done unless you know exactly what you are doing!
init systemd-sysv (due to init)
0 upgraded, 5 newly installed, 15 to remove and 0 not upgraded.
Need to get 147 kB of archives.
After this operation, 77.6 MB disk space will be freed.
You are about to do something potentially harmful.
To continue type in the phrase 'Yes, do as I say!'
?] ^C
ubuntu@ubuntu:~$ dpkg-query -W systemd
systemd 237-3ubuntu10.10

Thus it should also cause removal of systemd-shim on upgrades. Marking as validation done.