Bug Description

Hallo
Testing 20121014 amd64 desktop in virtual env.
in the disk was already present an encrypted Ubuntu installation, and when I try to install a second Ubuntu it don't consider the encrypted partitions, suggesting to erase all the disk instead to instal along side.
But I want a free system for my son and an ecrypted one for my work on the same disk.
Attached screenshot
Thanks
Fabio

We can also see that the partition is regularry mounted and showed in launcher.

In short it's non-trivial to resize lvm on top of crypt and partman (underlying partitioning backend) has no support for this currently.

You can use manual partitioning to install half the system encrypted, and then re-run the installer to create the second installation unencrypted. But in my opinion, this use cases is better covered by per-user encrypted home directories (check box at the user setup step).

Hallo Dmitrijs.
Sorry for the bad example of me and my son.
Here the problem is that the installer don't show the partition declaring that there is no operating system installed....creating a potential situation of loss of data.
I think that we have to see this bug under this aspect, Ubuntu is telling a wrong thing and this is critical.
We cannot tell that the disk is empty when is full and protected....
Partition encrypted needs to be shown and the installer has to inform that actually there's no way to install alongside and the only option is to erase the disk.
imho.
Fabio

Hmm... all we know there is encrypted disk on it and that's it. We can't peak inside it, hence the labels / indications that there is nothing present on the disk. Ideally we offer a way to unlock/lock the disks and display correctly the situation. But I can see where the confusion is coming from.

@vorlon Yes, that is true. The UI is confusing however because it states "This computer currently has no detected operating systems. What would you like to do?". Which is not true, because the same installer just produced an encrypted install.

This is actually a regression from the alternate installer. With that, you could select "Configure encrypted volumes" followed by "Activate existing encrypted volumes", enter your passphrase, and then existing encrypted volumes would become available in the partitioner.

* The desktop image installer cannot unlock existing encrypted (LUKS) volumes. If you need to make use of existing encrypted volumes during partitioning, then use the "Try Ubuntu without installing" boot option to start a live session, open the encrypted volumes (for example, by clicking on their icons in the Unity launcher), enter your password when prompted to unlock them, close them again, and run `ubiquity` to start the installer. (Bug:1066480)

Is there any way the regression can be fixed? As has already been pointed out this worked fine with the Alternate CD image installer - you just had to select 'activate existing partitions', enter your password and they were all there in the partitioner.

It seems very user-unfriendly that someone who uses encrypted partitions (something that IMHO should be encouraged) will have the installer tell them the only way to install is to delete all their data. Having the workaround in the release notes is good but doesn't fix the problem - it's a very frustrating user experience having to search forums and release notes.

Please read release notes for the workaround:
"The desktop image installer cannot unlock existing encrypted (LUKS) volumes. If you need to make use of existing encrypted volumes during partitioning, then use the "Try Ubuntu without installing" boot option to start a live session, open the encrypted volumes (for example, by clicking on their icons in the Unity launcher), enter your password when prompted to unlock them, close them again, and run ubiquity to start the installer. (1066480)"

Indeed, this just came up a bit too late for us to be able to squeeze a fix into 12.10, I'm afraid; we were pushing it as it was. I've adjusted the bug metadata to make it clear that this is something we must fix for 13.04.

I installed to a system with some disks pre-encrypted - an old /home which I was going to migrate to a new larger disk and some other data partitions. I also created some new encrypted data partitions on the large disk in preparation.

I assigned the encrypted partitions a mount point (/data, etc), expecting the installer to ask for the encryption password at some point

I specifically did not ask it to do any formatting.

On starting the install, the encrypted partitions were re-formatted and all the data lost.

Of course, I had a backup, but I ended up with a number of unecrypted partitions (other than home, which I chose to re-encrypt once I'd realised what had happened).

I can see that I made a mistake in assigning a filesystem type (ext4), expecting that the installer would want to know the underlying filesystem, but there should definitely be a warning before destroying data when the format flag is not set.

Workaround in the release notes for this bug does not work. After installation, when I boot the system up, it never asks for a passphrase to unlock the encrypted volume. init script waits for root device and finally gives up dropping me into initramfs. Last error messages are:

"Gave up waiting for root device. Common problems:
- Boot args (cat .........
.....................................
ALERT! /dev/mapper/concerto-root does not exist. Dropping to a shell!"

Good to see that this bug is being worked on for 13.04, when I commented it looked like the bug was closed completely with a fix of 'look in the release notes'. Better than nothing but not really a fix for the installer telling me my partitions are unreadable when the 12.04 and previous alternate installer can read them fine.

Not only the instalation procedure tells that there's no system installed, which is, at minimum, confusing for beginers, but the workarrounds do not work either.
I tried this: http://ubuntuforums.org/showthread.php?t=1205372 which uses the alternate instalation.
The solution was tried and the system was installed but, since my user directory was encrypted in /home/username/.Private using encryptfs, it failed to mount, and I didn't find a workaround yet, since no username nor any passwork seems to work, even typing the exact terms I entered during install. So I can only access a Guest account, which dosn't allow me to access any Terminal nor sudo neither any graphical interface that requires root.
It seems that alternate install doesn't have the necessary packages to deal with encryptfs, since it doesn't understands its partition type. I tried to get into recovery other times to try to figure out something, using the bash, in fstab, but nothing works. Even adding /home/username/.Private /home/username encryptfs defaults 0 0 makes the installed system (either the installed or the recovery from the alternate CD) tell me it is not a valid partition.
Maybe the instalation procedure in the common CD and ALSO in the alternate CD should be changed to alert users that if they have an encrypted LVM, they probably don't need an encrypted home. When I did this mess I was really thinking I was making a separate /home partition.
I also tried this one: http://ubuntuforums.org/showthread.php?t=823929 using the normal CD.
This didn't work for me. I did everything hastala told us to do and started the install procedure. It failled in just substituting the previous system (only erase /root, /bin, etc.). It wanted to format the whole partition. Since I have /home and / in the same encrypted lvm partition, it wouldn't wwork for me.

partman-crypto does support activating existing volumes on ubuntu.
In d-i interface, one needs to enter manual partitioning, enter encryption sub-menu and select option to activate existing encrypted volumes. At which point all encrypted volumes will be attempted to be activated and user maybe asked for a password (multiple times, if there is more than one encrypted volume). After which one needs to back-out to automatic partitioning & gain options to for example upgrade encrypted volume.

This "loop via activating crypt volumes" can be implemented directly, or we can modify ubiquity's embedded d-i to execute that option for us and do question pop-ups about that.

But there is a caveat, once activated one cannot "deactivate" those volumes. See bug 291494 .

Also "resize" option can be dangerous, as there is no proper resize support for neither LVM2 nor dm-crypt volumes.

The "upgrade" or "reuse" options can be dangerous as well, since I am suspecting that lvm2 and/or ecryptfs user space tools might not be installed. In any case those options should be tested.

Please fix this regression before 14.04, it's going to make installing the next LTS a real nightmare for those of us who use encrypted partitions. Making a note in the Release Notes is great, but not really enough since the feature was in there before and it's changes someone has made (dropping the alternate installer) that removed the feature.

I'm not sure what I could do programming-wise but I'd be happy to take time to test any proposed fix on VMs or even on a desktop PC if necessary.

I can't believe encrypted volumes is still such a hastle on Ubuntu. I haven't seen this with any other distro I've used. Even Archlinux is easier, because you actually know why it is or isn't working. On Ubuntu, you push a few buttons, and hope it boots when you're done. Absraction is great when it actually works. Otherwise you're just SOL, with no clear troubleshooting steps. This really needs to be fixed.

This is the first time I've tried this without the alternate install CD, which required a different set of post-install interventions to get working. If you, like me, followed the workaround and ended up at the initramfs prompt, this is what you need to do:

In addition to selecting mount points and FS types for your encrypted volumes, you also need to select the containing physical volume, and set it as "Physical disk for encrypted volumes," or something like that.

It's great to make things easy with a GUI installer, but there really should be instructions for how to do this stuff manually, so you have a some recourse when it doesn't work. Other than trying random install options or forum suggestions until it succeeds, but you don't know why. Thus far I've been unable to find such a thing for Ubuntu. Without good documentation, you also discourage contributors from helping to solve the problem.

I found a bug in the cryptroot script run by update-initramfs. On line 523 it tests for a return value from add_device(), and prints an error message.

if ! modules=$(add_device "$dev"); then ...

This statement will never be true, therefore the user is never notified of the error.

I did receive a warning message, indicating that there's no valid entry for my volume group in /etc/crypttab. However, I can't see why it's even necessary to have anything in crypttab, because by the time the rootfs is accessible, and thus /etc/crypttab, the crypt has already been opened.

The summary of the situation is :
- Ubiquity doesn't offer to unlock encrypted partitions, and instead say they are empty, hence offering to format them.
It is exactly the same case in Debian, where it is even worse because the default install medium is a net-install image, which doesn't provide the opportuinity to open a live session and thus applying the recommended workaround (access and unlock concerned partitions through GUI file manager/disk utility before launching the installer); in Debian, one has to change TTY and follow a cryptic procedure based upon anna-installing/modprobe-ing stuff... And then still has to chroot and create a crypttab before rebooting.

- What is causing this behaviour (which is apparently a regression)? Is it that ubiquity doesn't read the LUKS headers of said partitions, and hence doesn't recognise them as encrypted and offer to open them ? In that case is it possible to (re-)implement the detection and choice for opening (cryptsetup author says it's really easy to recognize LUKS container), while having a conditional jump saying "if partitions are encrypted don't offer to resize" so as not to mess with partman inability to resize lvm and dm-crypt devices ? <a href="https://code.google.com/p/cryptsetup/wiki/FrequentlyAskedQuestions#1._General_Questions">cryptsetup FAQ</a> is quite vocal about Ubuntu's installer being a "LUKS killer"...

- Users take care of partitioning and encrypting+lvming the device before installation, or more simply wish to re-use pre-existing partition scheme (clean upgrade etc.);
- They then want to install Ubuntu within the predesigned layout;
- They fall short doing so because the installer EVEN IN MANUAL MODE the installer doesn't recognise the encrypted volumes ;
- They feel weird or rightly outraged because it's perfectly do-able from a live session, by opening partitions first through a GUI utility and then launching the installer (i.e. release notes' workaround);
- As underlined in the original bug report, the current behavior has as much chance to cause data loss as the possibility to resize such encrypted/lvmed partitions (happened to me with d-i for Debian Wheezy)

csredrat, as long as it is listed known issue for Quantal, I believe they are not going to fix it even in next release, despite the fact it is marked as "critical".
So the "Fix" which "is released" is listing this bug as a known issue...

As several have pointed out, the workaround mentioned in the Release Notes for 14.04 doesn't work, and I couldn't find any mention of this problem in the release notes for 14.10.
I've also tried many of the remedies in the forums (most of which are aimed at earlier releases). Can someone provide a workaround for this that works?

Install using the Ubuntu Server CD/USB image and use its text UI installer to configure encrypted partitions. After the installation is done use apt-get to install ubuntu-desktop (for Unity), kubuntu-desktop (for KDE) or whatever desktop you like best.

I've done this on 2 laptops and a desktop and it works fine (the end result is no different from the GUI installer once you've installed your desktop env) and is the only workaround that works AFAIK.