Bug Description

Parted in Ubuntu is patched to sync the hybrid-MBR with the GPT partition table on Apple systems. Under certain circumstances, a broken hybrid-MBR will be produced with multiple overlapping partitions of partition-ID 0xEE.

The effect can be reproduced, and the cause diagnosed, see TEST CASE and DIAGNOSIS below.

INCIDENCE AND EFFECTS:

A typical case where this bug may manifest is a dual- or triple-booting Mac. First the user installs Mac OS X, partition table looks like this:

The author of gdisk has also seen this bug: http://www.rodsbooks.com/gdisk/hybrid.html
'I'm seeing a lot of problem reports from Mac users who attempt to dual-boot with Linux because one of the utilities involved creates a buggy hybrid MBR with an extra type-0xEE partition that overlaps others.'

I will attach the output of 'parted print' and 'fdisk -l' for a triple-boot system where I encountered this bug.

Both Ubuntu and Mac OS X will continue to boot in my experience, as will Windows if present. However, disk utilities may become very confused, and may cause damage attempting to fix the broken partition table. Also, Mac OS X will believe it is not on a GPT-formatted disk, and will not be able to install firmware upgrades.

TEST CASE:

1. Add a new disk to an Ubuntu system or VM. I'll assume this is /dev/sda .

2. Create a GPT partition table with an EFI system partition:
a. Run 'parted /dev/sda' as root
b. In parted, create the table and partition:

3. Now we'll add some partitions, and see the bug.
a. Run parted as root such that it will emulate running on an Apple
system: 'PARTED_GPT_APPLE=1 parted /dev/sda'
b. In parted, create a BIOS Boot Partition for grub2. We'll also create
some extra partitions to make the problem easier to notice.

Aside from the proximate bug in gptsync.patch, there are two reasons I think this shouldn't have become an issue in the first place. I'd appreciate any thoughts.

1. Not all partition types should go in the hybrid MBR

The hybrid MBR exists for just two reasons:
i. So non-GPT-aware OSes can access GPT partitions
ii. So BIOSes/bootloaders that use the MBR exclusively can jump to the Partition Boot Record of GPT partitions, for booting.

Neither of these uses are relevant to the grub BIOS Boot Partition. It's just raw code/data, there's no reason for a non-GPT-aware OS to touch it. And no foreign bootloader should ever jump directly to it, only grub should do so, and grub doesn't need the MBR to find it. Other partition types, such as Linux Swap, also have no use in a hybrid-MBR.

Parted's gptsync should therefore omit BBP and Linux Swap partitions from the hybrid-MBR.

2. Ubuntu's installer should use an EFI System Partition on GPT

Although it's not mandatory, many OSes and tools expect a GPT partition table to begin with an EFI System Partition. The elilo bootloader uses it for booting Linux. Mac OS X's disk utility and installer create an ESP on every GPT disk they format, and the OS can't do firmware updates without one. The rEFIt bootloader is often installed in the ESP. I'm not sure what Windows does, but MS's docs recommend that an ESP comes first in the GPT and MBR.

I don't think this is the right solution. If later another MBR/GPT-syncing tool (rEFIt, gdisk) is used, it will not give the BBP the MBR ID 0xEE, and booting Linux will fail. Other OSes may not install without an ESP, or may be otherwise unhappy.

Partman-auto should therefore put an ESP at the start of a GPT disk when possible. An example partition layout looks entirely reasonable:

Thanks for your detailed report. I'll look into fixing this as soon as I can.

Regarding item 1 (gptsync behaviour for irrelevant partitions), while in theory I agree, I wonder if it is wise to change that at this point. Changing that could mean that partition numbers change if people run a libparted-based program and just change one unrelated detail, and it strikes me that that is not unlikely to cause problems. I am leaning towards leaving this alone.

I hear your concern about changing whether irrelevant partitions are placed in the hybrid MBR. Note however that the only partition numbering that would change would be in the MBR. In a GPT + hybrid-MBR configuration, linux takes its partition numbers exclusively from the GPT, so linux device numbers won't change at all. Are you aware of any utilities or OSes that care particularly about the index of partitions within a hybrid-MBR?

If there are tools that care, another possibility is to simply zero out the entries corresponding to irrelevant partitions. They'll then be ignored, but the index of other partitons will remain the same.

I was just doing some testing and realized that there's a bigger problem: If a disk has no EFI system partition, and no grub_bios partition, then the hybrid-MBR we create doesn't contain an 0xEE partition at all, and Linux will not recognize the disk as GPT! We should definitely special-case this and refuse to create a hybrid MBR.

Here's a patch to libparted/labels/gpt.c that's a first stab at a solution. The changes:

1. Only the first partition may start at sector 1, don't modify later partitions that way. This avoids overlap.

2. If no protective partition at all is found, place at the start of the MBR a tiny fake protective partition, covering the space from sector 1 to the start of the first real partition. Partitions that follow are moved up by one, so that the hybrid MBR only includes three real partitions. (This approach is used by gpt-fdisk, see: http://www.rodsbooks.com/gdisk/hybrid.html )

* Creation of a fake protective partition causes the partition numbering in the MBR to change. However, since parted currently produces partition tables that can't be read by Linux, this is almost certainly worth it.

* It might be best to create a fake initial protective partition in all cases where the first partition is not of type 0xEE. Thoughts?

* No attempt is made to leave certain partition types out of the hybrid MBR, for now (see issue #1 in my previous comment).

Re your question about a fake initial protective partition, I think we can probably leave things as they are for natty. Fixing bug 746313 should reduce the incidence of this, and TBH I think that GRUB's behaviour of requiring the first partition to be the protective one should be changed anyway.

Regarding what cares about partition indices in the hybrid MBR: well, I would expect that'd be legacy systems that require the MBR in the first place, so indeed naturally not Linux. However, I guess in most situations where you're trying to use such systems on Macs there'll be an EFI System Partition anyway, so any problems caused by this should be pretty limited.

* There's no such thing as a "fake protective partition." The 0xEE partition in the MBR does not correspond to any real partition in the GPT. The purpose of the 0xEE partition is to keep GPT-unaware utilities from messing with the disk. It ordinarily covers the space from sector 1 to the end of the disk, but on a hybrid disk this is reduced to the space from sector 1 to one less than the first sector of the first (in disk space terms) GPT partition included in the hybrid MBR. Note in particular that the 0xEE partition does NOT correspond to the EFI System Partition (ESP), although at first glance it might look like it does. (The belief that these two partitions are equivalent can cause problems if a user attempts to access the 0xEE partition from Windows. If writes are attempted, the result can be damage to the GPT data structures!)

* IMHO, routinely creating a hybrid MBR is a Bad Idea (note the i-caps). See my Web page on hybrid MBRs (http://www.rodsbooks.com/gdisk/hybrid.html) for an extended discussion of this, but in brief, they're ugly and dangerous -- it's too easy for either an MBR partitioning tool or a GPT partitioning tool to get the two to be out of sync in one way or another, which can have disastrous results. Unfortunately, Apple chose to rely on hybrid MBRs to get Windows booting on Macs, so we're left dealing with their mess. In an ideal world, Linux could boot on Macs using EFI mode, obviating the need for a hybrid MBR on Linux/MacOS dual-boot systems, but this remains a bit tricky. As to rules for when to create a hybrid MBR automatically, that's even trickier, but I'd start with doing it ONLY on Macs or on BIOS-based systems that already have one. On UEFI-based PCs, a hybrid MBR should never be necessary (since Windows boots fine in UEFI mode from GPT on such systems). On a BIOS-based PC, a hybrid MBR might be necessary to get Windows booting (as in a Hackintosh configuration), but Linux doesn't need it, so there's no point in creating one if it doesn't already exist. I haven't looked at the code in question so I don't know how hard it would be to add such rules to it, but blindly creating hybrid MBRs when they aren't necessary is a recipe for trouble.

* I wouldn't worry about keeping GPT and MBR partition numbers in sync. I don't know of any OS or utility that cares about this detail, and as I wrote earlier, the MBR's 0xEE partition isn't in sync with anything on the GPT side to begin with.

* I agree with Dave that there's no point in including a BIOS Boot Partition or Linux swap partition in a hybrid MBR. I'd go further: there's little point in including ANY Linux or OS X partition in a hybrid MBR; neither Linux nor OS X uses the hybrid MBR at all, and GRUB reads the GPT data just fine. I know of only two reasons to include such a partition: To kick Apple's EFI into BIOS compatibility mode to enable booting Linux in that way; and to enable Windows to access a Linux or OS X partition when appropriate drivers are installed. Given all the other variables at work, IMHO it's best to err on the side of not including these partitions in the hybrid MBR (and thus conve...

[ Dave Vasilevsky ]
* Improve built-in gptsync implementation (LP: #757201):
- Only the first partition may start at sector 1; don't modify later
partitions that way. This avoids overlap.
- If no protective partition at all is found, place at the start of the
MBR a tiny fake protective partition, covering the space from sector 1
to the start of the first real partition. Partitions that follow are
moved up by one, so that the hybrid MBR only includes three real
partitions.
-- Colin Watson <email address hidden> Fri, 15 Apr 2011 17:29:54 +0100