Matthew Garrett

More in the series of bizarre UEFI bugs

A (well, now former) coworker let me know about a problem he was having with a Lenovo Thinkcentre M92p. It booted Fedora UEFI install media fine, but after an apparently successful installation refused to boot. UEFI installs on Windows worked perfectly. Secure Boot was quickly ruled out, but this could still have been a number of things. The most interesting observation was that the Fedora boot option didn't appear in the firmware boot menu at all, but Windows did. We spent a little while comparing the variable contents, gradually ruling out potential issues - Linux was writing an entry that had an extra 6 bytes in a structure, for instance[1], and a sufficiently paranoid firmware implementation may have been tripping up on that. Fixing that didn't help, though. Finally we tried just taking the Windows entry and changing the descriptive string. And it broke.

Every UEFI boot entry has a descriptive string. This is used by the firmware when it's presenting a menu to users - instead of "Hard drive 0" and "USB drive 3", the firmware can list "Windows Boot Manager" and "Fedora Linux". There's no reason at all for the firmware to be parsing these strings. But the evidence seemed pretty strong - given two identical boot entries, one saying "Windows Boot Manager" and one not, only the first would work. At this point I downloaded a copy of the firmware and started poking at it. Turns out that yes, actually, there is a function that compares the descriptive string against "Windows Boot Manager" and appears to return an error if it doesn't match. What's stranger is that it also checks for "Red Hat Enterprise Linux" and lets that one work as well.

This is, obviously, bizarre. A vendor appears to have actually written additional code to check whether an OS claims to be Windows before it'll let it boot. Someone then presumably tested booting RHEL on it and discovered that it didn't work. Rather than take out that check, they then addded another check to let RHEL boot as well. We haven't yet verified whether this is an absolute string match or whether a prefix of "Red Hat Enterprise Linux" is sufficient, and further examination of the code may reveal further workarounds. For now, if you want to run Fedora[2] on these systems you're probably best off changing the firmware to perform a legacy boot.

Maybe annnoying/DDoSing the sloppy vendor with all the different variations of things they broke with that helpful check (like, "I've tried these Linux distributions: [liiiist] and not a single one [but RHEL] worked after successful installation in UEFI mode while all of these work on an [HP/SONY/ASUS item]")...

Many of us have UEFI machines with no legacy boot option at all. So the legacy boot workaround is not a solution. Fortunately, my Dell Alienware X51 added Fedora 17 to the boot menu fine. I got nowhere trying to test F18 however.

That's strange...my W520 doesn't have that problem (although it has a plethora of others). It couldn't care less about what I call the UEFI entries, be it "Fedora 18", "Windows Boot Manager", or "My Computer Always Breaks".

Could you post a link to the relevant websites/forums on the issue? Maybe I can help :)

There's a UEFI specification, not a UEFI standard. What people do with that is up to them. As far as the Microsoft thing goes - there's some constraints on what vendors have to do with Secure Boot, but nothing that covers this.

I encountered the same problem with an MSI A55M-P35 motherboard I purchased last year. Except AFAIK there was no exception for RHEL, it was "Windows Boot Manager" or bust. Thankfully the latest firmware fixes the bug.

Does Windows always use that same string no matter which language version you install? ie does a German user and a Chinese user also always have the text "Windows Boot Manager"? Presumably when only Windows is installed it is rare to see the string.

I just bought a lenovo k410 desktop. It looks like I have ami aptivo ufei. I was about to install arch linux. I am confused by what is said above. Is it possible for me to 'identify' the os as rhel and boot arch instead? If so can somebody provide more explicit instructions?

A lot of laptops have "secret" partitions, where they store either recovery images or things like that. My Dell laptop came with one that had a media player (DVD, CD, &c); if you pressed the "Media" button when the machine was off instead of the "power" button, it would boot into this partition rather than Windows.

Maybe the point of this is to hide these "secret" partitions so people don't try to boot them? And during development, it was easier to have a "whitelist" rather than a "blacklist", because the team that made the partition was different than the team that wrote the BIOS?

I have a Lenovo W520, and what seems to be happening is that the firmware will ONLY boot the file "/EFI/Boot/bootx64.efi" on the EFI System Partition. It doesn't seem to matter what description you give it, if you rename your .efi boot manager to that exact location and filename, and if the system is set to UEFI boot, it will boot that manager. I've installed rEFInd boot manager and Grub2 boot manager in that location (renaming the .efi files) and both work fine. From rEFInd you can easily boot Windows or Linux .efi files if you put them in the right places for rEFInd to find. The Grub2 was installed by Ubuntu 12.10, which is UEFI and SecureBoot aware. However, Lenovo's firmware didn't like the location Ubuntu had put the file, so it wouldn't boot. Renaming it /EFI/Boot/bootx64.efi works perfectly. Note you have to make sure the disk containing the EFI partition is higher in priority order than any other disk that might boot for it to work automatically.

I'm not saying a string check isn't happening, but the location listed above seems to override any string check.

Did you add the efi file to the UEFI entries using efibootmgr. If you don't do that, the system will of course always boot "/EFI/Boot/bootx64.efi".
As far as I know, no UEFI bios will search for "*.efi" files. You need to add them explicitly by running:
sudo efibootmgr --create --loader '\EFI\whatever\grubx64.efi' --label 'My Installed OS'

There was a similar, but slightly different, diamond graphics card issue in the past. In short diamond thought that releasing programming information would lead to the loss of their uboat fleet in the north atlantic.

This resulted in a campaign of people buying new graphics card contacting their marketing people once and telling them that they were buying competing hardware due to diamond's refusal to support linux. Eventually this got the desired result and, if I remember correctly, diamond paid good money for their hardware to be well supported.

Maybe the same technique would work for Lenovo and others. I will either buy hardware which explicitly supports linux or build a box out of linux compatible bits. If I never buy a windows licence then it should be easy to avoid hardware which requires it :-)

I bought a HP printer because it did support Linux, and I wrote HP to tell them that that was one of the main reasons that I had bought their product. Let's hope that positive reinforcement works too! :-)

This issue looks like an innocent misfeature, but not all Lenovo's firmware quirks are.

They also have a WiFi/3G card whitelist that stops you using non-Lenovo cards. When pressed they'll claim that the FCC makes them do this. When you point out that (a) you don't live in the US and what the FCC says doesn't matter; and (b) they've never cited the actual regulation in question, they'll change their story to say that "regulators" around the world make them.

Oddly, those same regulations appear to also affect HP, but not Dell or Acer.

They don't disclose their hardware lockout on their web site, sales documentation, etc.

I wrote a bit of a rant about this when it bit me a while ago. On the upside they replaced my 3G card free of charge with a Lenovo one; on the flip side, they still haven't fixed their website and the new card doesn't work as well as my old one did.

They really shouldn't be able to sell machines as "Mobile Broadband Ready" when they mean "Lenovo Mobile Broadband Card ready (approved models only, machine will fail to POST with non-approved card installed)".

I don't think Lenovo will react to many calls about Linux support by improving it. More likely they will identify Linux users as troublesome, and remove whatever support they have. The only way to affect them is for very large purchasers to make demands on them. The entry for RHEL is no doubt a ham-handed response to a requirement from a large customer.

A colleague of mine pointed me towards your blog a while ago and I've been reading it for a while - good work. Normally I don't comment, but this entry echoes an experience I've been having.

I run a F17 and Windows dual system, where F17 and Windows are on separate disks each with its own EFI partition. When I unplug the F17 disk and boot the system, then (on next shutdown) plug the disk back in, the BIOS "can't find" the EFI bootloader on that disk. It is there. I can put it back via the livecd and grub2-install, which crucially prods the BIOS to update its boot order.

Curiously, however, the same sequence applied to the Windows disk brings the Windows Boot Manager back. Which is just as well as I don't have a PE disk capable of running bcdboot or whatever incantation is needed for Windows (yet).

This sounds like a similar issue to what you're seeing. I'm going to dissect my BIOS tomorrow and see if I see strings.

So yeah, I too have a Lenovo K410 and spent two days of hair pulling before I gave up. I'm confused on the whole UEFI thing. First off, after reading thru forum stuff, someone said to look at some directory to see if it was booting via UEFI or not, and because I didnt see that (can't remember exactly now, it was last week) I figured it wasn't. However no matter what I did, Windows 7 just boots, and Ubuntu, or rather the Grub menu, never shows up. Someone said something about the partition table being GPT instead of MSDOS, so I even did that, changed it and wiped the disk, creating new partitions, starting all over, installing Win 7 and then Ubuntu, but same damn problem.

Because I see no "legacy" over UEFI options in the BIOS menu, I see I can't do that. How would I go about making this work for Ubuntu? (12.10 to be precise.)

I have a Lenovo Ideapad Z580. It came with Windows 8 (not required by me for any purpose whatsoever), and I dual-booted with Ubuntu Linux 12.04.2 LTS. Upon starting the laptop every morning, it just goes into a blank screen. There are some solutions, however, none of them permanent as yet. My line of action now, is to totally get rid of whatever Windows 8 has, partitions, etc, and do a clean install of Ubuntu 12.04.2. Unless some solution does come up. BTW, Lenovo support in India also stated that if anything other than Windows 8 is found on the laptop, the warranty is void. They also did not offer option to downgrade Windows 8 to Windows 7. M$ does not sell Windows 7 anymore to normal Indian consumers anymore.

About Matthew

Power management, mobile and firmware developer on Linux. Security developer at CoreOS. Member of the Linux Foundation Technical Advisory Board and the Free Software Foundation board of directors. Ex-biologist. @mjg59 on Twitter. Content here should not be interpreted as the opinion of my employer.