Stuck with buggy micro code after recent BIOS update

‎01-22-201807:48 AM

We upgraded a bunch of T470s notebooks with the now withdrawn BIOS update (1.21) that included micro code updates to address the Spectre/Meltdown vulnerabilities. The devices (running Windows 10) now have intermittent hangs and are almost unbearably slow to work with, so we decided to roll back to 1.20. Turns out, that the micro code updates cannot be reversed by downgrading the BIOS. So we are now stuck with almost unusable notebooks until there will be a new update. What are we supposed to do now?

Re: Stuck with buggy micro code after recent BIOS update

How do you know microcode was not downgraded? Microcode is part of the FW so it should downgrade.

Did you load setup defaults after FW downgrade?Did you try to downgrade to an older FW version 1.19 or 1.17 on one of your devices?

<1.19>
UEFI: 1.19 / ECP: 1.17
- (New) Updated the CPU microcode.
<1.17>
UEFI: 1.17 / ECP: 1.16
- [Important] Update includes some security fixes.
(Note)
If the UEFI BIOS has been updated to version 1.17 or higher,
it is no longer able to roll back to the version before 1.17
for security improvement.

what to learn from this fiasco:

1. postpone flashing new FW until update is available on Lenovo System Update.

Re: Stuck with buggy micro code after recent BIOS update

‎01-22-201809:06 AM

There is info in security advisory on Lenovo webpage that once updated microcode cannot be reversed other way than replacing motherboard. So the only solution is to wait for better microcode which should arrive soon.

Re: Stuck with buggy micro code after recent BIOS update

‎01-22-201809:06 AM

Thank you for your quick reply.

I believe the code is not downgraded because the SpeculationControl script from Microsoft still shows hardware support for CVE-2017-5715. Also, the Lenovo Security Advisory LEN-18282 showed a back-flash recommendation after the updates have been withdrawn, but later removed it while commenting "MCU microcode updates cannot be reversed". You can still see that in the revision history.

Unfortunately, the older BIOS revisions also do not seem to roll back the microcode updates.

Regarding your points, this is why we updated only a couple devices (not a bunch as mentioned above, which was wrong). To be exact, 4 devices that have been actively used to evaluate the update.

Re: Stuck with buggy micro code after recent BIOS update

‎01-22-201810:10 AM - edited ‎01-22-201810:13 AM

The interesting thing here is: In theory, it should be possible to downgrade the microcode version. The reason for the blocked downgrade path seems to be the flashing program (WinFlash) used for the BIOS update procedure: The program has a special switch "/vcpu" that enables microcode updating.

According to the WinFlash documentation, the program evaluates three sources to determine the microcode version to be inserted into the BIOS:

the BIOS file to be flashed

an optional microcode file (that can be passed on the command line to the /vcpu switch)

the current content of the system EEPROM

WinFlash then chooses the most recent microcode version found in these sources. Since the EEPROM content itself is part of the evaluation, the system's microcode will never be downgraded.

Since microcode updates are not persistent across reboots anyway, I don't currently see a real technical reason for this behaviour, though. (It's not like there was a danger of "downgrading" your CPU into an unsupported state.) If I had to guess, I'd assume this was done for convenience reasons. Anyone with more knowledge on the matter is free to correct this assumption, though ...