Andi Kleen wrote:> > We're in real mode for now nd should not care about the hidden state.>

Sorry, Andi, that's not how real mode works.

That may be how real mode is *documented*, but that's not how it works.

The segment descriptor registers (what Intel calls "segment cache") is always active. The only thing that changes based on CR0.PE is how it is *loaded* and the interpretation of the CS flags.

The segment descriptor registers contain of the following sub-registers: selector (the "visible" part), base, limit and flags. In protected mode or long mode, they are loaded from descriptors (or fs.base or gs.base can be manipulated directly in long mode.) In real mode, the only thing changed by a segment register load is the selector and the base, where the base <- selector << 4. In particular, *the limit and the flags are not changed*.

As far as the handling of the CS flags: a code segment cannot be writable in protected mode, whereas it is "just another segment" in real mode, so there is some kind of quirk that kicks in for this when CR0.PE <- 0. I'm not sure if this is accomplished by actually changing the cs.flags register or just changing the interpretation; it might be something that is CPU-specific. In particular, the Transmeta CPUs had an explicit "CS is writable if you're in real mode" override, so even if you had loaded CS with an execute-only segment it'd be writable (but not readable!) on return to real mode. I'm not at all sure if that is how other CPUs behave.

The most likely explanation for this is that the VESA BIOS expects to be entered in Big Real Mode (*.limit = 0xffffffff) instead of ordinary Real Mode. Here is a completely untested patch which changes the segment descriptors to Big Real Mode instead. It would be worth testing out.