Snippets for detection of 32-bit/64-bit code segment

I accidentally came across this little snippet: https://twitter.com/VK_Intel/status/897206671231131649.*
And it made me smile, because it is so reminiscent of the tricks I've been using back in early 2000's when playing with first versions of my 32-bit unreal mode, where the interrupt handler needed to detect if the code segment is 32-bit or 16-bit and deal with it accordingly. You can read about it in my unREAL mode article.

Unlike 16/32, I never needed to do a similar thing to distinguish 32-bit from 64-bit. The only obvious application that comes to my mind would be malware (though the world of code is complex out there, so I do not rule it out that someone might use it for other obscure purpose). But still, this one made me think a little. I wondered if I could make a variant using the same principle as the original 16/32 "distinguisher" I made for my unreal mode.

You see, the one I made for interrupt handler did it without altering any register. The flags are stored automatically on the stack when interrupt is initiated, so I did not have to worry about them. So I used the CMP instruction, though it was not my intention to use the result for a conditional jump. The trick was that CMP used immediate operand, which was decoded as either 16-bit or 32-bit depending on mode, so there was a difference of two bytes in the interpreted instruction length, and two bytes is exactly what is needed to fit the "JMP SHORT" instruction. This jump would be seen in 16-bit mode, but in 32-bit one it would become part of the immediate of the dummy CMP instruction. It was a "conditional unconditional" jump.

We cannot use the exact same trick to distinguish 64-bit from 32-bit, as they both use the same immediate sizes for instructions like CMP. We could try CMP with a memory operand, because 67h prefix does switch to 16-bit addressing in 32-bit mode and to 32-bit addressing in 64-bit mode. The problem is, the memory address could be invalid. I can use LEA to avoid the exception, but this is going to trash a register:

Doesn't that depend on the OS? Also, shouldn't it be possible 100% all 3 of them? I mean, all 3 modes are valid in Long Mode, so there has to be a way to switch to it (even if, obviously, undocumented and unreliable between OS versions). Assuming it's not during kernel mode, which wouldn't make much sense to me (I mean, does Linux have 16-bit code in kernel? because it can run 16-bit Windows apps with Wine; Windows' limitation of lack of 16-bit in 64-bit comes from its APIs due to HANDLEs, nothing to do with the CPU).

https://en.wikipedia.org/wiki/Long_mode here it says it can run 16-bit protected mode code, I've never actually fiddled with this stuff, so maybe I'm assuming something stupid (maybe Linux does have special cases for Wine?)

I consulted the old manuals to be sure - yes, both AMD and Intel specifications allow D bit to switch to 16-bit code in the compatibility mode. But the Virtual 8086 mode does not work (perhaps because it messes too much with the addressing). So it should be possible to run 16-bit protected mode applications (like the ones for Windows 3), but not DOS programs, as they would require V86 mode.

This method is also closely related to the problem of having a snippet that executes as different instructions depending on what model of CPU is it run on. I'd been thinking that this was the thing of the past that died with the advent of CPUID and reliably signalled #UD, but recently I was reminded that there are instructions interpreted by Intel 64 differently than by the original AMD's x86-64.

So when consulting that 16-bit thing I went to both manuals, it is better to check twice, sometimes an interesting discovery may be made.

But wasn't push with 16-bit operand supposed to be valid in 64-bit mode? I mean it specifically said you can't encode 32-bit but instead 16-bit pushes.

It was in the original x86-64 specification by AMD. But when Intel made its own version, they made some changes, like removing the 16-bit pushes and jumps. And at the same time they added 10-byte immediate far jump that AMD was missing.

This board has a long history, if you dig deep enough you may find discussion of these details from the time when 64-bit instructions were being implemented into fasm.

You cannot post new topics in this forumYou cannot reply to topics in this forumYou cannot edit your posts in this forumYou cannot delete your posts in this forumYou cannot vote in polls in this forumYou cannot attach files in this forumYou can download files in this forum