ENGINEERING BLOG

Compiling Executables for the Classic POSIX Subsystem on Windows

/SUBSYSTEM:POSIX

You’ve seen it before, haven’t you? It’s strange. It’s like a face you passed on the street but can’t quite place. Was it déjà vu? A doppelganger? Maybe the first time you saw it it was in a sea of linker flags on MSDN, or perhaps when fumbling around with the project settings in Visual Studio some years ago.

You lingered for an extra second thinking “What on earth…?” while your eyes glazed over in reverie.

POSIX Subsystem Linker Flag in Visual Studio 2015

An artifact of evolution and monument to supporting legacy software. It was built by the ancients, forgotten, and left for new generations to rediscover.

Background

In the late 80’s and early 90’s IBM’s OS/2 and POSIX based systems dominated the computing ecosystem. In an effort to compete, Microsoft developed compatibility layers on top of the NT kernel to both attract developers onto the emerging platform of Windows and qualify for federal contracts seeking POSIX or OS/2 compliant machines.

These compatibility subsystems made it less painful for developers and enterprise to compile their existing POSIX or OS/2 application sources into a Microsoft Portable Executable (an .EXE).

Windows (NT) came to flourish in the 90’s. But the world of Windows that we know and experience today runs only the Win32 Subsystem. The OS/2 Subsystem was deprecated and removed after Windows 2000, and the POSIX Subsystem was removed over a decade later with Windows 8.1 in 2013.

Wait, what?

“My other subsystem is a POSIX one…”

The classic POSIX Subsystem persisted into ‘modern’ systems (albeit, disabled by default) for the curious to explore and pore over. It was known as the Subsystem for UNIX-based Applications, SUA, or Interix Subsystem.

Enabling the POSIX Subsystem on Windows 7 SP1, Enterprise

If you are running the Enterprise or Ultimate SKU of Windows 7 (or Windows 8, but not 8.1) you can enable the Subsystem for UNIX-based Applications from the ‘Windows Features’ dialog.

Compiling

Compiling executables for this legacy subsystem is a lost art in this day and age. There exists no documentation online, and even the SUA SDK offered little insight to actually building new executables. I am not convinced an executable has been compiled for this subsystem in years.

The secret was piecing together the build batch file and locating the relevant SUA libraries.

Open an Administrator Command Prompt to the extracted contents of the downloaded zip

Run the included build_new.bat file

These steps yield a Microsoft PE compiled for the POSIX Subsystem.

Compiling & executing a POSIX Subsystem executable on Windows 7 SP1

A Windows POSIX Executable

Inspecting the PE header of the created executable, one can see the 16bit Subsystem field indicates that this PE expects to be loaded by the POSIX Subsystem.

The 16bit Subsystem field in the PE header

The generated executable only has one dependency, PSXDLL.DLL. It takes no direct dependencies on KERNEL32.dll

The dependencies of a POSIX Portable Executable

A machine enlightened with the POSIX Subsystem now produces a number of interesting processes and services, such as: psxss.exe, init, inetd, posix.exe, and our main.exe.

Spectating the execution of a POSIX Executable from Process Explorer

The subsystem even supports some basic low entropy ASLR for the loaded executable.

ASLR can be observed by the relocation of main() between runs

Conclusion

The legacy Windows subsystems are rife with history. They offer a unique perspective into the versatile nature of NT Kernel, and the feats of engineering made by Microsoft (and Interix!) in the 90’s to win over devs & consumers alike.

As a product of my exploration into the POSIX Subsystem, I produced two CTF challenges for security enthusiasts looking to dig a little deeper. These challenges exposed hundreds of student teams to this dusty corner of Windows for the duration of CSAW 2016 & 2017.