Visual Studio 2015 Update 1: New Experimental Feature – MPX

Introduction

This post is about Intel® Memory Protection Extensions (Intel® MPX) support in Microsoft Visual Studio* 2015; content provided by Gautham Beeraka, George Kuan, and Juan Rodriguez from Intel Corporation.

Overview

Update 1 for Visual Studio 2015 was announced on November 30, 2015. This update includes experimental compiler and debugger support for Intel MPX. Intel MPX can check all pointer reads and writes to ensure they remain within their declared memory bounds. This technology can detect buffer overflows and stop program execution at runtime, averting possible system compromises. It enables C/C++ code to make use of the new MPX instruction set and registers introduced in the 6th Generation Intel® Core™ Processors (“MPX-enabled platform”).

The Microsoft Visual C++ Compiler* and linker can now generate checks automatically, a capability enabled by specifying a command line option.

This blog explains how you can use automatic MPX code generation and debug MPX-enabled binaries. For more details on Intel MPX, please see the Intel MPX Technology web page.

How to enable automatic MPX code generation

Visual Studio 2015 Update 1 introduces a new compiler option: /d2MPX.

/d2MPX currently supports:

Checking memory writes for potential buffer overflows. This provides protection for local and global pointers and arrays.

Extensions to the calling conventions to automatically propagate bounds associated with pointer arguments.

Usage Example

The following example is a program that contains an illustrative buffer overflow

Figure 2. Code with buffer overflow that will be detected with /d2MPX.

In Figure 2, the statement inside of the for loop would have overflowed the out array when it attempts to write past the end of the array since out is smaller than string str. Just before the program would have performed the out-of-bounds store, the MPX hardware will generate a #BR (bound range exceeded) exception, which is manifested as a structured exception handling (SEH) exception “Array bounds exceeded”. The default behavior in absence of an exception handler for the array bounds exceeded exception is immediate termination of the program. Alternatively, one can add an exception handler as shown in the example code to log the exception or to perform some context dependent recovery such as tearing down the process all the while having avoided the out-of-bounds store.

Steps to build and run the example:

Check that the Intel® MPX Runtime Driver is installed on your Microsoft® Windows® 10 November 2015 Update or greater system by verifying its presence in Device Manager under System devices (Figure 3). If it is absent, please download and install the driver from the Intel® Memory Protection Extensions Enabling Guide.

Install Visual Studio 2015 Update 1. Note, if Visual Studio is installed with the phone emulators, Hyper-V will have to be disabled (bcdedit /set hypervisorlaunchtype off and reboot) because this version of Windows does not expose MPX instructions to the guest.

Create a Win32 Console Application named “MPXExample” and use the code in Figure 2 for the driver code.

As noted above, please, double check that the /d2MPX option is enabled for the current Configuration.

Build the project for the X64 platform from within Visual Studio. This should produce an MPXExample.exe binary.

Execute the binary MPXExample.exe on an MPX-enabled platform with Windows 10 – which has the OS support for MPX.

To have the Visual Studio Debugger break on the array bounds exceeded exception, please enable the option for “Array bounds exceeded” in Exception Settings (Debug|Windows|ExceptionSettings) as shown in Figure 4. Executing MPXExample.exe in the debugger should now break on the exception (Figure 5). In this example, the #BR exception is thrown when MPX detects that we are about to write beyond the upper bound of the out array (Figure 6).

Figure 6. The exception is thrown when checking the upper bound as shown in this snapshot of the Disassembly window.

Visual Studio 2015 Update 1 supports the display and manipulation of the MPX registers via both the Register (Figure 7) and Watch windows (Figure 8) when running on an MPX-enabled platform.

Figure 7. To observe the contexts of the MPX bounds registers, enable MPX in the Debugger Register window.

Figure 8. Adding a bounds register to the Debugger Watch window is also simple. BND0.UB and BND0.LB in the Watch window refer to the upper and lower bounds in the BND0register respectively. Note that the upper bound of a bounds register is displayed in 2’s complement form.

How to tell if a binary is MPX-enabled

Run dumpbin /headers MPXExample.exe. The MPX debug directory entry should be similar to what is shown in Figure 9.

Figure 9. To tell if a binary is MPX-enabled, check whether a binary includes the mpx debug directory using dumpbin. The mpx debug directory should be listed in the Debug Directories section in the dumpbin output.

Must I compile everything with MPX?

You don’t have to compile all of your code with MPX enabled. A mixture of MPX and non-MPX enabled code will execute correctly. However, code compiled without MPX support will not have any MPX checks.

What hardware and version of Windows do I need?

To gain the benefits of MPX, MPX-enabled code should be executed on an MPX-enabled platform running a version of the Windows Operating System that is MPX aware. As of today, MPX is supported on the following:

What if I execute MPX-enabled code on a platform or on a version of Windowsthat does not support MPX?

The MPX-enabled code will execute correctly, but it will not benefit from MPX. You need to execute the code on an MPX-enabled platform with an MPX-aware operating system. The MPX instructions will be treated as NOPs, so you might experience a performance decrease in these scenarios.

Performance Impact

MPX technology provides a powerful safeguard against buffer overflow. Inserting checks for every write to memory may incur some execution time and memory footprint overhead. The amount of overhead is tolerable during testing. However, when enabled for production code, the developer must balance whether the improved memory safety outweighs their customers’ performance needs. We plan to improve performance based on feedback.

Known Issues

There is a known issue with x86 debug build where debug instrumentation interferes MPX operation.

Please try out automatic MPX code generation in Visual Studio 2015 Update 1. We are eager to hear about your experiences, especially in terms of usability, code size and runtime performance impact, and your suggestions for how to improve this feature. Please leave feedback in the comment box below or at the Intel ISA Extensions Forum on Intel® Developer Zone.

This is Juan from Intel. Thanks for noticing today's VC++ blog post. We have not done any specific comparisons vis a vis ASan. We'll take a look at the example on the link you provided.

One of the reasons why the feature is being rolled out in VC++ as experimental is to get the functionality out there and, based on feedback from users, develop actionable improvements in future VC++ updates.

Each compiler has implemented support for MPX given its design, deployment, and servicing constraints and priorities. What we wanted to focus on was on getting the functionality out to developers. Please take a look at the Updated Intel® Memory Protection Extensions Enabling Guide, which has been updated to cover VC++ 2015 Update 1.

We are looking for developer feedback on the feature: best way is to have developers out there give it a try, as your "mileage may vary" :)

Sounds very interesting technology but could you please describe your graphs using plain text? My screen reader can't read graphs and especially the platforms that this feature supports is important for me to know.

Running an application with MPX without the VS debugger attacher causes the application to crash when a buffer overflow occurs. Trying to attach VS to it when this happens causes the application to exit with code 0xc0000409

@Michael – MPX is not supported in Intel(R) Xeon(R) ES-2667. That product line is not based on the 6th Generation Intel® Core™ (code named "Skylake"). Please, follow up with me via email to juan.a.rodriguez at intel.com.

@Mike Diak – MPX is not supported on the Lenovo Z50 – per the specs, it is based on 4th Generation Intel® Core™, not the 6th Generation Intel® Core™ (code named "Skylake"). Please, follow up with me via email to juan.a.rodriguez at intel.com.

Jose – I will follow up with you, but this seems as it stands terribly flakey and fragile.

Microsoft – If only the very very very latest CPU's support this ability, is there really much point in supporting it yet especially in this immature format? Clearly it will be 1-2 years before there are significant numbers of PCs in the field with this hardware, particularly with the generally slower purchase rate of PCs and longer lifecycles before they are replaced.

Comments to both Microsoft and Intel.

1) Most users will not know whether their CPU supports this. Thus as a minimum – the driver installer needs to be a wrapped (e.g. Installshield installer) that can interrogate the CPU to check if the support is available BEFORE installing the driver.

2) The driver needs to fail safe. If it's running on an unsupported CPU, it needs to do something more sensible than just throw a BSOD, e.g. unload itself? This seems a terrible omision.

3) Surely the compiler/runtime (which supposedly treats this as a NOP) should do something sensible at application startup. If things are this risky stability wise, it would be a very brave developer who currently shipped software compiled with this option etc, due to the risk of BSODs.

Thanks for your feedback. I have pasted the Figure 2 sample code below..

// mpxexample.cpp

// compile with: /d2MPX

#include "stdafx.h"

#include <windows.h>

const int OUTBUFSIZE = 42;

wchar_t out[OUTBUFSIZE];

void copyUpper(wchar_t* str, size_t size) {

__try {

for (unsigned int i = 0; i < size; i++) {

// buffer overflow when attempting to write the 43rd wchar

out[i] = towupper(str[i]);

}

}

__except (GetExceptionCode() == STATUS_ARRAY_BOUNDS_EXCEEDED) {

wprintf(L"Caught array bounds exceeded exceptionn");

}

}

int main(int argc, char* argv[]) {

wchar_t str[] = L"the quick brown fox jumps over the lazy dog";

memset(out, 0, OUTBUFSIZE);

copyUpper(str, wcsnlen_s(str, 255));

wprintf(L"%sn", out);

return 0;

}

To take advantage of MPX you need a system that support the MPX instruction set. Our recently introduced 6th Generation Intel® Core™ Processors include MPX. In addition, you will need Microsoft® Windows® 10 November 2015 Update ("November Update") and the Intel® MPX Runtime Driver installed. Please refer to the Intel® Memory Protection Extensions Enabling Guide landing page link for further details.

I've emailed Jose. I can't help thinking that for most developers using the existing hardware techniques for finding writes/reads of unallocated memory via various libraries such as heapcheck (one I contributed to, based on Bruce Perens' famous electric fence) and various other techniques to do similar things in hardware, are preferable.

While these are not just a matter of recompiling, they do have the ability of running on todays and yesterdays hardware, and use hardware not software, so don't necessarily occur huge performance overheads….

MPX technology is designed to protect static as well as dynamically allocated buffers, such as those coming from a heap. For the dynamically allocated case, the allocators/libraries need to be provided which are MPX aware. The upcoming version of GCC 6 will enable MPX library support by default. However, at this point in time we do not have a similar set of library support for UCRT VC++ libraries.

Custom code can be written to “wrap” heap allocators like malloc with MPX intrinsics in lieu of the default version until a suitable MPX enabled set of libraries or wrappers can be made available. Please see the MPX Enabling Guide for further details.

Below is an example of an MPX wrapped malloc version.

void *__wrap_malloc (size_t n)

{

void *p = (void *) malloc (n); // call original malloc

if (p)

{

return __bnd_set_ptr_bounds (p, n); //bnd: [p, p+n-1]

}

return p;

}

Your feedback on support for MPX enabled libraries is something we are interested in capturing.

The feature can be enabled for both debug and release builds. The design of MPX technology is flexible and as such you, the customer/developer, needs to determine its suitability.

VC++ already provides features such as /GS, /analyze, /guard, secure CRT, and other capabilities. MPX is one additional capability available to help mitigate potential buffer overflow vulnerabilities.

As for whether you should only test/enable the functionality on x64, the answer is no. The issue is documented as issue 1 in the list of known issues in the Section 8.5.1 in the MPX Enabling Guide, which I am including below — this only impacts x86 debug builds.

1. Issue: The bound registers may be set to init [0, -1] in debug builds built with /RTC1 or /RTCs (Runtime checks flag). This is because of calls to RTC-related legacy functions _RTC_CheckESP and _RTC_CheckStackVars in the function epilog.

Please see Section 8.5.1 in the MPX Enabling Guide for the known issues and workarounds that exist. Good news is that we have fixes already implemented and are currently under internal review for them to be incorporated in a future Visual C++ update.

@ Juan Rod Are you sure they have been added ? They're not listed on MSDN anywhere and don't show in any VS2015 headers. Are you confusing with _byteswap_ulong ? Specifically I'm referring to the new Haswell instructions that are faster.

We can confirm that we have been able to reproduce the Blue Screen you and others experienced with the downloadable MPX driver package on systems that do not support MPX. The fix has already been identified, implemented, and is undergoing final validation. We plan to release the updated driver shortly for download and will get back here and on the Intel® Memory Protection Extensions Enabling Guide web site, software.intel.com/…/intel-memory-protection-extensions-enabling-guide, with an update.

If you do not have a 6th Generation Intel® Core based platform, please do not install the posted Intel® MPX Runtime Driver at this time.

We can confirm that we have been able to reproduce the Blue Screen you and others experienced with the downloadable MPX driver package on systems that do not support MPX. The fix has already been identified, implemented, and is undergoing final validation. We plan to release the updated driver shortly for download and will get back here and on the Intel® Memory Protection Extensions Enabling Guide web site, software.intel.com/…/intel-memory-protection-extensions-enabling-guide, with an update.

If you do not have a 6th Generation Intel® Core based platform, please do not install the posted Intel® MPX Runtime Driver at this time.

Adding /d2MPX options is not working, no MPX instruction is generated. The example is working on Update 1, but the same code and the same solution on Update 2 cannot generate any MPX related instructions.

I finally got a Skylake CPU to test this. Release mode works great with Update 3 RC and performance isn’t too bad. I would love to enable this feature for our unit tests and/or for debug mode, but unfortunately the incompatibility with the other runtime debug checks is a blocker. I really don’t want to disable them and I don’t think we’ll upgrade the CPUs in our build system, where unit tests are run in release mode, anytime soon.

A few folks from Intel monitor this blog on an ongoing basis, so please forgive our delayed responses.

Generally speaking: external customer input and requests are always taken into consideration…sharing your specific scenarios in your input is helpful as well in order to help build support for your use case. The more feedback/interest we will get collectively is appreciated. Please us this and other available mechanisms like User Voice: https://visualstudio.uservoice.com/forums/121579-visual-studio-2015