How to use CPU instructions in C# to gain performace

Introduction

Today, .NET Framework and C# have become very common for developing even the most complex applications very easily. I remember that before getting our hands-on on C# in 2002, we were using all kinds of programming languages for different purposes ranging from Assembly, C++ to PowerBuilder. But, I also remember the power of using assembly or C++ to use every small drop of power of your hardware resources. Every once in a while, I'm getting to a project where using regular framework functionality puts my computer to the grill for a couple days to calculate something. In those cases, I go back to the good old C++ or Assembly routines to use the power of my computer. In this blog, I will be showing you the simplest way to take advantage of your hardware without introducing any code complexity.

Background

I believe that samples are the best teacher; therefore, I'll be using a sample CPU instruction from Streaming SIMD Extensions (SSE). SSE is just one of the many instruction set extensions to X86 architecture. I’ll be using an instruction named PTEST from the SSE4 instructions, which is almost in all Intel and AMD today. You can visit the links above for supported CPUs. PTEST helps us to perform bitwise comparison between two 128-bit parameters. I picked this because it was a good sample using also data structures. You can easily lookup online for any other instruction set for your project requirements.

We will be using unmanaged C++, wrap it with managed C++ and call it from C#. Don't worry, it is easier than it sounds.

I would like to point out that there are many ways to develop a software and the one I’m providing here is maybe not the best solution for your requirement. I’m just providing one way to accomplish a task, it depends on you to fit into your solution.

Ok, let’s start with a fresh new solution: (I’m assuming that you have Visual Studio 2010 with C++ language installed on it.)

Add a new C# console application to your solution, for testing purposes.

This is the infrastructure for our unmanaged C++ code which will do the SSE4 call. As you can see, I placed our unmanaged code between #pragma unmanaged and #pragma managed. I think it is a great feature to be able to write unmanaged and managed code together. You may prefer to place the unmanaged code to another file, but I used one file for clarity. We used here two include header files, smmintrin.h and memory.h: the first one is for the SSE4 instructions and the other one is for a method I used to copy memory.

This _mm_testc_si128 will emit the SSE4 PTEST instruction. We have a little memory operation right before it to fill out the __m128i data structure on the C++ code. I used memcpy to transfer the data from the bufferA and bufferB arguments to the __m128i data structure to push it to the PTEST. I preferred to do this here to separate the whole SSE4 specific implementation. I could also send the __m128i to the PTEST method, but that would be more complex.

As I mentioned before, in this example I used the PTEST sample with a data structure, you may run into other instructions which require only a pointer, in that case you don’t need to do the memcpy operation. There might be some challenges if you are not familiar with C++, especially when the IntelliSense is removed in Visual Studio 2010 VC++, but you can always check out for online answers. For example, the __m128i data structure is visible in the emmintrin.h file, which is located somewhere in [Program Files]\Microsoft Visual Studio 10.0\VC\include. Or you can check all fundamental data types if you are not sure what to use instead of __int16*.

Now paste the following code on top of your managed C++ code. Which is on the bottom of your TestCPUInt.h file in the namespace section.

What we did here is to pass forward the pointers pBufferA and pBufferB, which we are going to call from C#, into the unmanaged code. For those who are not familiar with pointers, the * sign defines a pointer: __int16* means a pointer to a 16 bit integer. In our case, that is the address of the first element of an array.There are also ways without using managed C++ to call a dynamic library, but as I mentioned before, I’m showing only the simplest way for a C# developer.

Let’s go to our C# code in the console application to use this functionality.

First, we have to switch the application to allow unsafe code. For that, go to the project properties and check the “Allow unsafe code” under the build tab.

If you never used unsafe code before, you can check out unsafe (C# Reference). Actually, it is fairly simple logic; the PTestWPointer required a pointer to an array and the only way to get the pointer to an array is to use the fixed statement. The fixed statement pins my buffer array in memory in order to prevent the garbage collector to move it around. But that comes with a cost: in one of my projects, the system was slowing down because of too many fixed objects in memory. Anyways, you may have to experiment for your own project.That’s it!

But we will not stop here, for comparison I did the same operation in C#, as seen below:

staticint TestCLR(short[] bufferA, short[] bufferB)
{
//We want to test if all bits set in bufferB are also set in bufferAfor (int i = 0; i < bufferA.Length; i++)
{
if ((bufferA[i] & bufferB[i]) != bufferB[i])
return0;
}
return1;
}

Here, I simply calculate if every bit of bufferB is in bufferA; PTEST does the same.

On the rest of the application, I compared the performance of these two methods. Below is a code which does the comparison for sake of testing:

On my environment, I gained %20 performance. On some of my projects, I gained up to 20 fold performance. A last thing I would like to do is to show you how to move the fixed usage from C# to managed C++. That makes your code little cleaner like the one below:

This time, our method takes a managed array object instead of a __int16 pointer and pins it in the memory like we did using fixed in C#.

Conclusion

I believe that as much as higher level frameworks we are using, there will always be situations where we have to use our hardware resources more wisely. Sometimes these performance improvements save us big amounts of hardware investment.

Firstly, thanks for the article - intrinsics have largely passed me by before but they're something I'll look into.

I've one question for you. Is there any reason you create a native object simply to call the intrinsic? It seems to me that the cost of allocating the memory would outweigh the improvement of using the intrinsic and that wrapping the call in a static method would be more efficient.

The reason I ask is that I've recently been doing similar managed to unmanaged switches for performance gains but I've found there was a potentially large penalty from all the transitions from managed to unmanaged and back. These could be designed around to negate the cost but not always easily - have you found that using the separate object helps here? Is that a tip I've missed in my research?

Exactly that was my plan in one of the next weeks. Now I know how in a good article. Thanks

Btw. I know SSE and think your performance gain is understated. SSE is used for much bigger memory buffers. Use 10MB or more and do your performance test again. I rate about 50% and faster with SSE.
P.S. replace your memcpy calls with

a = *(_m128i*)bufferA;<br />
b = *(_m128i*)bufferB;

It's much faster. (I assume more CPU time in your code is used by memcpy than with the SSE command )

Do you think it is possible to remove all the copying - because if your pointer is pinned, you can access it from everywhere if I understood MSDN right. But for SSE we need to be sure that this pointer has correct memory aligning (16byte). Do you know how to ensure that? (Maybe an option of the garbage collector)

Yes, memory copying takes more time . It is possible to send a pointer to __m128i directly, but as I mentioned it lays outside the purpose of this example.

Pointing directly would realy make it faster if you can build the __m128i data structure in C# and provide a pointer to it. The example does not do that; it only sends a pointer to the raw data so we can fill the m128i_i16. But be carefull, with your change the pointer will be on m128i_i8 and not m128i_i16.