RAM-less Buffers

Introduction

Have you ever looked at the x86 SSE or AVX registers in a debugger and wanted to use their bytes as a contiguous on-core buffer?

Clearly you can't get pointers into a buffer fashioned out of registers but it turns out that such a construction isn't hard to implement with some careful considerations in mind. We'll be using the g++ compiler in this post as MSVC doesn't currently support inline x64 assembly. We will be focusing on Windows and will be using MinGW to compile our code. We could also write this technique entirely in assembly but this option isn't as enticing because switch statements, templates, and classes make things very convenient. We will be targetting 64-bit processors that support SSE4 which allows us to only use XMM registers (as opposed to the newer YMM or ZMM extensions). The goal is going to be creating an array ADT whos backing storage is going to be XMM registers.

The reason why we won't be considering other operating systems is because the System V ABI doesn't preserve any of the XMM registers between calls and puts the burden on the caller to save them on the stack. If you think about it, this sort of defeats the purpose of using a register buffer if we're always going to be pushing our bytes to memory in user space.

Now that that's out of the way, let's take a look at the implementation.

XMM Registers

There are 16 individual 16-byte wide XMM registers which are meant to be used for SIMD and floating point operations. We won't be using them for their intended reason and instead just want to use their bytes as storage space. The Microsoft x64 ABI treats XMM0-XMM5 as volatile registers which aren't preserved between function calls and are used to pass doubles to functions and perform floating point operations (x87 is all but deprecated these days) among other things. However, registers XMM6-XMM15 are required to be preserved by the callee. We'll be using these registers to form our buffer so that we won't have to worry about functions inadvertently clobbering our bytes. If a function decides to use one of our buffer registers it will have to save/restore it using the stack per the rules of the ABI. This sort of makes the title of this post a misnomer but the kernel also saves our registers on context switch so RAM cannot be completely avoided. We're just having fun in the end anyway.

Since we'll be using XMM6-XMM15 that gives us 10 registers * 16 bytes = 160 bytes to play with in our buffer. But how do we manipulate the bytes in our buffer? There are a few instructions that will come in handy:

movdqa

This instruction allows us to move double qword (double(8) = 16 bytes) values between two XMM registers. An example of its use would be:

movdqaxmm0,xmm5; mov the dqword value of xmm5 into xmm0

We won't be using this to move memory into an XMM register so we won't have to worry about required memory alignment restrictions. There are different instruction mnemonic suffixes for different sizes of data we want to move. Another MMX/SSE mov variant would be movq which deals with 8 byte qword values rather than 16 bytes at one time.

pinsrq

The pinsr* family of instructions allows us to insert data at a specified offset into an XMM register. The suffix specifiers to this instruction will select the size of data we want to insert into the XMM. Let's look at some examples:

; Insert the qword value of rbx into the 0th location of xmm0.; If xmm0 = 0x00000000000000000000000000000000 and rax = 0xAAAAAAAAAAAAAAAA; after the following instruction executes xmm0 = 0x0000000000000000AAAAAAAAAAAAAAAApinsrqxmm0,rax,0; Using the same initial values as above, after this instruction executes; xmm0 = 0xAAAAAAAAAAAAAAAA0000000000000000. The immediate constant specifies; the offset multiple of the data size rather than the byte offset.pinsrqxmm0,rax,1; insert the qword value of rax into the 1*8th byte location of xmm0.; Last example. Assume that ebx = 0xFFFFFFFF and xmm0 is zero'd out. This instruction; will produce xmm0 = 0x00000000FFFFFFFF0000000000000000 because it's putting the ; dword of -1 into the 2*4=8th byte position in the number.pinsrdxmm0,ebx,2

We'll be using this instruction to insert data into our buffer. However, we will only be using the pinsrq size variant.

pextrq

The pextr* family of instructions extracts data at specified offsets. It's the inverse of pinsr*. We will be using this instruction to extract data out of our buffer.

; Get the first 8 bytes out of the xmm0 register and move them into raxpextrqrax,xmm0,0; Get the second 8 bytes out of the xmm0 register and move them into rbxpextrqrbx,xmm0,1

That's it. We'll only be using these three instructions to do everything we need. Let's move on to seeing how we can write this in C++.

The Implementation

The first thing we want to do is write a helper function which takes an offset and returns the qword at that offset. Every XMM register holds two qwords so we will need another function called get_reg_in_xmm0 which will copy the data from the dqword XMM register that contains our qword into XMM0 which is a volatile register whose value we don't need to care about preserving. After we have the dqword that we need in XMM0, we want to extract the correct 8 bytes from it. the pextrq instruction requires that the extraction offset be an immediate constant rather than a register. Because of this we will check to see if we need to extract the high or low qword from the XMM register and move it into rax. After that we can return the value that we extracted.

For the above function to work we need to be able to move the value of the XMM register corresponding to the requested offset into XMM0. This is easy enough using integer division to find the exact register that holds the data we need:

We can get qwords out of our buffer but we need to be able to set them as well. To do that we will repeat the process of obtaining the correct qword from the offset into XMM0. After that we will see which qword we need to set depending on if the offset corresponds to the high or low qword currently in XMM0. After that we just move the user-specified uint64_t value into rax and then move that into the right spot within XMM0. We then do the reverse of the get_reg_in_xmm0 function and move the dirty value we just created back to the correct XMM register for the specified offset.

The reason that we have been working on qwords instead of other datatypes right away is primarily because the pinsr* and pextr* instructions require an immediate constant offset into the specified XMM register. Currently we only need to specify 0 or 1 as the offset because there are only two qwords per XMM register. If we had more offsets because the datatype used was smaller, we would need to specify yet more offsets and have even more tests to see which exact part the user-specified offset fell under. By just operating on qwords and then (as we'll see in a moment) modifying the data within the obtained qwords at a higher level of abstraction, we save ourselves a lot of trouble.

Now that we have our support functions for getting/setting qword values, let's extend that to getting/setting any datatype as long as it is of byte/word/dword/qword size:

classregister_buffer{public:register_buffer(register_bufferconst&)=delete;voidoperator=(register_bufferconst&)=delete;// register_buffer is a singleton because there is only one instance of // the XMM registers. You cannot share this buffer between multiple threads as each// thread gets its own buffer (that is not accounted for in this implementation)staticregister_buffer&instance(){staticregister_bufferrb;returnrb;}template<typenameT>Tget(std::size_tindex){static_assert(sizeof(T)==1||sizeof(T)==2||sizeof(T)==4||sizeof(T)==8,"Invalid get<T> type size");// Get logical index into our buffer ADT and obtain the containing qword// from the logical indexstd::size_toffset=index*sizeof(T);uint64_tqword_val=get_qword(offset);// Position the requested value to be in the least significant bytes of // the qword before casting the qword to be of size T.uint64_tret=qword_val>>(offset%8)*8;return*reinterpret_cast<T*>(&ret);}template<typenameT>voidset(std::size_tindex,Tval){static_assert(sizeof(T)==1||sizeof(T)==2||sizeof(T)==4||sizeof(T)==8,"Invalid set<T> type size");// Get previously existing qword that contains the bytes we// want to setstd::size_toffset=index*sizeof(T);uint64_tqword_val=get_qword(offset);// Round down to the nearest multiple of 8 and then obtain// the distance between that multiple and the current offset.// This lets us easily set the correct bytes by using// pointer arithmeticstd::size_treg_index=(offset-(offset&~7))/sizeof(T);*(reinterpret_cast<T*>(&qword_val)+reg_index)=val;set_qword(offset,qword_val);}// num xmm regs in buf * xmm reg sizeconststd::size_tsize=10*16;private:staticvoidget_reg_in_xmm0(std::size_toffset);staticuint64_tget_qword(std::size_toffset);staticvoidset_qword(std::size_toffset,uint64_tval);register_buffer(){}};

Now that our class is fully implemented let's play around with our buffer:

This is great and all but what about at higher optimization levels? That's where things get hairy. It's best to just completely disable gcc optimizations for this class as there will be problems for -O2 and above with how this is implemented. The readability is a major factor here and we wouldn't gain a significant amount by obfuscating our code (as well as increasing the code size) to work better under higher optimization levels.

A commenter pointed out on the Hacker News post that inline assembly shouldn't be split up around the code as shown here. That is true and if you were to actually use this idea (for some reason) you should be more careful with how the compiler clobbers registers or assumes registers to contain certain values. For this article I think that it's best to demonstrate the concept rather than write the most correct code.