In my previous post I started to talk about the architecture of Naver. But this is actually still too early to tell, since before anything else, I want to create the low level interface for working with pages. And even before that, I had to decide how to actually represent a page in C#. In LMDB, this is easy because you can just access the memory using a pointer, and work with memory in this fashion is really natural in C. But in C#, that is much harder. It took me some trial and error, but I realize that I was trying to write C code in C#, and that isn’t going to work. Instead, I have to write native C#, which is similar, but different. In C, I can just pass around a pointer, and start doing evil things to it at will. In C#, there are a lot of road blocks along the way that prevent you from doing that.

So I ended up with this:

1: [StructLayout(LayoutKind.Explicit, Pack = 1)]

2:publicstruct PageHeader

3: {

4: [FieldOffset(0)]

5:publicint PageNumber;

6: [FieldOffset(4)]

7:public PageFlags Flags;

8:

9: [FieldOffset(5)]

10:publicushort Lower;

11: [FieldOffset(7)]

12:publicushort Upper;

13:

14: [FieldOffset(5)]

15:publicint NumberOfPages;

16: }

This is the memory layout that I want. But, there is not way in C# to say, allocate this struct at this location. Luckily, I found a workaround.

What I can do, I can have a Page class that manage this for me. This means that I just give the class a pointer, and it can treat this either as a PageHeader* or as a byte array. This means that I also get to do cool tricks like having an array backed by memory directly in the code:

1:publicushort* KeysOffsets

2: {

3: get { return (ushort*)(_base + Constants.PageHeaderSize); }

4: }

The Page class have a lot of methods relating to managing the page itself, and it should abstract away all the pesky details of actually working with memory directly.

So, this isn’t a really informative post, I fear. But it did take me a few hours to come up with the approach that I wanted. I kept trying to find ways to embed addresses inside the PageHeader, until I realized that I can just hold that externally. Note that all the important state about the Page is actually stored in memory outside the Page class, so that is shared globally, but you can have two Page instances that point to the same place. And that is where I decided to put local state. Things like where we are currently positioned in the page, for example.

Doing more research on this (http://lemire.me/blog/archives/2012/05/31/data-alignment-for-speed-myth-or-reality/) it seems there is no performance difference at all. Why are compilers avoiding this like the plague then?

Depending upon how you use the Page class, you can avoid the overhead of having a reference type altogether by just casting the byte* to a PageHeader* and then moving your methods/properties from the class to your struct directly. In that way, they will be manipulating the underlying byte[] directly, and there's nothing for the GC to clean up at the end of it.