Friday, September 23, 2011

Managing Decoupling Part 4 -- The ID Lookup Table

Today I am going to dig deeper into an important and versatile data structure that pops up all the time in the BitSquid engine -- the ID lookup table.

I have already talked about the advantages of using IDs to refer to objects owned by other systems, but let me just quickly recap.

IDs are better than direct pointers because we don’t get dangling references if the other system decides that the object needs to be destroyed.

IDs are better than shared_ptr<> and weak_ptr<> because it allows the other system to reorganize its objects in memory, delete them at will and doesn’t require thread synchronization to maintain a reference count. They are also POD (plain old data) structures, so they can be copied and moved in memory freely, passed back and forth between C++ and Lua, etc.

By an ID I simply mean an opaque data structure of n bits. It has no particular meaning to us, we just use it to refer to an object. The system provides the mechanism for looking up an object based on it. Since we seldom create more than 4 billion objects, 32 bits is usually enough for the ID, so we can just use a standard integer. If a system needs a lot of objects, we can go to 64 bits.

In this post I’m going to look at what data structures a system might use to do the lookup from ID to system object. There are some requirements that such data structures need to fulfill:

There should be a 1-1 mapping between live objects and IDs.

If the system is supplied with an ID to an old object, it should be able to detect that the object is no longer alive.

Lookup from ID to object should be very fast (this is the most common operation).

Adding and removing objects should be fast.

Let’s look at three different ways of implementing this data structure, with increasing degrees of sophistication.

The STL Method

The by-the-book object oriented approach is to allocate objects on the heap and use a std::map to map from ID to object.

Note that if we create more than four billion objects, the _next_id counter will wrap around and we risk getting two objects with the same ID.

Apart from that, the only problem with this solution is that it is really inefficient. All objects are allocated individually on the heap, which gives bad cache behavior and the map lookup results in tree walking which is also bad for the cache. We can switch the map to a hash_map for slightly better performance, but that still leaves a lot of unnecessary pointer chasing.

Array With Holes

What we really want to do is to store our objects linearly in memory, because that will give us the best possible cache behavior. We can either use a fixed size array Object[MAX_SIZE] if we know the maximum number of objects that will ever be used, or we can be more flexible and use a std::vector.

Note: If you care about performance and use std::vector<T> you should make a variant of it (call it array<T> for example) that doesn’t call constructors or initializes memory. Use that for simple types, when you don’t care about initialization. A dynamic vector<T> buffer that grows and shrinks a lot can spend a huge amount of time doing completely unnecessary constructor calls.

To find an object in the array, we need to know its index. But just using the index as ID is not enough, because the object might have been destroyed and a new object might have been created at the same index. To check for that, we also need an id value, as before. So we make the ID type a combination of both:

struct ID {
unsigned index;
unsigned inner_id;
};

Now we can use the index to quickly look up the object and the inner_id to verify its identity.

Since the object index is stored in the ID which is exposed externally, once an object has been created it cannot move. When objects are deleted they will leave holes in the array.

￼

When we create new objects we don’t just want to add them to the end of the array. We want to make sure that we fill the holes in the array first.

The standard way of doing that is with a free list. We store a pointer to the first hole in a variable. In each hole we store a pointer to the next hole. These pointers thus form a linked list that enumerates all the holes.

An interesting thing to note is that we usually don’t need to allocate any memory for these pointers. Since the pointers are only used for holes (i. e. dead objects) we can reuse the objects’ own memory for storing them. The objects don’t need that memory, since they are dead.

Here is an implementation. For clarity, I have used an explicit member next in the object for the free list rather than reusing the object’s memory:

This is a lot better than the STL solution. Insertion and removal is O(1). Lookup is just array indexing, which means it is very fast. In a quick-and-dirty-don’t-take-it-too-seriously test this was 40 times faster than the STL solution. In real-life it all depends on the actual usage patterns, of course.

The only part of this solution that is not an improvement over the STL version is that our ID structs have increased from 32 to 64 bits.

There are things that can be done about that. For example, if you never have more than 64 K objects live at the same time, you can get by with 16 bits for the index, which leaves 16 bits for the inner_id. Note that the inner_id doesn’t have to be globally unique, it is enough if it is unique for that index slot. So a 16 bit inner_id is fine if we never create more than 64 K objects in the same index slot.

If you want to go down that road you probably want to change the implementation of the free list slightly. The code above uses a standard free list implementation that acts as a LIFO stack. This means that if you create and delete objects in quick succession they will all be assigned to the same index slot which means you quickly run out of inner_ids for that slot. To prevent that, you want to make sure that you always have a certain number of elements in the free list (allocate more if you run low) and rewrite it as a FIFO. If you always have N free objects and use a FIFO free list, then you are guaranteed that you won’t see an inner_id collision until you have created at least N * 64 K objects.

Of course you can slice and dice the 32 bits in other ways if you hare different limits on the maximum number of objects. You have to crunch the numbers for your particular case to see if you can get by with a 32 bit ID.

Packed Array

One drawback with the approach sketched above is that since the index is exposed externally, the system cannot reorganize its objects in memory for maximum performance.

The holes are especially troubling. At some point the system probably wants to loop over all its objects and update them. If the object array is nearly full, no problem, But if the array has 50 % objects and 50 % holes, the loop will touch twice as much memory as necessary. That seems suboptimal.

We can get rid of that by introducing an extra level of indirection, where the IDs point to an array of indices that points to the objects themselves:

This means that we pay the cost of an extra array lookup whenever we resolve the ID. On the other hand, the system objects are packed tight in memory which means that they can be updated more efficiently. Note that the system update doesn’t have to touch or care about the index array. Whether this is a net win depends on how the system is used, but my guess is that in most cases more items are touched internally than are referenced externally.

To remove an object with this solution we use the standard trick of swapping it with the last item in the array. Then we update the index so that it points to the new location of the swapped object.

Here is an implementation. To keep things interesting, this time with a fixed array size, a 32 bit ID and a FIFO free list.

I don't really see the point of your suggestion. What you are returning externally is a pointer to an object member. For that to remain valid, the object cannot be moved or deleted. But it seems in that case, you might just as well return a pointer to the object itself.

1) How are the systems only coupled on a higher level if they for example share the same transforms? I am not sure if in your examples you are coupling from inside the systems (lower level) or from a higher level. Currently I do this by feeding the systems batches of arrays (transforms, bodies, render properties) from an higher level. Multiple systems may share the same arrays but internally keep them in a different order. Is that what you are doing?

2) How do the systems share the same transforms but access them in a different order while keeping all access patterns sequential? For ex. the renderer needs to access the same transforms in texture order, the physics engine needs to access the same transforms in proximity order. Currently I do this by sorted lists for the renderer (initial radix sort, then merge sort every several frames) and keeping separate lists for physics (for each hashgrid cells).

3) When an object/entity gets deleted or a component from an object/entity is getting removed: How are you finally removing the holes? Are you swapping out one element every frame? How are you skipping the holes when sequentially processing the arrays. Are you using the index array to only process what is needed? I am not sure if this is what you are already saying in the article.

1) Usually by feeding, the higher level system extracts a list of transforms from one low level system and sends them to another low level system. Both systems keep there own local lists of transform.

2) Each system has its own list of data that it needs to "crunch" on. So the animation system, the scene graph and the renderer all have separate lists of object transforms, each sorted according to that system's preferences.

3) With the last solution, sequential processing is only done on the "object array", not on the "index array", which means there are no holes.

4) Each system keeps its own lookup table... there is no master lookup table.

@Niklas. Thank you for the answers. I was a bit confused by the terminology used, so I understand the part about the lookup arrays now. Sorry for that ;)

About the answer on the 1st question. How exactly is the higher level system feeding the transforms from one low level system to the other? Are you doing this every frame? As I can imagine that could potentially cause each low level system to constantly (each frame) re-sort its list of transforms to its preferred order (for example when every transform was changed by the physics system and is fed into another system which requires another order).

What I am currently doing is keeping the list of transforms the same for most systems so I can pass them around without re-sorting. The only system that will re-sort the list of transforms, is the render system. Can you advice me a better way?

Some of the nodes correspond to bones, but there are also some nodes that don't have corresponding bones.

A higher level system AnimationToSceneGraph knows about both systems and the connection between nodes and bones. Each update it does (in pseudocode):

for (i,tm) in Bones: Node[bone_to_node[i]].tm = tm

As you see this does not cause any list resorting. There is some extra copying going on, that you wouldn't get if you shared the data, but I think that is outweighed by the fact that each system can organize its own data so that it can process it as fast as possible.

There are no one-to-many relationships here. An ID is a way for an external system to refer to a single object. If the external system needs to refer to more than one object, it can keep a list of IDs.

Niklas, correct me please if I'm wrong, but there seems to be a small bug in your packed array implementation. Steps to reproduce:

1) Add MAX_OBJECTS number of objects without any removals. After adding the last one _freelist_dequeue will be equal MAX_OBJECTS. 2) Remove any object 3) Add a new object. Since _freelist_dequeue still points MAX_OBJECTS a memory corruption will occur

I guess, the fix can be is to add the following to the end of the remove() method:

Currently trying to grasp those (that I find great) design ideas of DoD and back-to-C I still have trouble wrapping my head around something.

When you do (in your example of October 10, 2011 in the comments here):

for (i,tm) in Bones:Node[bone_to_node[i]].tm = tm

Doesn't that make random access through the Node array completely defeating the cache access? Granted you will have at some point to do it, and here it's the Bone that update the node. But what when it's the other way around? A renderer, audio source, script needing the node transform? Wouldn't it be faster to have each renderer stocking the id of the corresponding node, and grab the transform during the update of the RendererManager? Instead of copying it and then iterating over the array, because in most case, you will only need it once for each object update.

I just can't manage to find a design where I won't have to do those random array access during the update anyway, loosing the point of having my data well contiguous in memory, because I always need to do random access for each of them each update, will it be in a pre-fetch like your exemple, or in the loop iterating over each of them to update them...

It is not complete random access, because you could take steps to ensure that you bone list is sorted in the same order as the node list and then it would still be in-order access (just skipping the nodes that don't have associated bones).

Even if you don't do that, random access in a small contiguous memory area (the node matrix list) is still a lot better than random access all over memory, because you will get fewer L2 cache misses.

After having read this and the previous three articles in this series (And that's all I've read of your blog so far), it seems like you are really comfortable working with raw memory, bit manipulation, etc. Do you have any recommended readings regarding any of those things; books, articles, entire websites, or anything comprehensive? I searched quite a bit for a book that covers data oriented design and things like that but maybe I am searching for the wrong things. I guess I should also ask: Have you just collected and applied things from many different sources and from having a good background in general computer science? I would very much like to have the understanding that you have when you created these neat and intelligent solutions.

For me it's been a journey over time... seeing that something is slow in C++, understanding why, stepping through code... (seeing what actually happens when you copy a nested STL structure, or call atof() on windows can be eye-opening) then thinking about how things can be done better, getting experience with doing those kinds of designs, seeing what other people do, etc.

Good things to try:

* Read some C books. C books usually focus more on what actually happens in memory than C++ that tends to gloss over things.

Thank you very much for your reply and your time. I think you've given me some very good advice. I'll be reading some C books and studying some containers, allocators, and everything else that might do something interesting or useful with memory.

good eye ... when i first read about spare sets, my first problem was: we have dense, we sparse arrays of integers, but where the heck i store my actual data? I guess OP spares extra array by introducing the ID in the actual data, Objects

In the event that you're battling with Epson wf-3640 printer disconnected blunders code and neglecting to pass on your printer returned into the online mode, you can contact with our gifted specialists to fix these sort of slip-ups with flawlessness. However, before reaching us, you can go to directly hereepson printer setup

Hello! My name is Andy Nickerson, a professional technical writer written immense blogs on Epson printer related to its problematic issues and top-notch troubleshooting procedures. Therefore, you are not needed to feel helpless also even feel alone in this technical world,There are many supporting candidates who provide right solution. Rather than wandering for knowing the solution of the problem Epson printer in error state lend an instant hand with me. I and my experienced squad is working full time only for your help. Hence, dial toll-free provided number and make one-to-one conversation for reliable and cost-effective solution.

Thanks for sharing this post , We work independently as a third party tech support provider for brother printer users. We provide 24/7 live support or help for brother printer users in very nominal charges. If you’re facing error code 2147500037, you can call our trained printer experts.

Hii Guys hope everybody is doing great! I am here to ask you the reason for the norton.com/setup. Actually, I have been frustrated with this setup and tried my best to install this antivirus but couldn’t install it. Do anyone of you know what the reason for this antivirus is and how to install it. Guys if you know the reasons because tell me in detail. I will be grateful for your assistance. Waiting for your response!

If you are a new user of HP printer and want to setup appropriately, then visit the link 123.hp.com/setup and follow-up the instructions. Once you successfully setup your HP printer, you can effortlessly print, scan or fax. Setup process consists of a printer driver to install, load a stack of white paper into the input tray, installing the ink cartridge, network connection, downloading the printer software on system and many more.

Thanks for sharing useful information Brother Printer can’t connect to wifi, which is one of the most common problems nowadays. Every day there are millions of people complaining about this kind of issue.

Nowadays many apps are allowing the users to enable the dark theme. Also, Snapchat dark mode is among those with fascinating features. Here we discuss easy and best steps to enable the dark mode in Snapchat - Snapchat Dark Mode

Epson printer is better known for its exceptional user-friendly features. Yet, in many instances, especially new users do not know how to setup Epson printer to a Wi-Fi network. If you are facing the same difficulties to setup your Epson printer to a wireless network, get connect with our Printer expert team straight away.

Setting up an email in QuickBooks is an easy process one can do, however, some users are not able to do it one their own. If you do not know how to setup email in QuickBooks in your system, you must get connected with our tech team to fetch email in your software instantly.

Data Recovery Service Center - Virus Solution ProviderData Recovery Customer Care We at Virusolutionprovider, understand the vital importance of your data and its significance in your business. We help you retrieve and recover your lost data due to any technical glitch or human error. Our programs are specially designed to scan whole memory hierarchy for lost data files and to retrieve the lost data back to the initial storage location. Our aim is to retrieve all of your data without any data or information loss. We have a skilled team with years of experience in the field of data recovery. We are committed to provide effective solutions related to data loss to our customers, with minimum response time and at optimal price.Go to the Official Website https://www.virusolutionprovider.com Call our Support Number = +91 (999) 081-5450 (WhatsAap call or Direct Call)

Forgot Yahoo Email Password is not the big thing. Yahoo also provides a password recovery option. Still, some users face issues while proceeding on the process in order to recover the forgotten Yahoo account password. If you are getting similar issue, you must know that you are just little bit away to get instant help.

Yahoo mail is no doubt one of the tough competitors against all other popular webmail service providers. But some of its users are complaining about the issue of Yahoo mail not sending emails. If you are too resentful about the same issue, get connect with our Yahoo executive team. You can avail of our Skype facility to connect with one of our experts.

Choose Assignment Help when you have no one to ask your concerns while studying at American universities. Students can connect with professional academic writers using online assignment writing services and discuss their concerns even staying at home. You don’t need to make any physical contact with anyone when you have the option of online services.help with assignmenthelp with my assignmentOnline Assignment Help

Although the downloading process of the HP printer assistant is easy, yet most of the users are having difficulties to find that. If you are not able to find the software or unable to proceed towards the HP printer assistant download, do not get annoyed, our expert team would help you out from that easily.

Superb blog and really great topic you choose. You know the most amazing thing I like in your article is your choosing words. Many times I see lots of new words in your article. I am also here for my Garmin Express web page promotion. Garmin Express in an app, which is used to manage Garmin devices. So if you using Garmin product and your Garmin Express Not Working or product requires Garmin express update then contact us from the Garmin Express Updates team. And please visit the web page for more information. Download Garmin express

I enjoyed over read your blog post i hope that you will be post more informative articles. Waiting for next article. Get help for your printer issues viva certified expert technician of our company 123helpline which is certified by many customer at any time 24 x 7 or visit our website www 123 hp com setup.

Installing the Avast antivirus software is an easy one can do. Some users have been reported that they are failed to know how to install Avast antivirus on their own. If you are too stuck around to install the program, feel free to contact our Avast expert and get instant help.

Norton security programs are easy to download and install. Still, most of the new users are not able to proceed through the Norton.com/setup download in order to get the program. If you are possibly facing some sort of issue while downloading the program, you must get connected with our technical team.

I must say you have written very nice article. Thanks for sharing it. The way you have described everything is phenomenal. You can follow us by visit our Web page HP Printer Wrieless Setup or Call our Toll-Free Number at anytime 24 x 7.

The printer goes to mistake state when your framework gets new updates making the associated gadget stop the activity. You ought to apply some investigating steps to determine canon.com-ijsetup.com. In addition, you can connect with specialists to fix the issue right away.

Epson Error Code | How to Troubleshoot Error Code 0X97?Epson printer is one such printing device that allows users to have a wide range of benefits. Along with this, the printer is also associated with several error code issues. If you are too getting one of the errors – Epson error code 0x97, you must get connected with our Epson printer experts.

How Do I Troubleshoot Gmail Not Receiving Emails 2020?The webmail – Gmail has been known for its quick sending and receiving emails. Yet, in some instances, users have been facing the issue of Gmail not receiving emails 2020. If you are not able to fix this sort of issue on your own, you must try contacting our Gmail technical executive right away.

If HP Envy 5540 Printer Offline Issues is one of the common problem for new printer or in old one. Hi Guys if you are looking for help regarding the HP Printer related any issues such as setup your printer or printer offline etc visit our Official website. Call our toll-free number we are available 24 x 7. Thank you for the post it was really good keep it up.

Yahoo mail is one of the well-known email service provided by Yahoo individuals to share any sort of message instantly with other users. However, supposedly, some of its users are facing the issue of Yahoo mail not responding properly in receiving emails. If you are facing such a similar issue in your mail account, you are just a call away to get the technical help to resolve your issue.

In the first place, access the Roku channel store and search for the PBS Kids channel. Click the “Add channel” tab and include the platform to the account. After this, open a web browser on your PC and type pbskids.org/activate roku. Tap the enter button and you’ll be redirected to the page. In this webpage, you have to paste and submit the code. This thereby activates the platform in the process. In case of any queries related to the PBS kids on roku, call the support team at +1-844-879-5200and get the issues fixed.