Low level programming of the LEGO Mindstorms EV3 Brick

I have a confession to make. I wasn’t planning on writing about anything other than LMS stuff on this blog, but I have something I need to get of my chest.

I have been cheating on you.

It’s true. I have had other projects besides hacking away on the EV3. I’m not proud of it, but that is how it is. And that is why I haven’t made any updates on this blog in a while. My other project sort of have a deadline and with christmas coming up finding time for one project is difficult, and finding time for two projects is near impossible for me.

So I will keep working on my other project for a bit, and when that is done I can come back to wrestle with the LMS VM on the EV3.

For a while I’ve been wondering about the performance of the EV3 and I’ve been thinking that since it is a 300 MHz ARM9 processor I figured the performance shouldn’t be an issue.

But after building a simple framework that draws one image using UI_DRAW BITMAP, and drawing 2 sprites that does UI_DRAW PIXEL for each pixel in the sprite I am already using up 90 of the 100 ms I have for each frame. This basically means that to achieve 10 fps I can’t do more than draw one background and two sprites.

So that is a bit disappointing!

There is a trick to copy the current frame buffer to a temporary buffer and then copy that buffer back to the frame buffer, and it is a cheap operation to do, so when I removed the UI_DRAW BITMAP that draws the background and only do that once, copy the drawn image to a buffer and then for each frame copy that buffer back to the frame buffer I get down to 59 ms per frame. So that’s a bit nicer.

If I keep all my code but remove the call to UI_DRAW PIXEL the frame time is 44 ms, so the actual drawing is somewhat expensive, but since we use quite a lot of frame time even without drawing then perhaps there is a chance to optimize this.

If I remove the reading of the pixel in the sprite I get down to a frame time of 21 ms. In these 21 ms it is all “my” stuff, meaning it is my inner loop in the sprite drawing, my algorithm to sort the game objects (bubble sort ftw), my overhead for managing game objects etc..

And if I remove the call to Sprite_Draw entirely I’m down to 1.5 ms. So the time is spent in that function.

So to summarize.

Full, old drawing: 90 ms
Drawing with the buffer trick: 59 ms
Drawing with the buffer trick but removing reading pixels from the sprite and writing them to the screen: 22 ms
Drawing with the buffer trick but removing the call to Sprite_Draw: 1.5 ms

So 57.5 ms is spent in this function (20.5 ms if we remove the calls to ARRAY_READ and UI_DRAW PIXEL):

In the game engine thingie I’m writing I have game objects. Btw, game engine sounds way more fancy and pretentious than what this is. Anyways, in the game .. framework, that I’m writing, I have game objects. I want to be able to load a scenario from disk, and in this scenario there should be items, like enemies, pickups, etc..

So I’ve created the concept of game objects, which I explained previously. A game object is basically just an array of handles. In C language, it means an array of pointers. Each handle, or pointer, refer to another part of the memory where content is stored. So a game object can have a piece of memory that explain the world position of a game object, another piece of memory that explain the AI state, and a third piece that explain the animation state.

So it is a retained mode model where the game programmer doesn’t immediately tell the framework to draw a sprite. Instead the game programmer load a sprite and assign it to a game object and then the framework have its own draw function where all the enabled game objects are drawn. This allow the framework to sort the draw order for all game objects. And that is what I did today, I wrote the sorting function.

It is a quite simple bubble sort algorithm. So far performance isn’t a concern of mine. I just want to get something up and running so I can start experimenting with a gameplay mechanic.

When all the array headache has been sorted out I’ve been able to go back to work on the actual game. (The game engine, rather.. )

Today I’ve expanded a bit on the component way of thinking. I used to be able to create a game object and add transform and sprite components to them, and today I also added an AI component. I also figured I want each component to have an update function that is called each frame.

Each component have an index into an array, so the transform is index 0, sprite is index 1, AI is index 3 etc.. So I figured it would be sweet if I could have a look up table with function pointers for each component. So I tried simply reading a label into a variable, and that seems to work fine. I made this little test:

and after that the APP halted with a VM verification error. I had a quick look in the code and it seemed like only constants could evaluate as labels, but now that I think about it my code wouldn’t have to evaluate as a label. If the label have the value 2 and my variable hold the value 2, and the code looks like this:

So I found the ARRAY_READ and ARRAY_WRITE OP codes and I have just tried them out. I won’t make a long blog post about it now, I’ll just say that it seems to work fine.

First I saw the call to cMemoryResize and got concerned. But then I saw that it was encapsulated in an if statement so it would only be called if the index of the write was greater than the number of elements allocated. Ok, I’ll make this post a bit longer then, since you’re asking so kindly for it.

As you can see there is a scary realloc going on on line 3966, but it is only called if the array is too small to fit the index requested, whereas ARRAY WRITE_CONTENT always resize the array, even if the index is within bounds.

I decided to only try the case I knew was broken with ARRAY_WRITE, and that is to write to an arbitrary index within bounds, and then write to a lower index and confirm that the contents of the higher index is still intact. And according to the log it works!

So great success! I know, I know, it is dangerous to be optimistic. So far I’ve mostly found out that stuff never works as well as they seem at first glance. But having looked through the code and written a test case I am optimistic about this.

Another positive thing about this is that the index you use is actually the index of the element, not the byte offset in memory.

So I’ll revisit my game code and make it work with this new array stuff, and let you know how it goes.

It is with a heavy heart I must inform you that I just can’t get those stupid addresses and handles to work!

I have tried allocating memory using ARRAY CREATE and reading and writing to it using READ_CONTENT and WRITE_CONTENT. It sort of worked but turned out that WRITE_CONTENT reallocates the buffer and it only allocates up to the entry I am currently writing to. So if I have an array of 4 entries and I write to entry 3, the content of entry 4 will become garbage.

I have tried reading the address of an array allocated with ARRAY CREATE but I haven’t been successful. I did manage to read the content of the array by doing @myHandle, but reading was never the issue. Writing was.

I also tried to get the physical address of some memory by using the & operator, but what that does is take in the parameter, read the content of the parameter and use as address from which other content is read and used as pointer.

Result = (void*)*(DATA32*)Result;

I then went on to try and statically allocate some memory by doing this:

Quick update since I’m on the move and don’t have access to the source code.

My final experiment last night was sort of successful. I allocated an array using ARRAY CREATE and on the handle that was returned I used the @ operator, so I did MOVE32_32( @handle, variable ) and the content of the variable was something like 0x00032ac30.

Then I did MOVE32_32( &variable, var2 ) and var2 was set to 0x0201.

So that seems great but the reason why I’m not yet happy is because I then tried to set the contents of the array with ARRAY WRITE_CONTENT and after that my experiment fell apart and I could no longer use the & operator, not even when I reverted back to the exact same code I had used when it worked the first time.

So in conclusion, this seems like it might work but I’m also having trouble taming it.

I’ve been looking more into the & operator, which seems pretty great. But it doesn’t seem to be fully supported.

When reading through the code that decode parameters, and looking at the comments, the code seems to fully decode the address, but the comment have completely emitted the information.

So basically, the compiler builds a stream of bytes form the LMS code. The byte stream is super simple. It is a byte that represent the opcode, and then a bunch of bytes representing potential parameters. If we look at the byte code for:

That code contain two calls to PrimParPointer, and what that does is first look at the information for the parameter, and then either fetch a constant, fetch a local or global variable. The bitfield is explained as a comment for PrimParPointer and looks like this:

According to the comment it is in long format, it is a constant, it is a value and it contains four bytes. The PrimParPointer will the go on to read those 4 bytes from the byte stream, which are as we saw earlier

0x44444444

The second parameter information is then:

Binary: 0100 0000

So according to the comment that means it is in short format, it is a variable, and the variable index is 0.

So that’s it, that is how the parameter encoding works. Those of you with a keen eye may have noticed that in the long format there is one bit that isn’t explained. Bit 3. (If you follow their naming of the bits) Bit 4 explain if it is a handle or not, but bit 3 is left out.

So it seems like the @ operator (handle) will return the pointer to an address based on a handle used when calling ARRAY( CREATEx, … ) and the & operator (address) will type case the Result (which is a pointer) and dereference that address to get a new address, which it returns.

I’m confused..

In any case, that 0x00 that snuck in straight after the 0xc9 seems misplaced, but other than that it seems like addresses may work. I will remove the 0x00 and see what the VM think about that.

So I looked at all the existing LMS files that shipped with the LMS2012 and none of them used the & operator. I’ve done some simple tests to just look at the compiled RBF file when doing & and see if I get any wiser. After that I’ll look into @ and see if that is anything I could use.