I have to admit it, I like good code. By good code, I mean event models, design patterns, Hungarian Notation and all that assorted zarjazz (a Jeff Minterism for all you old schoolers). I like it especially as it applies to game programming. I wasn't always like this, and I completely understand that good game code and optimized game code are more than often not diametrically opposed. So, I have made many efforts to create optimized game rendering engines (optimized game code) while implementing these good code practices. It has taken me a long time to get here, and now that I am, I'm not completely convinced I should or need to be here. So, I have decided to do a speed test with my preferred rendering engine using good code practices against a stripped down model. I am sure this down and dirty model is going to win, but by how much? I don't know, but I really want to find out. For some reason, I need to do this, and if you are interested, read further for the results.

I started making games in basic on the Apple IIe and Atari 800 in 1980. I was only 10 years old and admittedly, I had no fucking idea what I was doing. There were no user created functions other than the use of gosub and goto statements in the Apple and Atari versions of basic,. All of my game code resembled the over used, but accurate term, spaghetti. When I started to program in GFA Basic and STOS on the Atari ST, I was introduced to better procedural coding structures, but I was too uneducated to understand exactly how to properly implement the new tools in my arsenal. My code improved, but I am sure if I looked at it today, I would be horrified. At the same time I was learning Cobol, Object Vision, and object oriented design in some advanced college classes, but I never thought to apply those principles to my micro computer game programming.

After college, I fumbled around making games in C and Perl in Unix, and C++ on the PC, but all of these games were just slightly more organized and sophisticated than my earlier efforts. In 1996, I started making some real money coding early ASP in vbscript, and Perl 5 in Unix. I had to set games aside for a few years (I did do a couple in pure HTML and ASP Tic Tac Toe like games - ouch!) Client projects were coming at such a rapid rate that I never had a chance to take advantage of the formal coding structures (those that existed at least) in Microsoft's early web development system. I did write procedural code with methods, etc, but formal design patterns were beyond my sphere of interest or knowledge. Even when I was forced into Java, C#, and early Actionscript 1.0, I shied away from anything too sophisticated because time was of the essence. At that time, no one had any experience, and we were all fumbling our way into this. Iinexperienced designers always delivered late, unorganized assets and truly awful .fla files. Because of this, fast and easy code became the order of the day. Plus, clients had deadlines - realistic or unrealistic - and those needed to be met.

In 2000, I began to collect and read as many books on game design and programming as I could get my hands on. The best ones were written with C++ or Java in mind (with the exception of the excellent Jobe Makar Actionscript books). They all had discussions of design patterns and good object oriented practices (See Game Programming Gems). With those fresh in my mind, I decided to finally get up on that horse and make my games and apps with at least a subset of formal design patterns and object oriented practices. I designed a couple game engines and found that I really like the State Machine pattern when applied to games and AI. I have also found that an event driven message and update model can really save time and creates some very elegant code. You know what? I loved it, and my code started to become good, not great, but pretty good. When I opened up my source code files, they were filled with great comments, properly named variables, and simple methods that accomplished a single task. I also had class files and packages full of inheritance, polymorphism and abstraction. I was not an expert, but I was getting to be at least a knowledgable hacker. Still, Flash games don't necessarily lend themselves to this type of code, and very few examples of games with good code were available for the novice / intermediate level game programmer to look at.

Ever since, I have been searching for the RIGHT way to program a game. What I have found it that there really is no RIGHT way. There are a lot of good ways, but there are also trade offs at every corner. If I want to have good code, it might run slightly slower than code optimized for games. So what? Well, in Flash, we need every millisecond and every Frame Per Second squeezed out to make games fun to play with nice AI, physics, collision detection, and difficulty. Plus, we want them to be rendered fast. What to do?

When I began my efforts to create optimized arcade game clones and updates in Flash, it because necessary for me to create optimized bitmap rendering techniques well before any were documented on web sites, etc. Since I was entering uncharted Flash waters, I referenced some of my books written for other languages, and all of them used sprite sheets and even arrays of bitmap data for quickly updating sprite surfaces to do animation. With the Flash vector engine being so slow to update all of the points necessary to re-created 80's action games, I transported some of the Java and C++ ideas to my game, Retro Blaster. In that game, while I had a new, pretty sophisticated rendering engine, I still used loops and method calls instead of a formal event model. I implemented a formal objected oriented structure though. This made the game code clean and easy to follow and update. I found that I lost some sheer horsepower with this formal class structure. But, I decided not to go back because I wanted good code. The game came out fine, but it was slightly slower in some places than I would have wanted. When I decided to make a new game engine in AS3, I wanted to implement everything I had left out of the AS2 version - Event Models, State machines, and an even better, improved rendering engine. With AS3 supposedly 10-100X faster in execution (not rendering) speed than AS2, I assumed that adding all of my favorite formal coding structures wouldn't hurt, would it?

I few days ago, I published a speed test article where I timed execution speed and frame rate counts for a number of different Actionscript 3 rendering practices. I segmented the test between event and looping method call models and found that the looping method call model was slightly faster. I was able to get 15,000 game objects on the screen with both models, using an array to hold the BitmapData frames of animation for each object (using a shared reference for each of the 15,000). All of the objects render themselves with copyPixels to a single display object canvas. At lower object counts (1000 and under), the single display object canvas took much more time to process and execute than the rendering models where each object is responsible for rendering its own sprite to the screen. But, the models that render objects to the screen with individual sprites also crapped out at anything over 5000 on screen objects. While the SLOWER to execute, single canvas render models could with 15000 and beyond. What does this tell me? That execution speed and frame rate are not as tied as I once thought, also, there isn't one RIGHT choice if my game is going have fewer than 1000 objects on the screen. Although higher frame rates are possible with some models, they also use up precious cycles that might be used for other game activities.

One test I left out completely, was a speed test where I run the stripped down code. This is where the entire process runs from a single class without classes for other objects. No getters. no setters, only generic objects and functions that take objects as parameters and work on them as a reference. I have not created a game like this in years, so I completely forgot to test this one. So, here it is, Good Code vs. Optimized Code, I am almost certain the optimized version will have a higher frame rate, lower memory usage, and faster execution times, but how much faster? Is my good code worth doing at all?

Model - Stripped down

A single class

Number of objects

100

500

1000

5000

10000

15000

Render (frame rate) Peak

125

125

102

27

14

10

Render (frame rate) Avg

124

124

101

26

13

9

Model Times Peak

Milliseconds

2

5

10

44

89

126

Model Times Avg

Milliseconds

0.93

3.639

7.369

36.373

73

107.936

Creation Time

Milliseconds

1

5

9

55

100

161

Memory Usage

Kilobytes

4612

4706

4796

5545

6447

7356

Frame Rate
The best frame rate I was able to achieve using the same render engine, but with an more formal object oriented structure was 8 FP. With the stripped down model I am able to squeeze out one more frame and get an average of 9 FPS. So, it seems that I cannot squeeze out too many more frames because the rendering engine is pretty solid already. At this low a frame rate though, every frame counts and this is an 11% improvement.

Execution TimeMy previous best average execution time for updating and rendering 15000 objects was 123 milliseconds. With this stripped down engine, my average is 107.936. This is a 12% improvement.

Object Creation Time
My previous best one time object creation time for 15,000 objects was 402 milliseconds. AS you can see above, this is where the stripped down model really shines. It takes only 161 milliseconds to create and initialize 15,000 generic objects. This is a a 249% improvement!

Memory Usage
Memory usage is also a place where the stripped down version clearly outshines the object oriented version. Previously, 17211 KBytes was the least amount memory used for 15,000 objects. With the new stripped down engine, I use only 7356 KBytes. This is a 233% improvement.

Overall winner: Optimized Code (Stripped Down engine)

What does this all mean?
Well, the rendering performance difference is not enough to make my feel I have to switch away from my Good Code to more efficient game code. I do like the improvements in execution time, and especially memory usage and object creation time though. I guess it all depends on the type of game I am going to make. I love Object Oriented Design Pattern code and how reusable and time efficient it is (from a development standpoint). I can't imagine making a game with absolutely no events or formal object structure because to me it would become a spaghetti mess very soon. Once the engine demo is done, all of the necessary evils in actually making a real game would get in the way. To me, there is no better way to handle things like sounds and music than with a singleton Sound manager class. The same goes for in-game rendered messages, screens, level data, tile data, etc. But, with this new information,when I start to design my GameLoop, I will think twice before I make formal classes for games objects that need high speed rendering.

If you want to test out Speedtest2 on your machine, be my guest. In the browser you can run the test below (your FPS will max out at about 60), and the swf file is available below for you to try from your desktop.

About a week ago I jumped into a discussion on Flashkit about AS3 sprite sheets, rendering, etc. There were a lot of good ideas being thrown around, and I put in my 2 cents on how my current engine works. Originally, back in the AS2 days (and early days of this blog), I did some tests with my original BitmapData caching engines and reported some of the results in a blog entry called Retro Blaster is Coming. In it, I detail how I used an Array of BitmapData objects in place of gotoAndPlay and achieved some very nice results. When I started to create a new game engine in AS3, I consulted Chris Cutler, who is an excellent Flash Game programmer and my partner in crime at my day job. He had created a fully blitted side scrolling engine for the Pixel Chix web site in AS2. After consulting him on his basic methodology, I set out to create my own version is AS3. These were the principals I wanted to follow:

1. I will display ALL game objects (including any text that occupies the same area as sprites and tiles) on ONE canvas. This would require the manipulation and blitting of BitmapData for all game objects and a copyPixels operation to update them to the screen.

2. I would hold all of the sprite sheets in arrays of BitmapData for ease and speed. I call these virtual sprite sheets.

3. I would use the new event model to fire off render events to my game objects.

With those goals in mind, I created my new game engine and the forth-coming game, Pumpkin Man to test it. The engine has been working pretty well for me, but as soon as I participated in the Flashkit discussion I began to wonder just how my current engine compares to other engines. Since this is the most optimized engine I have ever created, I dropped my game development for a few days and began to code up a way to test both screen render speed (using FPS as my units) and a method to calculate the length of time the Event model was taking compared to a straight method call to update objects.

I needed to bombard Squize (from Flashkit and the Gamingyourway.com site), as well as Chris Cutler, Jobe Maker, and Mike Grundvig to find any new ideas for rendering the screen. I discovered two methods I had not tried, so I added them to my test. The first method was to replace the Array that holds the Bitmapdata objects with a single sprite sheet strip. Each game object would be an individual sprite (as opposed to blitting the all to a single sprite). I would use the scrollrect property of the Bitmap object inside the sprite to control the animation. It turned out to not be such a hot idea. I can't blame any of those above for this method because I just made it up after talking to them. It seems that the individual sprites really slow the render throughput, and the scrollrect seems to be painfully slow given that is had to work on so many individual sprites at the same time. I tried and failed to create a version of this with one shared Bitmap (as to only have to update one scrollrect property) but when I attempted to copyPixels from it, I always got the first frame of animation.

After hitting up Squize a few more times, I came to realize that he was using scrollrect to actually only animate ONE sprite, that being the background and play areas of his great new Geometry Wars-like game (he let me preview it). So, it seems that I was using the scroll rect in a completely wrong manner. After talking to Mike Grundvig, I came to realize, that I needed to figure out a whole new way to attempt to scrollrect through a sprite sheet. What I came up was using the same sprite sheet strip instead of an array, but to still blit them all to a single sprite by changing the copyPixels rect (not the scrollrect property) on each frame to hover over just the bitmap I needed.

Anyway, after finally getting a solid foundation in the 5 different rendering engines I wanted to test, I coded up a Flash App that can be used to test and get the results for different combinations of object counts, rendering methods, and what I call models: using either and event or a method to update objects. In this way I would be able to tell which of my 5 different methods was the best. As an aside, even though I got some great results in some of my tests, I have not tested any of these against other people's rendering engines, so I have no no idea if these are the 5 best choices to test, but they are 5 of my methods. I am sure there are better methods out there.

Method # 1: Animate with an array. Output to one sprite.This was my current engine of choice when I began this test. The code for the render() method looks like this:

Basically, each game object runs this method to jump to the next frame of animation. The array of BitmapData that makes up this virtual sprite sheet is called aSpriteBitmapData. The index of that array is updated only after animationFrameDelay has passed since the last animation frame change. The blitPoint is always the current x,y of the object, and the blitRectangle is always just this: blitRectangle=new Rectangle(0, 0, 32, 32); All objects blit to a single BitmapData called : screenBD (in my parent SpeedTest class);
Up until now, I had assumed that this would be the fastest method I could come up with.

Model # 2: Animate with an array. Output to individual sprites
The only real conceptual difference between this and #1 is how the game objects are rendered on the screen. I still use a virtual sprite sheet Array of BitmapData to hold the animation. I just have each game object render itself as a sprite to the screen instead of blitting to one BitmapData object. This gives me more control over the the Sprites and allows me to make use of Sprite properties such as rotation and scale is needed. The code for the render() function looks like this:

Instead of blitting to a single BitmapData object, I simply update the BitmapData property of the Bitmap used to display the object on the screen: displayBitmap.bitmapData=aSpriteBitmapData[spriteBitmapDataIndex]; This animation Array is handled the same as above in Model #1. This is essentially the the render model I built in AS2, only in AS2/Flash 8 I used the attachBitmap method of a MovieClip as opposed to changing the reference to the BitmapData on a Bitmap object.

Model# 3: Animate with scrollrect. Output to individual sprites.This method makes use of a single BitmapData strip inside a Display Bitmap instead of a virtual sprite sheet in an Array. It is a literal sprite sheet. On each frame update the scrollRect property of the Bitmap is updated to the needed positron and it acts like a MASK, just showing the part of the sheet I want for that particular frame.

I had high hopes for this one, but it didn't turn out as expected. This was not quite as fast as I thought it would be. We'll examine the results in detail the later on in this article.

.Model # 4: Animate with GAS. Output to individual sprites.I did this one on a lark. I was frustrated with the results of my pathetic scrollrect experiment above, so I decide to test it against a basic gotoAndStop implementation. I built a MovieClip in the library with all 8 frames and a stop on each frame. I copied the Bitmaps from my sprite sheet to individual MovieClips and added one to each frame. This gave me basically what I would have had in AS2/Flash 8 before I began to use BitmapData.

The code for this render() is very straightforward, it just play()s to the next frame after the appropriate animationFrameDelay has been reached.

.Model # 5: Animate sprite sheet with rect. Output to one sprite.
This is a combination of #1 and #3. I use ONE BitmapData canvas inside one Sprite to blit to, but instead of an virtual sprite sheet array, I use a real Bitmapdata object in memory as the literal sprite sheet. The rect property of the copyPixels operation is manipulated to achieve the illusion of animation. This the one I came up with after consulting Squize, Cutler, Maker, and Grundvig.

The blitPoint and blitRectangle must be updated each frame to scroll along the sheet and ensure the data is blitted to the correct position on the screenBD canavas.

RESULTS:

My basic methodology for the test was to make sure the only programs running on my machine were Excel and the external .swf player for CS3. I did all of these tests outside of the Browser and the Flash IDE as to not be encumbered by the limitations of each. I wrote the testing program in a way that allows me to perform multiple tests without shutting down an restarting the program, but because I am sure garbage collection would have played a part in changing some of the results, I opted to close and restart the testing app between each run. Each test was run for 1000 frames.

100 Object Test. Below are my results for the first run with 100 objects animating and moving across and down the screen. I have noted the peak frame rate and render frame rates and model times. The model time is calculated through the use of the older get Timer() method because it is the only timing method I could find that would actually give me accurate results (Thanks for your help on this one, Chris Cutler). For the Method model, I take the get Timer() time before I start the loop through all of the objects and again when the loop is complete. I have optimized this loop to the best of my ability. For the EVENT model, I took the get Timer() before the event dispatch, and took one on the line directly beneath. It seems that Flash does call these in a synchronous manner, so it seems to work fine.

Model

METHOD

EVENT

100 Objects

Render (frame rate) Peak

FPS

FPS

#1 Array, One Sprite

125

126

#2 Array, Multiple Sprites

125

126

#3 Scrollrect, Multiple Sprites

125

125

#4 GAS, Muliple Sprites

109

109

#5 Sprite Sheet, One Sprite

126

125

Render (frame rate) Avg

FPS

FPS

#1 Array, One Sprite

124

124

#2 Array, Multiple Sprites

124

124

#3 Scrollrect, Multiple Sprites

124

123

#4 GAS, Multiple Sprites

107

107

#5 Sprite Sheet, One Sprite

124

124

Model Times Peak

milliseconds

milliseconds

#1 Array, One Sprite

4

11

#2 Array, Multiple Sprites

3

10

#3 Scrollrect, Multiple Sprites

11

10

#4 GAS, Multiple Sprites

1

10

#5 Sprite Sheet, One Sprite

2

2

Model Times Avg

milliseconds

milliseconds

#1 Array, One Sprite

0.958

0.978

#2 Array, Multiple Sprites

0.0069

0.013

#3 Scrollrect, Multiple Sprites

0.011

0.037

#4 GAS, Multiple Sprites

0.0519

0.065

#5 Sprite Sheet, One Sprite

0.975

0.969

Creation Time

milliseconds

milliseconds

#1 Array, One Sprite

3

3

#2 Array, Multiple Sprites

5

7

#3 Scrollrect, Multiple Sprites

6

8

#4 GAS, Multiple Sprites

13

13

#5 Sprite Sheet, One Sprite

3

3

Memory Usage

Kilobytes

Kilobytes

#1 Array, One Sprite

4747

4915

#2 Array, Multiple Sprites

4788

5050

#3 Scrollrect, Multiple Sprites

5279

5558

#4 GAS, Multiple Sprites

5152

5320

#5 Sprite Sheet, One Sprite

4661

5001

All of the render methods performed very well from an FPS point of view. All of these achieved at least 109 FPS with 100 objects. GAS was the slowest, but all of the others averaged at least 123 FPS no matter which Model was used. It is interesting to note that the average render time for the model was much much lower for the array of sprite versions (especially the event model). The single bitmap blit (#1, and #5) had a much higher average millisecond MODEL run rate. This didn't seem to effect the frame rate much, as those achieved some of the highest FPS rates.

The average model time difference between Event and Method was was not as significant. In some cases the EVENT was actually a little faster than the METHOD call. What really effected the model execution time was the render() method chosen.

Consistently, object creation time was the close between the two Models, but varied wildly between the different rendering methods, with GAS taking the most time to initially create 100 MovieClip objects. Memory usage

Memory usage was pretty consistent, with the EVENT model versions taking up a little more ram, while not surprisingly, the Sprite Sheet, Single Sprite (Method #5) overall used the least amount of memory (in METHOD model mode).

.500 Object Test. These were conducted in the same manner as the 100 object tests. Note that at 500 objects the scrolled (#3) and GAS (#4) are starting to become unusable.

Model

METHOD

EVENT

500 Objects

FPS

FPS

#1 Array, One Sprite

125

125

#2 Array, Multiple Sprites

125

124

#3 Collect, Multiple Sprites

21

20

#4 GAS, Multiple Sprites

25

26

#5 Sprite Sheet, One Sprite

125

125

Render (frame rate) Avg

FPS

FPS

#1 Array, One Sprite

124

124

#2 Array, Multiple Sprites

124

123

#3 Scrollrect, Multiple Sprites

19

18

#4 GAS, Multiple Sprites

22

23

#5 Sprite Sheet, One Sprite

124

123

Model Times Peak

Milliseconds

Milliseconds

#1 Array, One Sprite

6

14

#2 Array, Multiple Sprites

2

13

#3 Scrollrect, Multiple Sprites

103

106

#4 GAS, Multiple Sprites

2

12

#5 Sprite Sheet, One Sprite

7

15

Model Times Avg

Milliseconds

Milliseconds

#1 Array, One Sprite

3.962

4.1

#2 Array, Multiple Sprites

1.015

1.162

#3 Scrollrect, Multiple Sprites

1.086

1.173

#4 GAS, Multiple Sprites

1

1.128

#5 Sprite Sheet, One Sprite

4.028

4.585

Creation Time

Milliseconds

Milliseconds

#1 Array, One Sprite

9

16

#2 Array, Multiple Sprites

28

36

#3 Scrollrect, Multiple Sprites

29

37

#4 GAS, Multiple Sprites

79

92

#5 Sprite Sheet, One Sprite

10

15

Memory Usage

Kilobytes

Kilobytes

#1 Array, One Sprite

5058

5390

#2 Array, Multiple Sprites

5480

5996

#3 Scrollrect, Multiple Sprites

7364

9207

#4 GAS, Multiple Sprites

7767

8830

#5 Sprite Sheet, One Sprite

5033

5308

Again we see that the Event Model is slightly slower in execution speed than the Method Model. We also see that the average execution speed for #1 (Array of Bitmap Data, one display object) and #5 (Sprite Sheet and one display object) is the slowest, but did not affect average frame rate. Those were the fastest versions. It is interesting to note that I might actually choose #2 (array of bitmapData Multiple Sprites) if I was going to have about 500 Sprites on the screen because the frame rate is high, and the average execution speed is the lowest of the three that actually performed well. The execution speed difference would help with things like collision detection and other in game calculations. The down side to #2 is object creation time is the highest of the leading three, so the use of a sprite object pool would be absolutely necessary. Memory usage was pretty good across the board for #1, #2, and #5. These seem to be the best FPS models so far.

1000 Object Test. The same methodology as the 100 and 500 tests was used for this test. ScrollRect (#3) and GAS (#4) are becoming pretty much unusable, and the 1000 frame tests for both took quite a few minutes to complete.

Model

METHOD

EVENT

1000 Objects

Render (frame rate) Peak

FPS

FPS

#1 Array, One Sprite

94

93

#2 Array, Multiple Sprites

92

93

#3 Scrollrect, Multiple Sprites

8

8

#4 GAS, Multiple Sprites

11

11

#5 Sprite Sheet, One Sprite

93

89

Render (frame rate) Avg

FPS

FPS

#1 Array, One Sprite

93

91

#2 Array, Multiple Sprites

91

91

#3 Scrollrect, Multiple Sprites

6

6

#4 GAS, Multiple Sprites

9

9

#5 Sprite Sheet, One Sprite

92

87

Model Times Peak

Milliseconds

Milliseconds

#1 Array, One Sprite

11

20

#2 Array, Multiple Sprites

5

17

#3 Scrollrect, Multiple Sprites

103

134

#4 GAS, Multiple Sprites

4

26

#5 Sprite Sheet, One Sprite

12

22

Model Times Avg

Millisecond

Millisecond

#1 Array, One Sprite

8.22

8.682

#2 Array, Multiple Sprites

2.959

3.066

#3 Scrollrect, Multiple Sprites

2.055

2.02

#4 GAS, Multiple Sprites

2.008

2.36

#5 Sprite Sheet, One Sprite

8.707

9.01

Creation Time

Milliseconds

Milliseconds

#1 Array, One Sprite

29

49

#2 Array, Multiple Sprites

73

92

#3 Scrollrect, Multiple Sprites

71

107

#4 GAS, Multiple Sprites

155

187

#5 Sprite Sheet, One Sprite

29

49

Memory Usage

Kilobytes

Kilobytes

#1 Array, One Sprite

5414

6295

#2 Array, Multiple Sprites

6225

7061

#3 Scrollrect, Multiple Sprites

11943

12173

#4 GAS, Multiple Sprites

13479

13037

#5 Sprite Sheet, One Sprite

5369

5644

Consistently, #1 using a Method loop to update the sprites is the fastest frame-rate wise. The arrays of BitmapData consume a little more memory than #5 (Using a Sprite sheet instead of an Array). Both #1 and #5 take about 4x longer of processor time on average to call the rendering functions, but this is more than made up for with the throughput of blit rendering to a single BitmapData object. Creation time is still under 30 milliseconds for #1, and #5.

#2 (Using an array of BitmapData for animation, but rending to 1000 separate sprites is still a contender here, and in pure FPS, it even slightly beats #5. It has a lower execution time (2.9 milliseconds compared to 8+ for #1 and #5); The only drawback to #2 is the creation time for the 1000 objects. It takes 2x as long to initialize them as #1 and #5. Also You can see that the execution times for 1000 objects using the EVENT model, while slightly slower in all areas, is very competitive. There is no absolute clear winner here because while #2 is slightly slower frame rate-wise, the execution time would make it easier to squeeze in more collision detection and AI to your 1000 objects.

5000 and 10000 Object Tests. I lumped these two tests together because the results are very similar, though obviously the frame rates are going down, and the execution times are going up.

5000 Objects

Model

METHOD

EVENT

5000 / 1000 Objects

5000

5000

Render (frame rate) Peak

FPS

FPS

#1 Array, One Sprite

24

22

#2 Array, Multiple Sprites

18

18

#3 Scrollrect, Multiple Sprites

x

x

#4 GAS, Multiple Sprites

x

x

#5 Sprite Sheet, One Sprite

22

22

5000

5000

Render (frame rate) Avg

FPS

FPS

FPS

FPS

#1 Array, One Sprite

23

21

#2 Array, Multiple Sprites

17

17

#3 Scrollrect, Multiple Sprites

x

x

#4 GAS, Multiple Sprites

x

x

#5 Sprite Sheet, One Sprite

22

21

5000

5000

Model Times Peak

Milliseconds

Milliseconds

#1 Array, One Sprite

44

64

#2 Array, Multiple Sprites

30

48

#3 Scrollrect, Multiple Sprites

x

x

#4 GAS, Multiple Sprites

x

x

#5 Sprite Sheet, One Sprite

49

56

5000

5000

Model Times Avg

Milliseconds

Milliseconds

#1 Array, One Sprite

40.971

43

#2 Array, Multiple Sprites

27

26

#3 Scrollrect, Multiple Sprites

x

x

#4 GAS, Multiple Sprites

x

x

#5 Sprite Sheet, One Sprite

43.442

44.18

84.66

87.435

5000

5000

Creation Time

Milliseconds

Milliseconds

#1 Array, One Sprite

151

525

#2 Array, Multiple Sprites

601

1322

#3 Scrollrect, Multiple Sprites

x

x

#4 GAS, Multiple Sprites

x

x

#5 Sprite Sheet, One Sprite

139

528

5000

5000

Memory Usage

Kilobytes

Kilobytes

#1 Array, One Sprite

8826

10833

#2 Array, Multiple Sprites

12627

14573

#3 Scrollrect, Multiple Sprites

x

x

#4 GAS, Multiple Sprites

x

x

#5 Sprite Sheet, One Sprite

8806

9973

10000 Objects

Model

METHOD

EVENT

5000 / 1000 Objects

10000

10000

Render (frame rate) Peak

FPS

FPS

#1 Array, One Sprite

12

12

#2 Array, Multiple Sprites

9

9

#3 Scrollrect, Multiple Sprites

x

x

#4 GAS, Multiple Sprites

x

x

#5 Sprite Sheet, One Sprite

12

12

10000

10000

Render (frame rate) Avg

FPS

FPS

#1 Array, One Sprite

12

11

#2 Array, Multiple Sprites

8

8

#3 Scrollrect, Multiple Sprites

x

x

#4 GAS, Multiple Sprites

x

x

#5 Sprite Sheet, One Sprite

12

11

10000

10000

Model Times Peak

Milliseconds

Milliseconds

#1 Array, One Sprite

88

107

#2 Array, Multiple Sprites

64

85

#3 Scrollrect, Multiple Sprites

x

x

#4 GAS, Multiple Sprites

x

x

#5 Sprite Sheet, One Sprite

89

116

10000

10000

Model Times Avg

Milliseconds

Milliseconds

#1 Array, One Sprite

82.572

86.555

#2 Array, Multiple Sprites

59.82

61.61

#3 Scrollrect, Multiple Sprites

x

x

#4 GAS, Multiple Sprites

x

x

#5 Sprite Sheet, One Sprite

84.66

87.435

10000

10000

Creation Time

Milliseconds

Milliseconds

#1 Array, One Sprite

273

1764

#2 Array, Multiple Sprites

3847

7279

#3 Scrollrect, Multiple Sprites

x

x

#4 GAS, Multiple Sprites

x

x

#5 Sprite Sheet, One Sprite

290

1774

10000

10000

Memory Usage

Kilobytes

Kilobytes

#1 Array, One Sprite

13062

17108

#2 Array, Multiple Sprites

20627

23298

#3 Scrollrect, Multiple Sprites

x

x

#4 GAS, Multiple Sprites

x

x

#5 Sprite Sheet, One Sprite

13004

15085

Both the scrollrect and gas rendering engines (#3 and #4) ran too slow for me to get any good calculations. I left to take a shower and 30 minutes later the scrollrect version was still churning along. What is consistent is we cannot get an average frame rate above 23 for 5000 and 12 for10000, and #5 has finally started out outpace #2. The king of FPS is still #1 and the execution time gap between #2 and #1/#5 is closing. It seems that at 5000 objects, individual sprites are no longer a very good idea at all, especially give then 3+ seconds it takes just to initialize and create the objects. Memory usage is starting to become closer between #1 and #5 (arrays vs sprite sheets), and the Event model, while still slower, and more of a memory hog can sustain a pretty even frame rate (if slightly lower than the method call model).

15000 Object Test. This test started to show the seamy underbelly of my testing engine as the #2 tests could not finished because of exception errors in object cleanup and creation. The only number I could get was a max 6 FPS while watching the FPS counter.

Model

METHOD

EVENT

15000 Objects

Render (frame rate) Peak

FPS

FPS

#1 Array, One Sprite

8

8

#2 Array, Multiple Sprites

6

x

#3 Scrollrect, Multiple Sprites

x

x

#4 GAS, Multiple Sprites

x

x

#5 Sprite Sheet, One Sprite

8

8

Render (frame rate) Avg

FPS

FPS

#1 Array, One Sprite

8

7

#2 Array, Multiple Sprites

x

x

#3 Scrollrect, Multiple Sprites

x

x

#4 GAS, Multiple Sprites

x

x

#5 Sprite Sheet, One Sprite

8

7

Model Times Peak

milliseconds

milliseconds

#1 Array, One Sprite

130

141

#2 Array, Multiple Sprites

x

x

#3 Scrollrect, Multiple Sprites

x

x

#4 GAS, Multiple Sprites

x

x

#5 Sprite Sheet, One Sprite

157

180

Model Times Avg

milliseconds

milliseconds

#1 Array, One Sprite

123

126.972

#2 Array, Multiple Sprites

x

x

#3 Scrollrect, Multiple Sprites

x

x

#4 GAS, Multiple Sprites

x

x

#5 Sprite Sheet, One Sprite

126

129

Creation Time

milliseconds

milliseconds

#1 Array, One Sprite

402

3743

#2 Array, Multiple Sprites

x

x

#3 Scrollrect, Multiple Sprites

x

x

#4 GAS, Multiple Sprites

x

x

#5 Sprite Sheet, One Sprite

426

3695

Memory Usage

Kilobytes

Kilobytes

#1 Array, One Sprite

17244

21594

#2 Array, Multiple Sprites

x

x

#3 Scrollrect, Multiple Sprites

x

x

#4 GAS, Multiple Sprites

x

x

#5 Sprite Sheet, One Sprite

17211

23592

So, at 15000 objects, the winner is #1, the current engine I am using for my game, by a slight margin. The sprite sheets are starting to consume more memory than the arrays. #1 is also a little faster and takes less time to complete object update executions. 8 Frames per second may not sound like much, but the GAS, individual sprite, and scroll rect versions would be in negative FPS and result in full system crashes if I even attempted them.

Summary
If you didn't feel like reading all of the above (and who could blame you), here is a summary of the significant findings (at least to me).
The render Models will be referred to by number:

5000 Onscreen Game Objects Moving and Animating (#3 and #4 dropped because they were to slow to even run the test)

Fastest Average Frame Rate

23: #1 Method Model

Slowest Average Frame Rate

17: #2 Both Update Models.

Fastest Update Model

26 Milliseconds: #2 Event Model.

Slowest Update Model

44.18 Milliseconds: #5 Event Model

Fastest Object Creation Time

139 Milliseconds: #5 Method Model

Slowest Object Creation Time

1322 Milliseconds: #2 Event Model

Least memory used

8806 Kilobytes. #5 Method Model (#1 Method was very close)

Most memory used

14573 Kilobytes, #2 Event Model

10000 Onscreen Game Objects Moving and Animating ( #3 and #4 dropped because they were to slow to even run the test)

Fastest Average Frame Rate

12: #1 & #5 Method Model

Slowest Average Frame Rate

8: #2 Both Update Models.

Fastest Update Model

59.82 Milliseconds: #2 Method Model.

Slowest Update Model

129 Milliseconds: #5 Event Model

Fastest Object Creation Time

273 Milliseconds: #1 Method Model

Slowest Object Creation Time

7279 Milliseconds: #2 Event Model

Least memory used

13004 Kilobytes. #5 Method Model (#1 Method was very close again)

Most memory used

23298 Kilobytes, #2 Event Model

15000 Onscreen Game Objects Moving and Animating ( #2, #3 and #4 dropped because they were to slow to even run the test)

Fastest Average Frame Rate

8: #1 & #5 Method Model

Slowest Average Frame Rate

7: #2 Both #1 and #5 Event Models

Fastest Update Model

123 Milliseconds: #1 Method Model.

Slowest Update Model

87.435 Milliseconds: #5 Event Model

Fastest Object Creation Time

402 Milliseconds: #1 Method Model

Slowest Object Creation Time

3743 Milliseconds: #1 Event Model

Least memory used

17211 Kilobytes. #5 Method Model (#1 Method was very close again)

Most memory used

23592 Kilobytes, #5 Event Model

Ok, wow did that take a long time to write, run, and compile into useful (somewhat information). The whole process has taken me over a week to complete, but it was well worth it. I now have enough information to decide which model to use for which kind of application. I like the ease of Event driven update models, but as you can see above they are slightly slower than a method call model. It also looks to me like both the Array Copy Pixels and the sprite sheet Rect copy pixels (#1 and #5 respectively) are my best be for over 1000 objects. I will need to keep an eye out for their render update execution time if I am going to have a lot of other game logic going on.

Anyway, there you have it. You can download the .swf file here to run your own tests if you like. I don't advise putting it in the browser if you are looking for ultimate numbers, but then again, it might be useful for testing the qualities of the different plug-in versions and browser variations...now I'll save that for a post Christmas article.

On that note, below is a version you can play with. I don't advise using over 1000 objects with the GAS or Scrollrect render models though.

I hope this helps you in some way, it sure was a learning experience for me. I am sure you will come across better, more optimized models and engines. If you find one that can do 15,000 objects are higher than 8FPS on a Dual Core Pentium D (3.2 GHZ / 3.2 GHZ) with Radion 7300 card (not the best card, I know) then shoot me an email. I'd love to test it out myself.

I've written a few articles here on 8bitrocket.com filled with histrionics about the 'best' and 'greatest' Christmas mornings from my childhood, but until now I have avoided the talking about the some of the worst. The Coleco Shooting Gallery, Atari Jaguar, Action Max stand out as some of the worst Christmas presents I've ever given or received.

1980: Coleco Rifle Shooting Gallery

OK, the first story here is about Christmas 1980, and the Coleco Shooting Gallery (I'm not sure what the actual name of this piece of crap was, but this is close enough).This was not technically, a video game, but more like a shooting gallery coin-op from 70's.The toy consisted of a realistic looking rifle connected by a 15' (or so) wire to a set of plastic 'cans' on rack.The tops of the cans would open and close at random.By aiming the rifle at an open can, and shooting at the right time, you would knock it over.The toy advertised itself as a 'light gun', just like the arcade.Jeff and I saw this in the Sears Wish Book, and wanted it badly.It was the closest thing to a video gamethat we thought our parents would ever buy us.Our dad was a gun nut, and loved to take us to the shooting gallery at Disneyland, Knott's Berry Farm, and the Old Towne Mall, so this seemed like a perfect choice.We were right, he bought it for us and for the first couple hours of Christmas, it seemed like a fun toy.We took turns shooting cans, and pretending that we were in an actual arcade.The light gun seemed to work really well.The aim seemed very precise.If you shot an open can, it would fall down, no matter how close of far you were from the target

However, after Christmas dinner, I turned the game on and, by accident shot the light gun away from the cans.Curiously, one of the cans fell over as if I had aimed directly at it.I tried this again, and another can fell over.I got my dad and brother, and showed them.Since there was a light that emitted from the gun when you pulled the trigger, we all guessed that this phenomenon happened because the light was 'bouncing' back and hitting the cans (we had no idea how an actual light-gun worked at the time).After much experimenting though, we discovered that this was not the case at all.In reality, the gun did not care where you aimed, the light did not matter, nor did anything else that has do with aiming or skill.The only thing mattered was timing.If you pulled the trigger on the gun at the time that ANY of the cans were open, it would fall over.We were crushed.The Coleco Shooting Gallery was nothing more than a simple timer and switch, and it was nothing like any arcade game we ever played.My dad returned it (to Toys R Us), and vowed never to buy anything with the name 'Coleco' ever again.It's one of the main reasons we ended-up with a Vectrex in the summer 1983 instead of Colecovision, but that's a story for another day.

1996: Atari Jaguar

In 1996 Atari Corp was officially gone for good, and Atari Jaguar systems and games, if they were still on store shelves, were drastically reduced in price.For Christmas that year I decided to but my brother Jeff a Jaguar and as many games as I could find.I had never played a Jaguar prior, but there was suspicion in my mind that Atari's video game system never got a fair shake with reviewers or the public.At closeout prices, it would be a great time to see what we had missed.The Jaguar system itself cost $50, and the games were $5-$10 each.While I wanted to find the game Tempest 2000, it proved elusive, but I did mange to score Checkered Flag, Super Burnout and Zool 2 plus and extra controller.

Jeff was very happy to get his Jaguar.We set it up at our parents house on Christmas afternoon, (by then we had both moved out, but still exchanged gifts at our parents house) and put in the first game Cybermorph.As the pack-in game, we did not expect much, but what we saw was still shockingly terrible.

The sound in the game (and in most Jaguar games) was simply not up to par with modern video games (like the PSX I bought in October 1996).Even though the game made use of 3D polygons and digitized voices, the actual FX and game play did even match similar games like Starglider released for the Atari ST computer in 1987. I don't think this game ever made it back into the cartridge slot of the Jaguar for the next 11 years.

The next game, Checked Flag was a 3D polygonal racing game.However, it too suffered from what fast becoming a theme with these Jaguar game:graphics and audio that looked and sounded like ass.

The other two games, Super Burnout and Zool 2 were 2D games that were no better than Sega Genesis games from the end of the 80's.They were fine games, but not especially interesting.In the end, the Jaguar, even for the $75 or so I paid, was a Christmas bust.I tried to make-it-up to Jeff for our Birthday in January by getting him Wolfenstein 3D and Tempest 2000, but it was too little too late.His wife had bought him a PSX. In the end, the final video game system to display the Atari name and logo, proved itself to be worth all the non-success and non-praise it actually saw when it was an going concern.

1988: Action Max

The final worst video game Christmas present stands out the loudest because it was something not only I bought for my brother, it was something that I convinced (yes convinced) my friend Ian to buy for his brothers as a gift Christmas too.In last days December 1988 leading up to Christmas, Ian and I went to the Kay Bee Toys in the Manhattan Village Mall to looks for cool stuff for Christmas.Kay Bee was always a great store for video games because they sold mostly closed-out, cheap cartridges.This time though, we saw a full video game system up on an upper-shelf for $20.It was named the Action Max from World's of Wonder Toys.

The system was composed of a light gun, a base unit, TV connection, and a red light. It included a game on video tape named Sonic Fury.As well as the system, they had several other 'games' for the system each costing $2.99.I showed it to Ian, and he was initially concerned about the video tapes that were being sold as games.It did look odd to me.I thought to myself, 'Does the system load any information from the tapes ?''How does the light-gun work?''What is the 'red light' used for?' Instead of asking these question aloud though,I told Ian about the Supercharger Jeff and I once had for the Atari VCS.The Supercharger was an add-on cartridge that loaded games from audio-tape.The Supercharger was <b>awesome</b>, and some of the best VCS games were made for it (i.e. Dragonstomper).I told him that if the Action Max was even close to the Supercharger, it would be really cool.Ian seemed convinced.To make a long story short, we both left the store with an Action Max system and am arm load of tapes including.38 Ambush Alley, Rescue AtPops Ghostly, and Hydrosub: 2021.

When Jeff opened his Action Max, he was initially excited about it.I told him about the video tapes, and that I thought worked like the Supercharger.After all the presents were opened, we unpacked the Action Max, anticipating some really cool video game action.The first sign of a problem was the base unit itself.While it looked large, it was very light.The second thing we noticed was that there only seemed to be an output to the TV, but no input from the VCR.This was troubling because it meant that no data was being loaded from the video tapes back to the unit.Still, we thought it could not be that bad.The system billed itself as the first 'Real Action Game System', so something cool had to be in-store for us.

The system worked like this:You put the video tape into your VCS, pressed play, reset the score counter on the base-unit, and started firing at things on the TV screen with the light gun.We put Pops Ghostly in First, and this is basically what we saw:

You fired the gun at white circles on the screen.If you hit one, the red light would light-up, and score counter on the base-unit would update.That was it.That was 100% of the functionality.Even though the ghosts, bad guys, etc 'shot back' at you, it did not matter.How could it?There was no real connection from the VCR to the base unit.The light gun simply 'read' the light on the screen and registered a 'hit'.Each 'game' (tape) played the same.They were roughly 15 minutes long, and they had to be some of the longest 15 minutes of my entire life.I believe Jeff made it all the way through Pops Ghostly, and ' way through Ambush Alley before he simply gave up, never to try the Action Max again.It was worse than the Coleco Shooting Gallery from 1980, because we could not return the system.It was bought on close-out, and all sales were final.When I saw Ian the next day, he wove the same woeful of Christmas Day disappointment from his own brothers.I apologized, but I don't think he ever forgave me.

The Action Max sat unused on-top of our family VCR for the next 6 years until Jeff and I moved-out of the house in 1994.I'm not sure why Jeff never removed it, but I know why I didn't.It was my own personal albatross of video game shame.It sat as a warning to not waste money on crappy, closed-out video game systems.Too bad its memory was long gone before I bought the Atari Jaguar two years later.

Just about every year, Steve and I coordinate a great from the gaming industry to visit our day job and give the team a day-long talk on game design. We like to focus on classic game designers because we firmly believe that their experience in creating an entire industry from scratch will benefit our game creation process. If it has been programmed, they have programmed it, if it has been created, they have created it, and if it was a mistake, they have learned greatly from it. We have had Chris Crawford and Bill Kunkle visit in the past. This year, we had Rob Fulop join us for a lively discussion on games and online communities.

If you don't know Rob's name, I'll fill you in a little on his experience. He was a game programmer and designer at Atari where he started by programming arcade conversion: Night Driver (Atari 2600), Missile Command (Atari 2600), Space Invaders (Atari 800). He went on to Imagic where he created arguably the best 2600 game of all time, Demon Attack. Wanting to start creating games that didn't only rely on shooting, he created Cosmic Arc, Fathom, the unreleased (but soon to be in limited release) Actionauts, and Cubicolor (another recent limited release from the Imagic Days). Actionauts did see the light of day as a C64 release. When AOL was first starting in the mid 80's, he created the worlds first online gambling game, Rabitjack's Casino. After creating some titles for the ill fated CDi, he created both Sewer Shark and the controversial Night Trap for the Sega CD. The furor over that game caused him to create the nicest, most enjoyable game he could think of, Dogz. Coming out at about the same time as the Japanese toys with the same premise, he virtually invented the computer version of a virtual pet. His company, PF Magic (Pure F*cking Magic) was sold off to The Learning Company in the 90's. He currently focuses on online titles, social interaction games and consulting.

Rob is full of stories from the classic era. One of the best is how he got his job at Atari. Be was in a backgammon tournament, and his opponent happened to be Nolan Bushnell. He really wanted a job at Atari, so he proceeded to try to let Nolan win. Backgammon is not a game that lends itself to trying to lose, so Nolan was amused and befuddled by what appeared to be his opponents attempt to lose the game on purpose. Sensing that he was choosing the wrong path to employment, Rob went on to win the game, and convinced Nolan to hire him anyway.

On the title of his company, he said : We called it Pure F*cking Magic because that was our response when people ask us how our technical wizardry was accomplished! It was also the only comment Rob would leave in the code of his Atari 2600 games.

I don't want to give away's Rob's farm, as he usually gets paid for the information I am about to give you, but with his permission I would like to give you a taste of his game design ideas. My hope is that you will be tantalized enough to invite him out to your game shop and have a fun day of discussions with this game industry great. These are not verbatim, but my interpretation of some of his ideas.

Rob Fulop's Game Design Essentials, as Paraphrased, summarized, and added to by Jeff from 8bitrocket.com

1. All games need a premise of some sort. Without one, there would be no reason for the game to exist and it wouldn't be much fun to play.Common premises are: Treasure Hunt, Find Secrets (or keep secrets), go fast, clean up, chase with the knowledge that you can turn the tables, fight (any kind of head to head competition where one much fall), etc. Without a premise, you really just have an activity or a puzzle. Not that activities or puzzles are bad choices for online entertainment (Sudoku, and Barbie Dress up are two great examples).

Pacman is fun not simply because you are being chased, but because you can turn the tables and chase back, Most 'extra' weapon space shooter games are fun because of this simple idea: The bad dudes may have the edge at the beginning, but you can attain enough power to get them back at some point.

The premise of racing games is simply to go really fast. Games can have more than one premise, so a game like Burnout 3 has more than one: Go Fast, and break things.

Mario games don't simply have exploration as a premise, but also they are a supreme treasure hunt. You collect coins, find secrets, and rescue Princess Peach. It was highly successful because it combined a number of basic premises into one awesomely complete game and it revitalized and entire industry to boot.

I think it might sound obvious to most that a game needs a premise, so the concept is not overly revolutionary, but the message is this: Sit down and think about the premise (or multiple) before you do any other game design. Without a solid premise, you probably will not have a good game.

2. Design RiskBy choosing a tried and true concept, you limit design risk. The more unique and untried concepts you put in a game, the more design risk you incur. You tend to be a marketing person's dream if you simply copy what was successful before (and who wants to be that?). The key is the balance between tried and true design decisions and new concepts. A space shooter, for example, was done 1000 time before R-Type was created. R-Type, though, took a tried and true concept (Space Invaders), turned it on its side, scrolled it, and added in massive bolt-on weaponry the likes of which had rarely been seen in a game of this type.It is very very difficult to think of a brand new game type - Most puzzle games of today are Tetris games, most shooters are Doom clones, most RPG's are Dungeon Master, and most RTS games have Dune II and Command and Conquer to thank for their success. Who would have thought Brain Age, Guitar Hero, or Cooking Momma would have been as great as they are. I am sure the first guy/gal that pitched those games was laughed at by a marketing genius who wanted another FPS or RTS. So, The second key to making a good game is to decide on the level of design risk you are willing to take before you start creating the game.

3. Game CharactersOne of the most important parts of a game is the main character. Rob cited the latest version of Linerider, where the rider has been programmed to react to his line riding actions with verbal and physical cues, as a great example of giving in-game characters the ability to create a relationship with the player. The response that virtual characters give to user about his/her environment (even breaking the wall between game and user) makes the character more real to the user. When you begin to design a game character, you should make sure that the character has some sort of identifying characteristic(s) - like the 7 dwarves, or characters on Gilligan's island: Each had one defining characteristic. Prominent in-game characters (player and NPCs) should have at least on define characteristic. Reactions to the environment should be believable and based on this/these characteristic(s).

Note: The above is not straight from Rob's mouth, but my interpretation of what he said. Consider them my design-risk adverse version of his game design ideas I don't want to give Rob's entire discussion away, so if you are interested in what he had to say, even if you disagree (especially if you do), I encourage you to visit his site, email him, maybe even invite him out to discuss game design with your team. It will be well worth your time.

I recommend visiting his site http://www.robfulop.com to check out more details on his games and his consulting services. His is a wealth of knowledge on game design, and all current and future game designers can learn a lot from his experience.