i'm not sure if the number of places has to do with code design, depth of simulation or a little of both. most features have been implemented with their own iterator loops for processing PCs. i suppose one big loop that handles many features at once would reduce the number of loops, and therefore the number of checks for (! alive)....

but does 99 sound like a lot of things to be updating for each player entity? note that some of those are checks, not updates. also, some things like a list of their traps and the location of their house are stored in the PC's data, which accounts for some of the loops and checks.

Yikes. It looks like you are building yourself a dependency nightmare there. Any time you have an internal member of a class or structure referenced in 99 places throughout your code, you should hear an alarm bell ringing in your head.

Why does the outside loop even care about if an object is alive, dead, active, etc? What happens if you add paralysis, sleep, hypnosis, and so on? Are you going to modify those 99 places yet again? Why can't the loop just iterate the objects and pass them a message to DoSomething, and let the objects themselves decide whether or not to actually DoSomething, based on their own status? That way, the enclosing loop doesn't need to know the internals of the objects, which you are free to modify to your heart's content without breaking 99 different locations.

You might want to start looking into alternative means for structuring your game. For example, you might want to look into object composition as a means for reducing dependencies. Functionality is broken out into components which are owned by objects, and which handle their own narrow, domain-specific tasks. For example, some sort of Renderable component, which is responsible for managing the renderable (sprite, mesh, whatever) that gets drawn in the world. The Render system iterates all Renderables and draws them according to their internal state, and that state is managed by other components. There are many ways you can structure such a system, and it can get complex, but if you do it correctly you can drastically reduce the occurrence of redundant, duplicate code and unnecessary connections between objects.

as i said, some i believe is organizational, with regard to using one loop per update type instead of one loop per update frequency (IE one loop for all stuff that updates every frame, one for all stuff that updates once per game second, one for stuff that updates once per game every minute, etc). or using a single loop that updates every frame, then calls the less frequent updates using conditional control flow.

but even with one loop, and separating the list of traps and the "where's my house" info from the player info, i'm still looking at a PC entity type with something like 90 update and check methods!

whether its OO and all in one place or not, it still seems like a lot. i mean, know its an in-depth sim, but ~90 types of updates and checks?

OTOH, i wrote down the name of each of the 99 methods, while counting them, just in case i had to post some examples <g>. while looking over the names, each seemed to be something that an entity must have - at least for this game.

different updates occur at different rates, once per frame, second, minute, hour, or day, and also at other intervals like 5 minutes, 10 minutes, 15 minutes, etc. all the game's update() routine does is increment the clock and then call the appropriate update routines. since it appears that it models or checks ~99 things in relation to a PC, it would seem that i can reduce the number of loops, but not the number of methods.

consolidating the many small loops into fewer larger loops would make changes like this easier. might be less cache friendly though.

and none of the check loops could be consolidated. in OO terms, they are basically methods of the "list of PCs" object. so they have to iterate through the list of PCs to do their checks, stuff like "who's the closest player to player X's house", and stuff like that.

here's some of the types of updates (in the order they appear in the code):

WARNING! LONG LIST!

nuke_mood_friends_of(A). severely reduces mood (a la the sims) of all players who are friends with NPC "A" (who just died).

run_bandmember. update for a PC under AI control. the game is single player, but the player can control multiple PCs, like a household in The Sims. the player can switch between PCs at any time by hitting TAB. PCs not being controlled by the player are controlled by AI. this is the update routine for when they are under AI control.

nuke_band_mood. severely reduces the mood of all PCs (called when a PC dies).

kill_bandmember. hey, every object needs a dispose, right?

try_move_rafts. part of this routine adds a raft's movement to a PCs movement (you can walk around on a raft in motion!) it also calls automap for each PC onboard.

model band member surprise. surprise is modeled in encounters. surprise gives you a few seconds to move before your opponent can, or causes you to be unable to move for a few seconds while your opponent can. this updates those counters.

resolve animal attack. a section of this routine checks for a PC in the line of fire of an animal's attack, and if found, checks for hit, applies damage, and kills the player if needed. so this the equivalent of where the game's "damage PC entity" method gets called.

check for 100% fatigue. checks for fatigue hitting 100%. when fatigue hits 100%, if the PC is doing an action (swimming, moving cross country, fixing a bow, gathering berries, etc), the action is pushed on the PC action stack. a "rest" action is then started. the rest action is designed to interrupt other actions, and therefore automatically pops the action stack if needed on completion.

check rock shelter encounters. checks for player near rock shelter. generates encounter if needed. removes encountered nps/animals, and resets "encounter checked" flag when no PCs nearby any longer. the game has 5000 rock shelters based on real world frequency distribution and the size of the game world.

check cave encounters - same idea, for caves. the game has 60,000 caves based on real world frequency distribution and the size of the game world.

check hut encounters - same idea, for huts. the game has 18,000 huts based on estimated population densities, the percentage of the population staying in one place at any given time, and the size of the game world.

check hut takeovers - checks for abandoned huts near the PCs. gives them the option to take them over. huts are typically "abandoned" because the player has killed or chased off the occupants.

model climbing animals - one of the actions in the game is "climb tree". if you get an encounter with a wild animal, you can climb a tree and wait for them to wander off. but some animals can climb up trees after you. this checks each PC to see of they're up a tree and in an encounter with an animal that can climb trees. if so, it forces them to jump to the ground to get away from the animal, at which point combat resumes as usual, although the player can always climb a tree again, not that it will do much good.

model fatigue due to damage - most shooters don't model this. but you DO get tired faster when wounded. this might be an example of one of the update methods required due to the depth of the simulation.

model fatigue due to encumbrance - another real world effect most shooters don't model.

and thats it for the update type stuff. a few things in there may not technically be PC updates. it looks like a lot of the methods are checks such as "band member in way of band member", "band member near dropped object", "someone paddling" - returns true if a PC is aboard a raft and is doing the "paddle" action, and so on.

there are also render calls for drawing a PC in the world, on the world map, and the local map.

the AI uses six different type of target selection routines that iterate though the list of PCs looking for potential targets. I'm surprised it isn't more, given that there's something like 14 types of AI in the game.

I think if its the same tests in all 99 loops you could sort all those entitys once into active and inactive. Then you can delete 98 of those 99 "if", loop only until maxactivecavemen instead of maxcaveman and it gets even more cachefriendly.

when making a change to one piece of functionality effects a lot of other things, this is generally a bad sign.

usually, it makes sense to try to organize things when possible, that making changes does not have widespread effects.

usual strategies are what I call "modularizing", "layering", and "pipelines".

modularizing is basically breaking parts into independent "black boxes", where code in one "module" ideally minimize dependencies and interactions with code within another module.

layering is basically building an API such that a range of common functionality can essentially be "abstracted out" of the code.

ideally, code building on top of the API needs to not know or care how things work inside the API, nor should the API code care what happens in client code or how it is used.

pipelines are basically where the code is broken up into a number of stages, typically with each stage transforming input into output in some well-defined way. typically either, each stage is stateless, or the existence of state is handled explicitly in the design.

in this case, a piece of code to achieve a task then becomes a number of interconnected stages, with each operating independently of the others.

typically, all this is then applied at various levels. this allows generally getting more things done with less effort, and basically more freedom with going mix-and-match with various parts of the project while minimizing need for large-scale changes.

for game logic, this would generally mean trying to abstract the logic from the entities from the handling of entities themselves.

for example, does the entity-system logic even need to know or care whether the entity is alive or dead? generally not, and instead being alive or dead would be a property of certain types of entities (such as AI controlled mobs). so, the entity system will primarily concern itself with things which are fairly universal.

generally, we also want to isolate any loops or iteration over entities from the logic for the entities themselves. this would mean that general update loops will primarily consist of calling methods in the specific entity objects (this is independent of language choice), which may itself implement the relevant behavior logic.

but, yeah, if things are organized well, than more types of similar work can be done more quickly and with less effort, even if potentially at some cost in terms of additional up-front effort.

though, yes, there is also a problem that too much up-front generalization can also be a problem, resulting in a lot of work for not a whole lot of gain (or a project that is 90% infrastructure and 10% program logic), or use of inappropriate abstractions which don't really fit the use-case, ..., so striking a balance is needed.

Having had this sort of problem in the past (more common with my SOA organized data when I had to do AOS access style in specific cases), I generally used a helper+filtered iterator approach to simplify things, though it does not completely remove the 99 cases, it should at least reduce the code overlaps:

This does not attempt to correct code structure though I tend to agree with others that you might consider this is a problem, but that's completely up to you. What it does do though is allow you to break up the functionality such that in the future at least the key pieces of the code beyond the localized "update call" are nicely organized somewhere central.

NOTE: Obviously I'm using C++11 in the above. You can of course replace it all with TR1, Boost or just plane old functors. Of course each of the other solutions has it's own downfalls, generally in terms of typing.

I think if its the same tests in all 99 loops you could sort all those entitys once into active and inactive. Then you can delete 98 of those 99 "if", loop only until maxactivecavemen instead of maxcaveman and it gets even more cachefriendly.

actually, the trick there is to the old "Swap with last and decrement count" trick.

IE if num_PCs = 5, and you get a kill_PC(2) call, you do:

pc[2]=pc[4];

num_PCs--;

that way there's no sorting, and you only iterate over 0 through num_PCs-1. this only works cause the list is unsorted.

but for the moment, i haven't had to get into optimizing update() too much. like all good simulators, the game supports accelerated time: 2x, 20x,and (up to) 900x. but 900x currently only runs at about 30x on a baseline PC. so some optimization is probably in my future.

all the updates don't happen every frame. some happen every frame, some every second, some every 5 seconds, some every minute, some every 10 minutes, some every 15 minutes, and so on. this lead to the following type of code for update():

well that will nicely encapsulate the "filerting test" in a generic manner. however, "is_active && is_alive" is the only filter used, except in the render call where its just "is _active".

99 little for loops is probably inefficient, and is the root cause of so many changes being required.

but its fast enough for the moment, so maintainability and modifiability are the only reasons to refactor at this point.

OTOH, it will probably need to go faster before its all over. so i think i need to take a little time out, read up on code cache behavior, then decide how to refactor the loop layout (if at all), and THEN do the changes for "is_alive".

worst case, i know 99 little loops is the most cache friendly, and i just add "if (! cm[a].alive) continue;" everywhere.

best case, one big loop or "whatever is easiest to code and maintain" is sufficiently cache friendly, i refactor, then make my "is_alive" change in just three places (update, init, and render).

well that will nicely encapsulate the "filerting test" in a generic manner. however, "is_active && is_alive" is the only filter used, except in the render call where its just "is _active".

Yeah, I figured with all the other comments that maybe something more directed at the specific case would be useful. But I personally think it has more utility than simply removing the filtering issue. By abstracting the loop itself in addition to the filter, other ways of organizing the data becomes possible without rewriting it in 99 different locations. For instance, if you separated the lists as someone mentioned, then you would only modify the new abstract list iterator to handle that case. At that point the filter abstraction becomes nothing more than a "tag" which identifies which sublists to include. Or, as you go into below:

99 little for loops is probably inefficient, and is the root cause of so many changes being required.

but its fast enough for the moment, so maintainability and modifiability are the only reasons to refactor at this point.

That is always the key, if it is fast enough, don't mess with it. Though, with the abstraction you are protecting yourself from hunting down the 99 cases again in the future. Change them all in one place. Even if you decide to simply push the changes to be performed all at once later, you can modify that function to simply store the lambda for later use. The abstraction is a bit more flexible than you might think initially.

OTOH, it will probably need to go faster before its all over. so i think i need to take a little time out, read up on code cache behavior, then decide how to refactor the loop layout (if at all), and THEN do the changes for "is_alive".

worst case, i know 99 little loops is the most cache friendly, and i just add "if (! cm[a].alive) continue;" everywhere.

best case, one big loop or "whatever is easiest to code and maintain" is sufficiently cache friendly, i refactor, then make my "is_alive" change in just three places (update, init, and render).

Cache behavior is going to depend on a very large number of things, saying it is faster with the small loops is not a guarantee, especially if there are many things going on between the different loops. I.e. run a loop, do stuff that starts evicting objects from cache, run another loop, do other stuff that starts evicting object data repeat. On the other hand, the small loops are generally easier to later merge as possible and even apply multi-core distribution to. Either way, unless you have really sat down with performance counters and measured cache miss rates and all that, I almost always find my initial assumptions to be way off the mark.

Just a little note as a general encouragement. On a certain game which was mentioned originally in your message there was a 38,000 line file which consisted of a single function. Every variation of if/else/for/while/switch/case crap you've ever seen was packed into this massive monolithic function. It was the most disgusting piece of code I ever have seen, it shipped a game and many expansion packs without ever being cleaned up, just further 'hacked'... I'd prefer to have 99 places in the code to clean up than a single monolithic interdependent disaster area function.

got a boatload of good links. i'll post them in my game development journal. it'll be a nice companion to my journal entry on data cache info.

long and short of it, the instruction cache situation is similar to that of data caches.

and let us hope that none of us ever has to go there when it comes to optimization! <g>.

it looks like in my case, the best approach will be to combine loops as much as possible. this should improved modify-ability.

then - if and when the time comes, i can optimize. but odds are that i won't be forced to go to an " architecture (drives->) cache design (drives->) data structure design (drives->) code layout " type of optimization - which, of course, is the the ultimate way to write high performance code. but it also doesn't really consider any other coding issues like maintainability and such.

i decided this is probably the way to go based on what i learned, and on the issues mentioned by AllEightUp, which were also mentioned in the info i found online. IE SERIOUS cache optimization is not easy, although writing cache friendlier code is not so hard.

ApochPiQ:

I DL'd the pdf, but haven't had a chance to watch the whole video yet. i take it the general idea (you know how slides are) of the first part of the presentation is to create a generic iterator (to replace raw loops) that can (in my case) be passed the various update functions.

So i've started coding up a new update_all() routine for band members.

at first, no problem.

i implemented "the swap with last and decrement count" for deactivating a band member. so now i can just iterate from zero through num_bandmembers - 1, and don't need to check for " .active=1 ". but i do need a check for " .alive=1 " now.

then i started adding the various update calls:

(in CScript code...)

' band member update all

fn v BMupdate_all

i a

4 a num_bandmembers

== cm[a].alive 0

>>

.

' do update stuff here

c BMdecrease_fatigue a

c BMmodel_attack a

c BMmodel_surprise a

c BMmodel_full_fatigue a

c run_bandmember a

c BMmove_raft a

.

.

so now i'm at move raft - and i've hit a snag - or a complication to be more precise....

i'm doing an update_all_rafts() type thing. and in update_raft(a), i have to do a call to update_bandmember_aboard(b,dx,dz) where b is the bandmember number, and dx,dz is the raft movement. so it looks like that loop cant be consolidated (easily). i'd have to update all rafts first, and save their movement deltas, then update band members, and use the saved deltas in a routine such as BMmodel_raft_movement(a).

so now the question is:

1. should the raft movement "event" get processed immediately - requiring another iterator?

-or-

2. should the raft movement "event" be stored for later "batch processing" with the rest of the band member updates - which keeps things more in one place - but requires storing deltas?

when looking at update_raft, storing deltas for later processing doesn't make it obvious what the downstream effects of handling the event are.

when looking at BMupdate_all, processing raft movement immediately leaves you no clue that there's additional updates going on outside of BMupdate_all.

Just a little note as a general encouragement. On a certain game which was mentioned originally in your message there was a 38,000 line file which consisted of a single function. Every variation of if/else/for/while/switch/case crap you've ever seen was packed into this massive monolithic function. It was the most disgusting piece of code I ever have seen, it shipped a game and many expansion packs without ever being cleaned up, just further 'hacked'... I'd prefer to have 99 places in the code to clean up than a single monolithic interdependent disaster area function.

last night while i was collapsing loops together, i became curious, what did the function do?

Just a little note as a general encouragement. On a certain game which was mentioned originally in your message there was a 38,000 line file which consisted of a single function. Every variation of if/else/for/while/switch/case crap you've ever seen was packed into this massive monolithic function. It was the most disgusting piece of code I ever have seen, it shipped a game and many expansion packs without ever being cleaned up, just further 'hacked'... I'd prefer to have 99 places in the code to clean up than a single monolithic interdependent disaster area function.

last night while i was collapsing loops together, i became curious, what did the function do?

It was basically just an update function, of course it did everything from pathing to animation blending and probably 50% of it was dead code that could never be reached. Of course, going through the first thousand or so lines you are generally so lost in possible branches that there was simply no way to clean it up without a top down refactor which no one was willing to allow.

ok, new update_all is coded. but not hooked up yet. one loop to rule them all.

here's what the new code looks like (CScript code). i've added comments in red to make it easier to follow. i'll post the C++ translation output as well.

( since embedded code seems to clip messages of late, i'll just use font and color )

band member update all:

' must call move_rafts first! // i decided to store raft dx,dz and process them here with the rest of the updates
// so move_rafts must be called first to set raft dx,dz
fn v BMupdate_all
i a // int a loop counter the band member number
4 a num_bandmembers // it now only iterates over active band members, not all band members (active or inactive).
== cm[a].alive 0 // here's the check for "is_alive". if alive==0 continue.
>>
.
' do update stuff here - do global frame
c BMdecrease_fatigue a // call BMdecrease_fatigue with "a" as a parameter.
c BMmodel_attack a
c BMmodel_surprise a
c BMmodel_full_fatigue a
c run_bandmember a
c BMmodel_raft_movement a
c BMmodel_paddling_fatigue a
c BMmodel_falling a
c BMmodel_sneak_detection a
!= frame 0 // if frame != 0
>> // continue
. // end if code block
' do global second
c BMmodel_fatigue_dueto_damage a
c BMmodel_fatigue_dueto_encumbrance a
c BMrun_quests a
c BMclear_rockshelter_enccouter_flags a
c BMcheck_rockshelter_encounters a
c BMclear_cave_encounter_flags a
c BMcheck_cave_encounters a
c BMclear_hut_encouter_flags a
c BMcheck_hut_encounters a
c BMcheck_hut_takeover a
c BMcheck_climbing_animals a
!= second 0
>>
.
' do global minute
c BMcheck_animal_encounters a
c BMcheck_caveman_encounters a
c BMmodel_intox a
c BMextinguish_torches a
== minute%7 0 // if (minute % 7) == 0
c BMreduce_hygiene_dueto_movement a
.
== minute%10 0
c BMreduce_water a
c BMreduce_sleep a
.
== minute%15 0
c BMreduce_food a
c BMaffect_mood a
.
!= minute 0
>>
.
' do global hour
c BMmodel_dehydration a
c BMmodel_food_spoilage a
c BMmodel_exposure a
c BMmodel_heatstroke a
c BMmodel_drown_in_flood a
c BMmodel_wearNtear a
c BMmodel_perm_shelter_raids a
c BMmoodboost_nature_lover a
== hour%2 0
c BMmodel_damage_dueto_illness a
.
? ((hour>7)&&(hour<19)) // if hour > 7 and hour < 19
c BMcheck_quest_encounters a
.
!= hour 0
>>
.
' do global day
c BMmodel_starvation a
c BMmodel_getting_sick a
c BMmodel_background_radiation a
c BMmodel_perm_shel_weathering a
c BMzero_god_relations a
c BMreduce_social a
c BMmodel_traps a
c BMmodel_skill_reduction a
. // end for loop
. // end function definition

i'm thinking this gives me an opportunity to run side by side timing tests of the two loop layouts. to see what the difference is. results might be interesting.

so the next step is to make a version of run_simulation (whose code appears in a previous in this htread) that does everything except the BMupdate_all stuff.

then hook the two versions of run_simulation up to a hotkey, add timers, and stir! <g>.

not sure how i'd call it. the one big loop eliminates a lot of iterator overhead. but the loop bodies of the 99 little loops tend to be small and instruction cache friendly.

see what happens.....

i also implemented a "find first / find next" system for both active and alive band members, as a low overhead way to de-couple the iterator from the code being iterated. that too has yet to be hooked up. i should probably go back and see if some standard C++ data structure or template can do the same thing with no type check or other overhead.