My main concern is the amount of complexity that would be needed to make this work for all ships and situations. Especially battle situations where some thruster would be destroyed.

My vote is to push all that complexity into the PID controller, and then make it as efficient as possible.
Although, currently I don't know the differences in performance and if there's a large enough drop just from PID controllers, then I agree that there needs to be a middle ground between using them and having faster custom controllers.

PID's are very few lines of code per axis, a lot less calc than the rest of the physics for any ship.

And what complexity?
if theres realistic thruster influences on motion you need the axis <-> thruster matrix solver anyway.
What you put on top of it doesnt exactly matter for its complexity.

I thought that's what PID controllers are for? To remove all the code needed for thrusters and have the PID output the thrust and direction needed

An idea that is not dangerous is unworthy of being called an idea at all. - Oscar Wilde

We often refuse to accept an idea merely because the tone of voice in which it has been expressed is unsympathetic to us. - Friedrich Nietzsche

I thought that's what PID controllers are for? To remove all the code needed for thrusters and have the PID output the thrust and direction needed

Yes, they are. But not directly.
All they do is "go more right" or "rotate more left"
What "translate right" and "rotate left" means for the thrusters inst the domain of the PID, thats the domain of some transformation matrix that knows which thruster provides how much effect on each axis.

The PID provides the commands of how much action is required on each axis.
The transformation matrix translates the directional commands to individual thruster settings.

the PID sits in the same place as the player does in KSP for example.
"give me x thrust in direction y", which RCS nozzles then fire dont exactly matter for you or the PID

From what I can tell, PID is a way to modulate the power output based on a "goal point".
I'm under the assumption that Josh is using it to steer/move ships towards a "goal point".
But is he using a single PID controller per ship? Or is it one per thruster on a ship?

Do read at least the first few paragraphs of the PID article that Josh linked too. My guess is that the goal point can be a vector of a 3 dimensional location and a 3 dimensional velocity (I’m struggling to remember the precise term). With sufficient good undergrad level maths, this vector may be treated as a single value. And therefore, the PID algorithm can work on that vector as a single value. From my brief reading of the article, the PID is a very general and fairly optimal approach to control.

Any problem where the goal can be expressed as a value and the path to the goal can be described in a ‘continuous’ (no gaps) and ‘differentiable’ (no corners) mathematical function probably can be solved by the PID algorithm.

This is not about ai logic BFett, but about getting the ai to move its ship from point a to point b. Not about determining what a and b should be in the first place, which is what you think of.

Kinda surprised Josh hadn't heard about PID before though

^ Dino is correct here. This is the lowest-of-the-low level AI we're talking about, where the AI has already figured out what position and orientation it wants to achieve, and now has to figure out how to set thrust power accordingly (the equivalent of the player pressing WASD and mousing around to control direction / using a gamepad / joystick).

And yeah, I'm really surprised I hadn't heard of it too!! It's such a great nugget of knowledge. It seems like the usage/knowledge of it is weirdly domain-specific atm (even the article talks about it heavily in terms of real-life control processes + the math analysis is noticeably light). Can't believe this hasn't come up in my studies of math or programming as a more general concept (e.g. control of dynamical systems). Wot else am I missing out on???

Since the output is a single value (1 dimension), I guess the error is the lateral devition from the target point, and the controlled value is the steering angle to apply.
(trust left/right)
Then there is probably another PID for the forward/backward trust. (error is distance to target-point)
And one for up/down. (another angle error)

Those 3 PIDs control then forward, yaw and tilt trusters.

Wich makes it 9 Parameters to tune.

Yes, basically. I have six controllers running on each AI ship, corresponding to 6-DOF flight controls (right, up, forward, yaw, pitch, roll). Target thrust values are just computed from some simple vector math involving the current position/orientation and the target position/orientation. Currently I use the same constants for all params but scale linear thrust differently than angular. So effectively one set of params for linear and another for angular (makes sense IMO).

My main concern is the amount of complexity that would be needed to make this work for all ships and situations. Especially battle situations where some thruster would be destroyed.

My vote is to push all that complexity into the PID controller, and then make it as efficient as possible.
Although, currently I don't know the differences in performance and if there's a large enough drop just from PID controllers, then I agree that there needs to be a middle ground between using them and having faster custom controllers.

I can't say for sure yet because I haven't tested it, but part of my thinking was that this would be a boon for those situations as well, since PID will adjust accordingly if a thruster is destroyed. So I believe we are pushing the complexity into the PID; it just handles complexity quite nicely without a lot of code (kind of the point).

I can say that escorts currently adjust nicely when I shoot them with big asteroids, pushing them out of formation, or make them run into large obstacles (since they're not avoiding them yet) and have to re-form. Their maneuvering looks basically ideal to me -- very responsive but still smooth, and taking all factors into account automatically. For example, if I'm going very fast and knock a forward escort out of formation in my forward direction, he'll do the smart thing, re-orienting himself in my forward direction and slowly accelerating until he naturally falls back into formation due to me catching up, by which time he's matched my velocity. This is the fastest way he could possibly get back into formation in that situation. Compare to simple algorithm that would try to come back to the current formation position (hence orienting opposite my forward direction), only to overshoot and fall behind due to having to stop and rotate back to match my forward.

We'll see how it keeps up with more and more complex situations. I'm definitely all for pushing the complexity into the PID controller. I also really like the thought that we can tune constants differently based on AI personality/skill (less skilled pilots might have a low Kd / Kp ratio, for example, causing them to oscillate sometimes instead of nailing the damping just right.)

Do read at least the first few paragraphs of the PID article that Josh linked too. My guess is that the goal point can be a vector of a 3 dimensional location and a 3 dimensional velocity (I’m struggling to remember the precise term). With sufficient good undergrad level maths, this vector may be treated as a single value. And therefore, the PID algorithm can work on that vector as a single value. From my brief reading of the article, the PID is a very general and fairly optimal approach to control.

Righto; it's general enough that I think you could pretty easily do the vector-valued version of it, where the deriv would just be the Jacobian of the error function. I haven't done this yet, but would be interested to try it. I believe it would further increase stability especially for angular terms, which don't have the same degree of linear independence that the thrust vectors do (in fact, thinking about it, I don't think treating the linear thrust term as a vector error would make any difference due to linear thrust being LI). YPR, OTOH, are sometimes difficult for the current PID to get right due to their nonlinearity & interdependence. I suspect converting to vector form would see nice improvements.

So what performance benefits from the xxHash are we talking about here? it's faster, but what does that ultimately mean for the game experience? Does it mean higher AI counts? Quicker load times when travelling through a wormhole while the game generates the assets in the next system? Higher framerates with lots of objects on screen? Does it mean we can now have "real" debris fields as ship modules can blow up dynamically and now you have to worry about flying chunks of dead ships slamming into your own? (Especially with the new physics and ECS able to handle oodles of objects)

Hard to say since it's such a low-level primitive. It should basically have 'relatively small' yet far-reaching implications. I was already using Murmur3 in most places, so I had a decent hash function already. But, as a concrete example, I use hash lookups as part of glyph caching to accelerate font rendering. While the cost of the hash is quite small, I still noticed several hundred microseconds in frametime reduction when I converted that to xxHash. There's currently not a lot of text being drawn compared to what you could imagine in certain game situations. So in that particular place, xxHash might save a millisecond or two when the player has lots of UI on-screen. Might sound small, but 'a millisecond or two' is massive when you consider how small of a change that is + the fact that we are on a 16ms budget.

It's used in broadphase collision (the hashgrid -- go figure!) I implemented broadphase after I already had xx available, so I don't know what the speedup is like there. But I imagine that, in this place, xxHash might buy us the ability to have, say, 20-25% more collision-enabled objects in the scene. That's just a really rough estimate based on my knowledge that the hgrid is pretty hash-intensive. Could be way off.

If we use hgrids to implement frustum culling (which is 'very likely' to happen, I would say), that may again translate to more objects that can be put on-screen, although in this case I would put the gain at less than 10% since the object rendering will outweigh the hashing by far (and the cull grid would be coarser than the broadphase phys grid). Then again, hgrids are not a normal choice for frustum culling, so we may use something more conventional like octree. We'll just have to see.

If you want to know if you can convert a shape to a formation, ask yourself these questions:

Viable Formation Shape Questions:

Can you write an algorithm that, given access to a random number generator, produces a random point on the boundary of this shape?

Can you write a parametric function f : Z -> R3 (integers to 3D space) that maps integers to points on the boundary of this shape?

Given a point in 3D space, can you tell me, at least roughly, how close I am to the boundary of this shape?

Can you write a function that deforms some other simple shape (like a sphere, cube, torus, etc.) into this shape? In math terms: can you give me a homeomorphism between this shape and a simpler primitive shape?

If you can answer 'yes' to any of them, then your shape can be made into a formation. There are probably other mechanisms, those are just a few that I thought of off the top of my head and would be pretty easy to implement.
Note that I was really talking about 'surface' formations above (2-dimensional manifolds). Line/curve formations are a bit more restrictive; the easiest way to make them is to write the parametric function directly. OTOH, volume formations (e.g. a thick shell) are more-or-less trivial since if you can write a function that tells me if a point is inside the volume, then I can use rejection-sampling to build formation points.

TL;DR: fancy formations are pretty easy.

“Whether you think you can, or you think you can't--you're right.” ~ Henry Ford

Hard to say since it's such a low-level primitive. It should basically have 'relatively small' yet far-reaching implications. I was already using Murmur3 in most places, so I had a decent hash function already. But, as a concrete example, I use hash lookups as part of glyph caching to accelerate font rendering. While the cost of the hash is quite small, I still noticed several hundred microseconds in frametime reduction when I converted that to xxHash. There's currently not a lot of text being drawn compared to what you could imagine in certain game situations. So in that particular place, xxHash might save a millisecond or two when the player has lots of UI on-screen. Might sound small, but 'a millisecond or two' is massive when you consider how small of a change that is + the fact that we are on a 16ms budget.

It's used in broadphase collision (the hashgrid -- go figure!) I implemented broadphase after I already had xx available, so I don't know what the speedup is like there. But I imagine that, in this place, xxHash might buy us the ability to have, say, 20-25% more collision-enabled objects in the scene. That's just a really rough estimate based on my knowledge that the hgrid is pretty hash-intensive. Could be way off.

If we use hgrids to implement frustum culling (which is 'very likely' to happen, I would say), that may again translate to more objects that can be put on-screen, although in this case I would put the gain at less than 10% since the object rendering will outweigh the hashing by far (and the cull grid would be coarser than the broadphase phys grid). Then again, hgrids are not a normal choice for frustum culling, so we may use something more conventional like octree. We'll just have to see.

As you can tell this is a tough question for me to answer But we should see 'nontrivial' perf gains in lots of different areas. Hard to quantify that all the way down to game experience impact.

The metaphor which helped me understand it best was that you upgraded from a hand whisk to an electric mixer. It will speed up the time it takes to bake your cakes and pastries, but how much it helps really depends on how many cakes you're baking and how fancy those pastries are. Perf gains are perf gains, more stuff for the same cost or same stuff for less cost with the savings going elsewhere

If you want to know if you can convert a shape to a formation, ask yourself these questions:

Viable Formation Shape Questions:

Can you write an algorithm that, given access to a random number generator, produces a random point on the boundary of this shape?

Can you write a parametric function f : Z -> R3 (integers to 3D space) that maps integers to points on the boundary of this shape?

Given a point in 3D space, can you tell me, at least roughly, how close I am to the boundary of this shape?

Can you write a function that deforms some other simple shape (like a sphere, cube, torus, etc.) into this shape? In math terms: can you give me a homeomorphism between this shape and a simpler primitive shape?

If you can answer 'yes' to any of them, then your shape can be made into a formation. There are probably other mechanisms, those are just a few that I thought of off the top of my head and would be pretty easy to implement.
Note that I was really talking about 'surface' formations above (2-dimensional manifolds). Line/curve formations are a bit more restrictive; the easiest way to make them is to write the parametric function directly. OTOH, volume formations (e.g. a thick shell) are more-or-less trivial since if you can write a function that tells me if a point is inside the volume, then I can use rejection-sampling to build formation points.

TL;DR: fancy formations are pretty easy.

Fantastic! I have numerous ideas spawning for the application of more interesting formations. Matrioshka Spacer cities, super weapons, containment fields, giant works of celestial scale art...

Challenging your assumptions is good for your health, good for your business, and good for your future. Stay skeptical but never undervalue the importance of a new and unfamiliar perspective.Imagination Fertilizer
Beauty may not save the world, but it's the only thing that can

less skilled pilots might have a low Kd / Kp ratio, for example, causing them to oscillate sometimes instead of nailing the damping just right.

I have seen one of such pilots driving on a snowy road. It was slippery, and when his car started loosing traction on one side, he was totally oversteering, and oscillating into the ditch. (just minor damage)
But I dont think telling him "your Kd / Kp ratio is way too low" would have helped ...

less skilled pilots might have a low Kd / Kp ratio, for example, causing them to oscillate sometimes instead of nailing the damping just right.

I have seen one of such pilots driving on a snowy road. It was slippery, and when his car started loosing traction on one side, he was totally oversteering, and oscillating into the ditch. (just minor damage)
But I dont think telling him "your Kd / Kp ratio is way too low" would have helped ...

Without objecting to what anyone else likes, I will say only that perfectly geometric formations look to me as though the individual ships must be remote-controlled by computer or piloted by robots, rather than by living people.

Living things are messy; they under- and over-correct, just as do individual birds in a flock or human players in Josh's example of someone learning to move in the world of a new game.

For a space game with large fleets, I think my expectations for formations would be guided by the following questions:

Are the pilots human? If so, IMO there should be some slop in their positioning.

Are the pilots part of a disciplined faction? More factional discipline = tighter formations, less discipline = what's a formation?

What formations make tactical sense in fleets containing ships with different capabilities?

2. DYNAMIC SYSTEMS

Einstein is supposed to have said (but probably didn't) that the most powerful force in the universe is compound interest. Regardless of who was first with this quip, I suggest that it was a special case of a more general statement about dynamic systems: the most powerful force in the universe is the feedback loop.

Systems become dynamic, and therefore potentially interesting, when they take as inputs some of their own outputs. Feedback -- and I'm using the term in a more technical sense than human-behavioral communication -- can be negative, such as a "governor," or positive, such as the horrible squealing of an audio system when speaker output can be picked up by a microphone. Complex systems can include a mix of both positive and negative feedback loops. When they do, then, depending on how the system is designed its behavior may range from "interesting" to "this thing seems to be intelligently goal-seeking" to "OMG this thing is completely unpredictable."

The reason I mention this is not specifically about the relatively simple goal of efficient movement using the PID model. It's about Josh's comment:

I'm interested to see if this algorithm can be applied to higher-level bits of the AI, like project management.

Me, too. A well-designed but still fairly simple dynamic system should be able to evoke the impression of intelligent goal-seeking. I think the general project system should be able to achieve that.

What I'm wondering (and this isn't a preemptive criticism) is how well this will scale up into a game in which many, many projects (at some LOD) are active -- and interacting with each other -- at the same time.

At this point you may have a very complex dynamic system-of-systems. What kinds of feedback loops will exist in this ubersystem? And how will the typical player of LT perceive the effectiveness of this system based on those multiple interacting feedback loops?

I see a couple of possibilities here. One is that Josh and Adam somehow make all the planned game systems work right out of the gate. That's... possible.

Another is that the systemic interactions add up to chaos, and Josh and Adam are forced to scale back some of the dynamic features to be able to deliver a product that is confirmed to be a fun game to play.

Yet another possibility is that the planned systems are implemented, but there's so much complex interactivity that Josh and Adam can't hope to find every possible fun-destroying effect themselves. So an extended beta testing period (i.e., weeks to months) is required to discover the most serious glitches and figure out how to correct them without deleting/breaking other working features.

So I look forward to future dev logs in which the whole-game systemic stuff -- which goes beyond math and algorithms into systems design and "is it fun?" game design -- starts to get focused attention.

So I look forward to future dev logs in which the whole-game systemic stuff -- which goes beyond math and algorithms into systems design and "is it fun?" game design -- starts to get focused attention.

Flatfingers wrote:Without objecting to what anyone else likes, I will say only that perfectly geometric formations look to me as though the individual ships must be remote-controlled by computer or piloted by robots, rather than by living people.

Living things are messy; they under- and over-correct, just as do individual birds in a flock or human players in Josh's example of someone learning to move in the world of a new game.

For a space game with large fleets, I think my expectations for formations would be guided by the following questions:

Are the pilots human? If so, IMO there should be some slop in their positioning.
Are the pilots part of a disciplined faction? More factional discipline = tighter formations, less discipline = what's a formation?
What formations make tactical sense in fleets containing ships with different capabilities?

I would definitely agree that perfectly geometric formations have a limited place, more reserved for auto-pilot, robotic civs, and hiveminds. They should, like other cultural variants be an occasional-and-to-degrees encounter, sometimes being ubiquitous, sometimes nearly non-existent. One of the things about them though is that they can suggest a clearly visible cultural/factional signature, with different cultures/factions adopting different formations.

Flatfingers wrote: Dynamic Systems...
What I'm wondering (and this isn't a preemptive criticism) is how well this will scale up into a game in which many, many projects (at some LOD) are active -- and interacting with each other -- at the same time.

I think the consistent design philosophy for having a relative modularity of the various systems, and the ability to put what appears to be an almost arbitrary selection of systems in a testbed to see how they interact will serve them well here. However, I also think that there will still need to be Substantial Beta Testing. Even if the systems play nicely with each other and do as they are supposed to, that doesn't guarantee they'll be fun...and sometimes fun will emerge unexpectedly from something which might otherwise be considered broken.

Given that everything derives from an initial seed, including graphics and AI, I wonder what sort of effects different seeds will have, could some seeds be terribly imbalanced and kill the fun, while other seeds under the same algorithm be quite balanced and quite fun...

Also, loved the video review David. I'm glad you're actually waiting a bit and leaving an opening to discuss the Q&A in addition to the main post.

Challenging your assumptions is good for your health, good for your business, and good for your future. Stay skeptical but never undervalue the importance of a new and unfamiliar perspective.Imagination Fertilizer
Beauty may not save the world, but it's the only thing that can

The actual game design will be the hard part. The technical side (performance and project structure) is something that can be measured quite clearly.
The impact of game design decisions on the other hand are much harder to predict. Especially if they lay outside of the immediate gameplay experience such as the combat mechanics or visuals.
(long term fun, player progression, economic stability etc, long term behavior of ai)