Display posts from previous

Sort by

Happy April! Boy was it a long week. So much stuff happened it's hard to remember it all, but that's definitely better than the alternative. This week was all about two big pushes toward the completion of two big systems: the entity component system (ECS), and the physics engine. Neither was finished, but major progress was made on both fronts.

---

Entity Component System: Just to quickly recap, this is the part of the engine that powers all objects in the game (hence, the part of the engine most relevant to gameplay). It defines how the data that describes objects like ships, stations, colonies, asteroids, etc. is defined, managed, and altered over time in response to gameplay. From the point-of-view of most would-be LT modders (as well as Gameplay Josh), it's the most important part of the engine, perhaps followed by the UI system.

I spoke last week about the abundance of trade-offs inherent in designing a game engine's ECS. This week I continued to explore that abundance of hard choices by...implementing them. Describing all those choices in their technical detail would take too long and bore you all to tears. I started describing my current implementation in quite some detail in this log, stopped, and deleted it because I'm still too unsure about it With the lessons I learned from trying several different architectures, I started implementing a different one yesterday. But yesterday is...well, yesterday. Practical Josh, in general, does not feel confident about things being 'the answer' after only having had them around for a day.

But since I don't want to talk implementation details, I'll mentioned a new (to me) development paradigm that I've been using while playing with all of the options. Like many things I do, the idea sounds stupidly-obvious...but it almost felt like a revelation to me when I started doing it. I've been writing a good bit of gameplay pseudocode. Yep. Quelle breakthrough! But seriously:

/* Basic Mining AI Pseudocode. */
start:
self.clearActions()
targets = self.getSystem().getEntitiesWithComponent(Component.Harvestable)
miningTarget = argMax over target in targets of
target.getInventoryUsed() / self.getTravelTime(target)
self.pushAction(TravelToObject(miningTarget), align)
align:
/* Align myself 'tangent'-ish to the target. We could get way fancier here,
taking into account turret line-of-sight. Quick version is to just assume
being tangent will yield goood LOS for all. */
toSurface = miningTarget.getNearestSurfacePoint(self.getPos()) - self.getPos()
self.pushAction(AlignAxisYTo(toSurface), harvest)
harvest:
/* TODO: Maybe move around the target a bit so as to be less...boring? */
/* TODO: Fire each turret at a different (visible) surface point. */
for turret in self.getTurrets()
if turret.getType() == TurretType.TransferUnit
turret.fireAt(miningTarget)
self.notify(InventoryFull(self), full)
self.notify(InventoryEmpty(miningTarget), start)
self.notify(NotInRange(self, miningTarget), reposition)
full:
self.pushAction(DockAt(dropoff), unload)
reposition:
/* Calculate a point near the surface of the target such that all transfer
units are in range but I'm still a safe distance away */
self.pushAction(TravelToPoint(point), align)
unload:
self.transferInventory(dropoff, ItemType.RawMaterial)
self.pushAction(Undock(), start)

It's so simple, incomplete, and really quite boring. Yet amazingly, it helps me in so many ways to think about how I should design the ECS from a practicality standpoint to do the stuff that I'm actually going to want to do. Even writing this silly code, I can get a feel for challenges that aren't obvious when designing from a blind, theoretical perspective. That code isn't real; it's not any real language, just my made-up pseudocode. But what we really want is to design things such that we can write gameplay code in Lua that rivals the simplicity of the code above.

Frankly, this has been equal parts difficult and rewarding. I'm bouncing back-and-forth between the highest-possible levels of gameplay coding (I mean, it's pseudocode, it's higher-level than the high-level gameplay code), and the lowest levels of engine architecture. It's mentally jarring to constantly bounce between those levels. But it's also rewarding to feel that these low-level choices are being made with very concrete goals in mind.

Physics: Great progress and several new structures/algorithms that I've never implemented before in LT, even back in the C++ engine. The LT Core picked up a lot of code. Octrees. AABB trees. K-d trees (unfinished). I have efficient code for building octrees & AABB trees from meshes or arbitrary volumes, as well as for raycasts. Full intersection tests will come soon.

On top of implementing a ton of physics-related data structures and algorithms, I did a lot of testing to determine which ones best solved the previous performance problems with the narrow-phase collision detection on large meshes. Not surprisingly, AABB trees are winning, at least with raycasts. I can do raycasts against a large, ultra-quality station mesh in something like 3 microseconds now. Although broadphase is not really a pressing issue and will come later, it was interesting to see that my tree construction code is fast enough to actually do the dumbest thing possible without killing FPS (that is, to reconstruct the whole acceleration structure every frame, inserting all objects in the system; this is incredibly stupid compared to real broadphases, which exploit temporal coherence to do dramatically less work).

Lua & LuaJIT: As per my concerns last week, I did some investigation with respect to how to use LJ most efficiently. I used the profiling framework that I mentioned last month (and have expanded on greatly this week) to guide my investigation. I also started examining LJ bytecode dumps & trace information so that I can better understand how and where I'm losing cycles. The profiling framework has already proven itself to be invaluable!

I now have a better (but still expanding) understanding of writing efficient scripts, especially for gameplay logic. I implemented a system (in the LT Core) for script loading & running that allows better memory usage and performance than my previous monolithic architecture.

---

The coming week should be all about the entity component system. I've made great progress so far with trying a lot of implementations, and I'm getting ever-closer to a design that fits all of the requirements of LT. I'm putting this at max priority and giving myself full permission to do nothing but ECS work for the coming week. I want to start getting 'real' gameplay (not LTDemo gameplay) up; I'm ready for that pseudocode to become reality!

Hey everyone! Talvieno here, with another non-technical summary!

Josh has spent most of the past week or so working on the completion of two things - entities and physics. Just to recap, the entities are every object in the game - ships, stations, asteroids, planets, colonies - everything. The physics engine is how these objects interact physically with game space and each other - collisions, acceleration, motion and so on.

You may have noticed that he dumped a ton of "mining code" up there, and may have no idea what he's doing. Josh is basically doing his own version of "Rubber Duck Debugging". Rubber Duck Debugging is an old programmer method to figure out how to fix bugs: you talk to a little rubber duck sitting beside your computer and tell it what the problem is. In talking about the problem, you understand it better, and are often able to solve it much more easily than just from staring at the screen. Josh is writing pseudocode (fake code - "code" that's not actually code) to help him understand his problems, and finding it incredibly helpful.

The physics stuff is much more complicated, but in summary, Josh is working on collision detection. Basically, it splits up the game space into smaller, more manageable areas - splitting them up means the program has less space to check at any one time, which means a faster program. Josh includes a few visualizations of this in the form of shiny pictures.

As to the LuaJIT stuff - he's been experimenting with LuaJIT (the modding/scripting language) to try to find out how to get the most out of it - to squeeze out every last drop of efficiency for maximum framerate.

tl;dr: Josh has worked a lot in several areas the past week, mostly in solving complex problems. He's almost ready to start putting all the pieces of the puzzle together and make something you can actually play as LT.

“Whether you think you can, or you think you can't--you're right.” ~ Henry Ford

Nice! I love seeing progress happen!
I started following LT around the last few video devlogs, and joined the forums much later, so I didn't get to see much of "Progress Josh."
Also, I'm going to start learning Lua, since it seems to be going well. Hopefully I don't jinx it.

Although it looks like you are doing Octtree divisions for your collisions...
Not sure I would do that myself, but then I do 2D stuff not 3D stuff.

I would likely do something like simple regular polygons for the individual parts, and then a convex hull mesh for close coarse detection. (possibly over just a few parts per mesh)
And then a radius check where you just see if you should even bother looking at the others.
And then use very large octrees for determining if things are close enough to even bother checking the radius check.

(I can't presume to know what you are doing with detections past the coarse mesh)

Oh well, I expect you have that part nicely encapsulated, so it could be replaced later if required.

Silverware wrote:Although it looks like you are doing Octtree divisions for your collisions...
Not sure I would do that myself, but then I do 2D stuff not 3D stuff.

I would likely do something like simple regular polygons for the individual parts, and then a convex hull mesh for close coarse detection. (possibly over just a few parts per mesh)
And then a radius check where you just see if you should even bother looking at the others.
And then use very large octrees for determining if things are close enough to even bother checking the radius check.

(I can't presume to know what you are doing with detections past the coarse mesh)

Oh well, I expect you have that part nicely encapsulated, so it could be replaced later if required.

Yes, only the asteroid shot is using an octree, which was just for some variety. You're right, they're sub-optimal for narrow-phase. All the other shots are AABB trees, which will most likely be the final narrow-phase structure (although I am interested to see how K-d tree turns out; I suspect it will beat AABB tree in several cases). And yeah, this is just for the finest-of-fine intersection testing. Naturally a sphere test ('radius check') and bounding box test always come before full narrow-phase. I threw in a shot of the dynamic AABB tree over a bunch of ships just for fun, but, as mentioned, that won't be the real broadphase structure.

I may also implement convex hull, but I'm skeptical that it will yield enough perf gain in the end to matter. A good BVH (bounding volume hierarchy) should very quickly reject things that are 'obviously' not in contact, so it's questionable whether convex hull makes a real difference as an intermediate acceleration structure. With regards to LT, it's hard to see where convex hull would help a lot. The real killer cases are the ones where we have something small getting very close to a large, complex mesh (so...a small ship docking at a massive station). In this case, for example, the convex hull buys us basically nothing, because most of the time the ship will easily be inside of it, and if not, the BVH will reject quickly (because those aren't the cases that hurt). Of course decomposing meshes into convex components and using GJK would obviously be faster, but that's non-trivial and as long as the BVHs alone provide good enough perf, I'd rather not take on the extra complexity. FWIW Bullet uses AABB trees for concave meshes, although Erwin does mention convex decomp & GJK as being preferred. A better solution all-around would be to build the BVH at the same time that the mesh is being built by the procedural algorithms, so that the BVH has better information about the structure of the mesh. I will almost certainly integrate this at some point. Still, convex hull and GJK, among other things, are always in the back of my mind

It's hard to see how good the BVHs actually are for complex meshes when you visualize the entire tree at once, because you're seeing all levels of it, all the way down to the per-triangle boxes. But visualizing the tree incrementally by limiting the depth shows just how effective it is, and it's quite amazing to see how few levels it takes to reject most of the empty space around a complex mesh like a station (my AABB tree construction is volume-optimized, although not perfectly, it trades some optimality for fast construction time). I mean, obviously that's the point of the BVH, to hierarchically reject as much volume (or surface area) as possible. In practice most non-collisions should be rejected very quickly (within a few levels of the tree).

And yeah, the interfaces for all of the BVHs are the exact same, so they can be swapped out easily.

(My real-time collision detection book came in this week as well, I'll need to read it and maybe will glean some more insight).

Anyway, all this aside, I'm quite sure I'll have something good enough soon. The LTC++ detection was bad compared to this stuff, and it performed well up until the pathological cases (like the ship/station scenario I described)

“Whether you think you can, or you think you can't--you're right.” ~ Henry Ford

Thank you for another tasty update, Josh! I hope the ECS updates you're working out accomplish your design goals.

Also, yay for using very basic gameplay pseudocode to clarify ECS requirements.

Physics/Collision Detection

There is one possibly difficult question that comes to mind regarding the physics/collision work. Bearing in mind that I don't know how much time was actually spent on that, here's the question: is testing various implementations of collision detection the most important design/development work to be doing right now?

Certainly it will need to be done at some point; I understand that. But is it something that will deliver the most fun for the overall game, and so deserves to be prioritized right now? Or are there other features that are more fundamental to core gameplay that could be getting some broad-strokes outlines at this time?

The answer to this may very well be, " Yes, accomplishing highly accurate and performant collision detection really is something so critical to the overall feeling of 'fun' in this game that I do need to be working it out to this much detail right now." Or it might be, "Well, no, not really, but it's something I'll have to do eventually, and I really didn't spend much time on it, and I needed a little break from ECS work, and doing this physics work now is actually helping me figure out some bigger-picture architectural/design decisions."

If so... OK. Asked and answered.

If not....

LT Scripting For Non-Joshes

The other question I have is not a concern. It's just something that occurred to me to ask now, rather than later.

Basically, I'm curious -- yes, again -- how hard it's going to be for anyone who's not Josh to use the final scripting language to mod and extend LT.

This really consists of several sub-questions:

How much knowledge of LT internals (such as ECS rules/dependencies) will modders need?

How will mods consisting of multiple scripts and/or config data changes be packaged?

I understand that many or even all of these questions may not have answers at this time. I'm not expecting answers today. I'm simply asking these questions now to express the hope that the capabilities of LT modders and extenders who don't have Josh-like coding powers will be taken into account as the language interface (including the ECS) is worked out and the choices are made of which scripts will remain exposed to modders.

Honestly, I don't even expect direct answers to any of these. I don't want to be responsible for costing a chunk of development time; that's the evil, goateed opposite of what I want! If these questions spark any useful thoughts, I'd be perfectly happy with that.

Plus I'd be shocked if forum members didn't have some opinions of their own on these subjects. For this thread, I even promise not to complain if anyone thinks some of these questions are dumb -- though in that case, let's hear yours. Anyone else wondering about these things?

Certainly it will need to be done at some point; I understand that. But is it something that will deliver the most fun for the overall game, and so deserves to be prioritized right now? Or are there other features that are more fundamental to core gameplay that could be getting some broad-strokes outlines at this time?

I think the idea is that we are still in an uncertain phase of development where Josh is testing his lua/c++ implementation. The big question is : is it powerful enough to match Josh ambition ? collision is a game-play critical feature (obviously) so it's very important to get it as fast as possible. Think of it that way, if he doesn't optimize collision now, and later run in performance problem, he will not know if the bottle neck is with collision or another feature. If he know for sure that collision are as good as they can be, then he can rule them out which will sped up development.

Plus, the more efficient collisions are, the more objects (ship, asteroid) he can cram in the game.

Heh. I played Minecraft back in the days with the "Computercraft" mod. It only used Lua, did learn it back then through that. It was really practical because you could SEE what you just programmed IN ACTION. Hmm...but that was a few years ago. Might be a little rusty with my Lua now...

Flatfingers wrote:snip >
how hard it's going to be for anyone who's not Josh to use the final scripting language to mod and extend LT.

I kind of disagree here that this is a concern at this point. Mainly being, Josh has had so so many challenges getting LT moving and I'd personally prefer him concentrate on getting a good finished game out the door and not waste undue energy on whether some modder is going to be able to add his own ships or not.

I know it was a promise Josh made, but I'd hate to see LT derailed/delayed because of difficulties in making the game code understandable for the limited modding that will happen. As a heavy modder in other games, I'd prefer to have a rock-solid LT game whether or not I can personally make it my LT.

Thanks for the poignant questions as always Flat, since I'm working on all this code right now (especially ECS), I think answering in detail is actually as valuable to me as it is to you guys, so I will do so!

That's a tough question. First, I'll disclaim that about 2 to 2.5 days were spent on that, so the week was pretty heavily-slanted towards ECS. Still, I will try to justify the "why now?" of it, as I can see how it might not seem that (1 - dot(physicsWork, practicalityDir)) < MIN_PRACTICALITY atm. I contend that it is close. Hats off to those who understand what I just said :V

So, to get broad-strokes of gameplay outlined, we can probably agree that we need at least some collision. Lasers need to be able to hit other ships in order for combat to work at all. Scene raycasts need to be possible so that I can click on objects in-game. Real collisions (even coarse) would be nice but are not necessary for broad-strokes gameplay. Mesh raycasts would be really nice to have for shipgen algorithm, but again, not necessary for gameplay outlining.

So indeed, there aren't many cases where we really need colldet for gameplay prototyping. But there are, of course, a few (combat-related ones are the arguably the most important right now; combat simply can't be prototyped without some basic point test and raycast abilities). Now, the annoying and problematic thing about the physics engine: doing a highly-simplified version for the time being has serious drawbacks. Performance is probably the most important one, but there's also the way it affects gameplay. I can talk about this quite concretely due to the fact that I hacked together the simplest possible solutions for things that I required to make the LTDemo @ PAX and beyond work. Those hacks are still part of the demo. I see them every day, and every day it is reinforced to me that they're screwing with both the way the game works and the performance measurements thereof.

Take, for example, weapons & shooting. Lacking a broadphase engine, I can't actually detect projectiles hitting things without it becoming LTSlideshow. Even the cheapest possible test (point-sphere) will kill the perf in a battle of any real size, since we're talking O(n^2) for naive broadphase (or, to be more precise, O(n*m) where n = collidables and m = colliders, in this case n = numShips and m = numProjectiles). So how do I have combat working? By incredible hackery: when an AI shoots, the projectile knows the intended target. It will only check for intersection with that target (and by intersection, I mean the super-cheap point-sphere, not real intersection). So now it's just O(m). Ramifications: projectile hits are really reduced to 'did my projectile get within x meters of my specific target?', where x is an obscene over-estimation, because bounding spheres are bad, especially when the object is significantly larger along a certain spatial axis (like....most ships, which tend to be bigger on the Z (forward-backward) axis). Projectiles obviously pass through everything that's not their intended target, meaning combat AI doesn't have to be smart *at all*, because it can shoot right through all the asteroids it wants and still nail the poor guy on the other end. You see, the problem here is that, were I to start prototyping non-braindead combat AI, I would be prototyping under completely different game rules than will be reality. It wouldn't be throw-away code, but it'd be close...because, at least in LT, combat AI that doesn't consider the environment whatsoever isn't really worth a damn.

I haven't really put the nail in the coffin yet. One might still imagine prototyping gameplay under the restrictions I mentioned. But, now consider another demon: when I perf test battles, I'm not getting accurate readings. Even this hackery is terribly slow compared to a good solution, so it's hard for me to tell whether a big battle with lots of ships and projectiles in motion is lagging because of sim logic, rendering, or this placeholder physics. Sure, I can pull it apart piece-by-piece if I want to get a real measurement. But I can't just set up battles and get a good idea of how perf will actually be (this is also true ATM because of the lack of an ECS in C; I've spoken at lengths about how per-frame object updates in Lua are killing me). So, the whole time I'm prototyping, I'm experiencing a dramatically less-performant version of LT. That's not fun for me; more importantly, it can cause me to spuriously believe that other systems are too slow when, in reality, everything would be silky-smooth with a decent physics engine and a performant ECS. That translates to higher-than-necessary anxiety and, possibly, to wasted time investigating bottlenecks that won't actually be bottlenecks. During a time when I'm trying so hard to put FPLT in the grave, it's quite detrimental to be working in the midst of placeholder systems in performance-critical areas.

Oh, and it's even worse for the player. I've seen several people get confused by this while playing the demo. Obviously, my hack doesn't work so well for the player because...unlike the AI, I can't read the player's mind and know what ship he wants to hit. So I do the next best thing and assume that he's trying to hit the thing closest to the reticle. There are a lot of situations where this works poorly (and I've seen 'em). So again, it's not just that we have a placeholder system, it's that the placeholder system is deeply affecting the gameplay in such a way that things are fundamentally different from how they would be in LT. Not a great situation, even for broad-strokes prototyping. Of course, I've only talked about combat so far, but I'm sure I will run in to similar baddies if I were to start prototyping other types of gameplay.

Take all that, then take into account the fact that a 'good enough for now, easily-refinable later' solution is very nearly within reach, and sprinkle in how badly it breaks immersion for me when I zoom through asteroids, and you've got a clear answer: yes, it makes sense to be working on a solution that is easy enough to get to quickly, but will be performant and robust enough to give me a much better approximation of gameplay when I do broad-strokes. Don't get me wrong, I am not going to spend time making it 'UberPhysics 9000' right now. A simple broadphase and a good-but-not-microoptimized narrowphase are within reach and will make a big difference in prototypability!

The defense rests

(I didn't mean to write that much. Whoops. )

Re: LT Scripting for the Non-Josh

Short answer: basically as easy as modding can be made to be, realistically. Which means easy, provided you can get a cursory understanding of Lua.

Long answer: easy to hard, depending on how 'deep' the mod is. Let me try to give some mock-real (as in real code, not necessarily exactly how it will be though) examples. Here's an example of a mod template that might be included with the game...'modtemplate.lua'

-- This is a simple mod script template. It's for us simple folk.
-- Returns basic information about your mod: name, author, version string, description, dependencies (if applicable)
function getModInfo ()
return {
name = "My First Mod",
author = "John Smith",
version = "1.0.0",
description = "This is a Limit Theory mod!",
}
end
-- Called before scripts are loaded, during runtime compilation of game-related ECS code
-- Here you can change the low-level data and code used for implementing gameplay
-- WARNING: This is a low-level modding layer intended for mods that deeply change the game; most should not do anything here
-- You can, for example:
-- Add fields to datatypes (e.g. add a new property to items)
-- Define new C functions that can be called from your Lua code but have direct access to ECS data
-- Override the default ECS C functions
-- Write high-performance helper functions if your Lua scripts need more speed
function onCompile (ecs) end
-- Called during engine initialization, after the default LT scripts are loaded but before any game code runs.
-- Here you can change constant data, re-bind default functions to change scripted behavior, add new behaviors, etc.
function onLoadScripts (scripts) end

Notice that there are two fundamental levels at which you can mod the game. Most mods will require only the second level, which is purely dealing with changing and extending the default Lua scripts. Notice also that the architecture of a mod is fundamentally 'diff-like.' Instead of a mod being like a file-replacement, mods are just scripts that get called when they're loaded. Due to Lua's design, this gives you all the same power as file replacement, but this architecture makes mod conflicts far less likely and far more manageable. Even mods that require the first (low-level) modding layer can work together with no problem due to how the ECS is structured. You could have several mods active that add several new properties to the Item class in the game and they would work perfectly as long as no mods try to use the same name for a new property. The engine handles 'merging' in a very natural way at both modding layers. As a last-resort you can still override files, but this will be discouraged for scripts (it makes sense for other asset types, though). In fact, the packaged mod file will know which files the mod alters, so LT can automatically notify of conflicts. It can also automatically show warnings for mods that override scripts. I plan to have a full set of mod 'warnings' that give players information about a mod (for example, a mod that accesses the networking API will be branded with a big fat "Warning: This mod is requesting access to networking functions. It may attempt to communicate over the network. Only enable the mod if it is clear why it should require such access.")

Without further adieu, a simple example mod that removes drag from the game:

function getModInfo ()
return {
name = "Out of My Way!",
author = "Flatfingers",
version = "1.0.0",
description =
"You're a pilot of importance...at least to your subordinates. They should move the frak out of the way when you're trying to get by.\n" ..
"Especially useful when you're in formation with hundreds of fighters.",
}
end
-- Higher value means stronger repulsion field from the player's ship
local kRepelStrength = 1.0
-- Higher value means repulsion field is attenuated more strongly with distance
-- 2.0 is an inverse square field like gravity & electrostatics
local kRepelPower = 2.0
local function moveAway (asset)
local toPlayer = getDirection(asset, getPlayerShip())
local distance = getDistance(asset, getPlayerShip())
asset:applyForce(-toPlayer * (kRepelStrength / (distance ^ kRepelPower)))
end
local function attach (asset)
if asset:hasComponent(Component.Motion) then
asset:addListener(Event.InProximity(asset, getPlayerShip()), moveAway)
end
end
function onLoadScripts (scripts)
scripts.callbacks.onPlayerAcuireAsset:add(attach)
end

Note that I could have separated my script logic out into a new lua file and loaded it in my main script file instead. Note also that, again, this mod is virtually guaranteed to not conflict with any others. New functionality is (often) added via the event system. Old functionality can typically be modified in a similar manner. Finally, note that this is legitimate Lua code and is actually representative of what a real, working LT mod might look like. It's not even a trivial mod: it adds a handy repulsion field to the player's ship so that his own ships move out of the way. Sure, it's simple: the field is actually physical force; it would be more elegant to have the pilots reposition their ships instead. But this is a demo of a simple, practical, quality-of-life mod It could also be fun to make the constant very high and push big fleets of ships around like a ship-shepherd (that is...a...Commander Shepard ).

Finally, the pièce de résistance, an ancestry mod, requiring use of the low-level api:

function getModInfo ()
return {
name = "Ancient Ancestry",
author = "Talvieno",
version = "1.0.0",
description =
'"Josh Parnell? As in...THE Parnells? The same ones that own BumbleBoom Projectiles?" Now you can find out.\n' ..
"All NPC players keep track of their parents; their ancestry can be explored freely for the curious (or bored) pilot."
}
end
function onCompile (ecs)
local GenderT = ecs:addEnum('Gender')
GenderT:add('Male')
GenderT:add('Female')
-- Look, it's a game, for the purposes of AI reproduction I need gender to be either male or female, sorry :P
local PlayerT = ecs:getType('Player')
PlayerT:addField(GenderT, 'gender')
PlayerT:addField(CType.RefPointer(PlayerT), 'father')
PlayerT:addField(CType.RefPointer(PlayerT), 'mother')
end
local function recordAncestry (player, colony)
local colonists = colony:getSpawnedPlayers()
local father, mother
local rng = getRNG()
while not (father and mother) do
local parent = colonists:chooseRandom(rng)
if parent.gender == Gender.Male and not father then father = parent end
if parent.gender == Gender.Female and not mother then mother = parent end
end
player.father = father
player.mother = mother
if rng:chance(50) then player.gender = Gender.Male
else player.gender = Gender.Female
end
local function buildMenu (player, menu)
menu:getSection('BasicInfo'):addPropertyLabel('Gender', tostring(player.gender))
if player.father and player.mother then
local section = menu.addSection('Ancestry')
section:addLabeled('Father', section:addTextButton(player.father.name, function () createInfoMenu(player.father) end))
section:addLabeled('Mother', section:addTextButton(player.mother.name, function () createInfoMenu(player.mother) end))
-- TODO: Use NodeTree widget to show a graphical family tree!
end
end
function onLoadScripts (scripts)
scripts.callbacks.onAIPlayerCreate:add(recordAncestry)
scripts.menu.info.player.callbacks.onCreate:add(buildMenu)
end

Boom. There you have it. Legitimate example showing how low-level ECS data structures can be modified to painlessly and efficiently accommodate a mod that, for instance, needs to keep track of parent information (plus gender in order to generate that information). Also some mockup-real-code showing adding information to the UI. Wwhen the player pulls up the information UI for an AI player, they will now see a text label giving the player's gender, as well as a new sub-section that shows the mother and father, which are clickable links that go to their respective info menus...allowing a player to quickly browse back through time and look at the ancestry of LT AI players!
I hope this answers the sub-questions via example. To clarify further, answering your bullet points in-order:

Modders will need at least a basic understanding of things like ECS, the event system, the built-in basic functions, etc. I will provide documentation on all of the above. Even better, I will provide as many mod examples as possible that demonstrate the 'right way' to do various things, straight from the horse's mouth.

Functionality can be dramatically affected. I try to design the API so as to minimize unintentional effects. As usual, the architecture is designed to make performance something that most modders don't have to worry about if they're following the right way of doing this (i.e., using the event system rather than trying to run every frame). Mods that do things that require extreme performance will have to be more careful, as they will be defining new C code. Most mods won't need that (like, the vast vast vast majority; but it's there because it's how I will lay down a lot of the ECS fundamentals that don't belong in the LT Core yet still require max perf). Even then, I provide an API for adding new low-level functionality (i.e., compiled C) that handles as much of the 'nasty' stuff as possible (in other words, you're not writing full C files yourself). So even the most beastly of mods (I'm looking at you, LT: Multiplayer) will still only require minimal raw C in the perf-critical parts; I provide many mechanisms to avoid having to resort to this

Some scripts will be considered 'not advisable to change,' but anything that isn't baked into the LT Core is, by definition, subject to change. Code that absolutely should never change is compiled into the library. Some low-level Lua support scripts may be tagged as 'non-overridable.' In general, though, I will rely on modders to be sensible: you don't really get much out of breaking everything. But, ideally, you'd need to be trying to break things to really break them. (I SAID IDEALLY )

That kind of thing would likely be enforced by path. Off-limits scripts would be in a different folder that the engine internally considers non-overridable when it loads mods. Alternatively, I can bake those scripts as Lua bytecode and keep them in binary format, so that there would be no 'off-limits' lua files.

Modders have access to virtually every function in the LT Core engine + everything in script so...yep, anything I use to debug / profile will be equally available to modders.

Multiple scripts / mod assets are packaged in a folder that mirrors the folder structure of the LT assets. A simple mod like above would just be a folder with a single file, 'mod.lua' in it. A more complex mode might have 'myMod/mod.lua', 'myMod/tex2D/coollogo.png', 'myMod/shader/fragment/metal.glsl', 'myMod/bindlogo.lua' ...for example. The folder is then fed into a simple tool that I provide that turns it into a single, compressed .ltm file (or whatever extension) that includes the compressed directory structure as well as metadata about the mod. Simple, simple, simple.

Whew. Back to coding. FWIW that WAS development time. Especially those mod examples. They helped me think through a lot of things! So...thanks for the questions

“Whether you think you can, or you think you can't--you're right.” ~ Henry Ford