AuthorTopic: Future of the Fortress (Read 727132 times)

Not to extend the FPS discussion, but since optimisation in real time systems was my job for a decade or so, I thought I might chime in. One of the things that's tempting... well, not so much tempting, but virtually impossible to stop your self from doing, is thinking that you have a good idea how to speed something up. In my experience, the problem with performance is never what you think it is. Frequently people will decide what to do to speed something up, do it, and walk away patting their own backs -- but nothing has really improved. In fact it often gets *worse*. It's incredible how often I've seen that professionally.

When I go into the problem I always caution people: It's never what you think it is, so stop thinking. You really need to set up experiments, measure, analyse the code, go back to the beginning. It usually ends up being something ridiculous that you would never even imagine. I remember one time going through this insane parser with crazy logic and discovering that something like 99% of the execution time was in the string allocator. It never even needed to allocate memory for strings -- it was just a convenience in some of the code. Another time I remember working on some completely over the top database query code only to notice that it was allocating 500K date objects in a few seconds -- it turned out that someone created a date in a function rather than using a constant.

When you have thousands, hundreds of thousands or even millions of lines of code, there will be some stupid code and it will be on the critical path. Usually (but not always) that's the problem. Very, very occasionally you discover that you just build something that doesn't scale and you have to start again. That latter thing is unusual in my experience, but very frequently people jump to the conclusion that "It's all crap and we have to start again!" And when they do, they put some stupid code on the critical path and the new code is just as slow as the old code. The obvious things usually end up not being the problem precisely because they are obvious -- the performance issues for those things have already been mitigated a long time ago.

Suggestions for what Toady should do to improve the FPS are obviously well meaning, but they are unlikely to be helpful. It's never what you think it is. Whether or not Toady will spend a large effort to optimise at this juncture, I don't know (he's reported *some* optimisation recently!) I don't think it's bad to say that it's your highest desire, but "Would you consider multithreading" (or some other proposed solution) is unlikely to be constructive.

And to add to mikekchar's comment: Before you start to optimizing the code by doing "clever" things to wring a few additional CPU cycles out of the code, check that the algorithm you're going to optimize actually is the right one. Optimizing a linear search implementation may give you 10% faster code (for that little part of the code), while switching to a completely unoptimized binary search when the data is ordered is going to give you much more (and probably with a lot less effort).

Thanks to Knight Otu, KittyTac, PatrikLundell, Shonai_Dweller, MrWiggles, King Mir, iceball3, and anybody I missed who helped to answer questions this time!

Quote from: FantasticDorf

Would this be the case for specific location structures indicated for scenarios, like a dungeon/central jail to keep a prison colony functional and under control? (not meaning exactly a Australia format wherein just being very far away on a strange continent is detainment with nowhere to run to) - rather than just a jail & justice system we have currently.

It's quite possible. In that specific example, a location might be a tad small as you suggest; perhaps it's more likely the current fort jails might be considered a 'dungeon' location properly, whereas the prison colony is a more site-level consideration. But yeah, the space in between is interesting, and might work as a location complex with several zones, if the prison colony sufficiently restricts movement and has a sufficient administrative or civilian presence.

Quote from: iceball3

Have you or threetoe considered constructing an written full psychological profile for dwarves, or other races, as part of development aims and Slaves to Armok canon, or will species psychology take a "feature first" implementation?By feature first, I mean that the behaviors ingame stand to be canonical for the time being, and (both emergent and intentional) behaviors that come in later version serve largely to supplement this model.Our dwarves and other species do have a general cultural and behavioral context portrayed by the game's features and snippets combined with the general consensus of what a "dwarf" is, though there are currently quite a few quirks as a result of bugs, limits of current implementation, etc where they seem to differ and vary dramatically in these contexts (for instance, noted complaints that dwarves seem somehow stress-proof to a lot of people as a result of the systems or morales changing, though whether it was aimed at making dwarves more resilient to stressful situations, or as resilient is a bit unclear.) Similar goes for things like the implementation of the currently somewhat implemented engagement level system.

We don't have particular additional plans for a push here; the raw files have been growing over the years, so that the psychological spectrum is actually present now (looking at PERSONALITY and MENTATT tags), but not more particular cultural information. That said, there's an implicit push within law/property/status/customs to define these things in more detail than the current "ethics" system, though having mythgen at that time will also introduce a procedural element.

Regarding stress, dwarves have [PERSONALITY:STRESS_VULNERABILITY:0:45:100], so they should be a touch more vulnerable than others. It's just the broken mechanics that cause the whole system to be messed up (human residents are even more 'stress proof'), making all of this irrelevant for the time being.

Quote from: PlumpHelmetMan

This might belong in the suggestions forum but...a number of fantasy stories out there feature mortal heroes that after a lifetime of great feats manage to achieve literal godhood upon their deaths. Might this possibility eventually be implemented into DF (it'd be quite a surprise following an unusually successful adventuring run to find one of my characters showing up in the deities tab of legends mode after a fatal battle with some terrible monster )?

The test myth generator doesn't code for this, but it's the sort of transition that's on the table. There's a notion of conduct and such that may or may not determine ones final disposition, and state changes might well include godhood or becoming a star/constellation or something -- it's certainly an option to be considered, and the underlying system will works with general-enough concepts that various outcomes can be introduced.

Quote from: Beag

1. Will the actual process of casting magic involves casting procedures with as much, less or more detail as the procedures we currently have for performing music, poetry and dancing?2. In terms of magic derived from gods would the procedures for casting it be interlinked with religious procedures for that god(ex. ceremony's, rituals etc.) ?3. Will the attributes needed to be talented at magic be different with each magic system(ex. one magic system relying heavily on creativity while another relying heavily on linguistic ability)?4. Can you please give some examples of procedures required to cast magic for some systems?5. Can you please give some examples of magical land masses and their implications on magic in that world?

Examples/answers were given for all of these, but expanding, it might not just be that magic involves the same amount of detail as the current dances; it might require one of the current dances explicitly. Generally, rituals (religious or not) will probably live under the same 'activity' structure guiding such actions, though 'interactions' and 'reactions' will also be fighting for a piece of the pie -- procedure-wise, the 'Do X to Y' stated by Shonai_Dweller is correct; some things are harder than others, but with reactions we already have access to tool and material use in a way that dwarves and adventurers can both utilize. In a sense, a magic ritual just requires you to check the same boxes as a typical dwarves task, in many cases. Fortunately, the infrastructure is already mostly in place, though there might be an uncomfortable merging to make them all play nice together (as with the building-room-zone-location-pile-burrow nexus of headaches.)

Different attributes is a definite yes, and moreover, certain supernatural attributes might be generated on a per-world basis and given thematically appropriate names. This is similar to the 'corruption' states, though it's unclear at the moment whether they'll be stored the same way; the range from transient state to skill to attribute to fixed state are all appropriate, and perhaps all will be used.

In the myth tests, land masses operated on a proximity or material basis, so you'd either mine, say, giant blue mana pillars or some crap, or be in the presence of a massive floating egg fragment. Both of those rely on the hard map rewrite, but in the event we don't jump on that first, the mineral layer code can still be co-opted for similar gains, in which case proximity/use of magical materials would still be on the table. In the harder and more interesting cases, magical land-masses would be more alone the lines of the turtle that holds up the world, or a world tree that connects to other areas, that sort of thing; the game would by necessity have a very granular understanding of how that tree works, so you'd get into magical bark and sap layers, that sort of thing (though I wouldn't expect a general liquid type rewrite for this! That might be a step too far.)

That's all quite material-mechanistic, but we also have access to other variables, including social relationships, which can be intermeshed with these matters. It still needs to be felt out, but we want to make at least a few bold moves.

Quote from: Max^TM

I think it's been asked before but not specifically in regards to the Focused! status: is there any reason why martial trances seem so rare for adventurers, even in situations where your companions fighting right alongside you activate them, and similarly does being focused/unfocused have any effect on the activation rate?

Could it be related to aiming attacks rather than move-autoattacking or something perhaps?

Hmm, it looks like that part might have gotten moved out for players when vision arcs were added, since trances rely on the number of people you see targeting you. I'll make a note to change/investigate.

Quote from: Inarius

Are you worried by the general and continuous reduction of activity on the forum for the last 2 years ?

I'm aware of the forum thread and stats and considerations and graphs there. It's not surprising that the general google searches have dropped after the NY Times and MoMA stuff dropped out of awareness/relinking, and there hasn't been anything comparable recently.

New member changes have to consider the screening put in a place a few years back, coupled with deletion of spam accounts; the 27000 new members recorded in 2011 in the forum stats is entirely inaccurate, as most were spammers; the new member forum stats up until 2015 are almost meaningless.

But yeah, more reliably, new posts and topics have declined. On the other hand, looking at say, the median number of people online on June 2011-7, it's a mixed picture: 539 -> 691 -> 600 -> 671 -> 502 -> 556 -> 585 (the average is nearly identical), indicating something of a two year recovery despite the decrease in topics/posts, and everything is in the same ballpark.

So, it's not worrying me currently, but I understand the concern, as some engagement metrics are definitely declining.

Quote from: StagnantSoul

Is there anything planned to deal with dorfs getting stranded in trees when picking fruit? I lost a lot of dwarves to this already, probably two forts worth.

As mentioned, I did fix a type of this for the latest clump of versions, and saw them come down from the trees, but I also saw the new bug report. I'll get to it and see what's up.

Quote from: TheFlame52

In worldgen, we can see nations fall. When can we expect to see them rise? Mythgen? Economy?

In between those two, perhaps; the laws/property/status/scenarios release is the first time we'll have enough control to split entities and form new ones at the civ/culture level, though it's still a difficult problem (if this is what you mean.) The other version, where new government structures are put on top of existing civs/cultures, is also a problem, and the code splits needed there are part of 'status'. All of this mess where culture and high-order governments are linked has to be teased apart, and it's one of the fundamental requirements of that release.

Quote from: thvaz

Do you still play the game for purposes other than testing? And what about Threetoe?

I thought about this after reading the last devlog. Even if you are playing for fun you could still see many problems players won't ever imagine as bugs. Though as DF is very complex and allow very diverse playstyles, most of the problems are very playstyle-oriented, I imagine.

No, we don't get much of a chance; Threetoe plays a bit more than I do. And yeah, when I get my hands on saves, I do catch certain things; though as you say, that might actually be the better way to do it. I play a certain way on the occasions when it comes up, and those narrow tracks are probably already less bug-ridden than others.

Quote from: iceball3

Hey Toady, what's your opinion on the Dark Souls 3 Mound-Makers?

Ha ha, I don't have an opinion on it. Is it a contentious issue? Or do you mean for world-building flavor? I looked up what they are, and it seems quite multi-player inflected and so not likely to arise, though perhaps if we're lucky the magic system could attach certain significance to signs in bones and such; though we don't currently have individual vertebrae!

Quote from: Shonai_Dweller

During the taverns update development you (maybe, correct me if my memory is faulty) mentioned the fortress guards getting involved in tavern brawls (breaking them up? Arresting folk afterwards?). Is that something you're still thinking of adding in this cycle or is that too big now?

Would be a nice addition. Balance reduction of tavern deaths through timely intervention of guards with having to fulfill mandates to avoid punishment deaths. Or just opt out with an alcohol(tavern keeper) free tavern as folk are free to do now.

Ah, yeah, I think I did mention something like that. It's possible the thought got shunted to law or crime like other things. I'm not quite sure how to make it timely without changing the escalation pattern in brawls; I imagine a lot of the tragedies strike within a few foot steps of the brawl initiating (and others maybe take days and days of interminable pummeling and might actually be stopped by a guard...) I'll mull it over a bit. The actual process of grabbing somebody and dragging them off to jail for assault or disorderly conduct (or to cool off) at least would just use existing code, so it's not out of bounds to consider.

Quote from: Lordfiscus

Will creatures with transmissible symptoms be able to intelligently apply their poisonous substances onto an object in order to lay a trap, or otherwise weaponize it for biological warfare? For example, thralls could try to coat doors or loose objects with the dust that created them, so anyone who touches that object becomes another thrall. Or, certain sentient civs might be able to capture a forgotten beast and coat their weapons in the toxic blood/dust of the creature to provide an edge in combat.Also, will there be a way to create temporary pocket dimensions to create bags of holding or similar? I'd like to be able to make a crystal or jar containing soldiers, then throw it at a target and have the dimension in the jar destabilize as the jar shatters, ejecting all of the occupants on top of the target. That, or do the same thing, but fill the jar dimension with lava instead, and make a magical incendiary weapon.Thirdly, will fleshcrafting and gene manipulation be a possible magic discipline? Will we be able to clone dwarves or summon syndromes as an area-denial weapon?Lastly, will necromancy be given a more complex skillset, and thus allowing dwarven necromancers to play a greater role in dwarf society? I was hoping to have necromancy be akin to a noble position, requiring a certain quality of room. One could designate the necromancer to raise a corpse, and the raised corpse could have certain skill specializations and respond to the appropriate labors. Better yet, a workshop by the name of Necromancer Altar or similar, where you could order the necro to raise a corpse and give them skills for certain labors. They would be able to perform labors befitting their skillset, but would be unable to improve their skills. If necromancy can be likened to a skill, then a higher necromancer skill level would allow the necromancer to raise corpses with higher skill levels or combinations of skills. Novice necromancers would raise corpses with one Novice level, Adequates could provide undead with 1-2 Adequate skills, and so on. The most proficient or legendary necromancers could raise highly intelligent undead with a wide variety of disciplines. They might even be able to give their undead necromantic skill, for the purposes of delegation or backups should the original necromancer die.

Poison. They used to do this long ago, and we never got back to coating weapons in full (though kobolds do it at their sites now). We hope to get back to it.

Pocket dimensions. It's on the menu, and not super-hard in certain senses, if the hard map rewrite is done; it's something we're fond of, so perhaps more likely than you'd expect. Otherwise, we'll likely wait, since any stop-gap measure would be completely tossed later on.

I'm not sure what fleshcrafting and gene manipulation specifically mean here; we've had things that seem related to those terms in the notes anyway, but I can't promise anything.

When necromancy is proceduralized by necessity with the first myth release, and magic is incorporated into societies, the idea is yeah, that certain things along these lines will happen, as it regards positions, apprentices, fort-mode buildings and so forth. I'm likely not going to have such a specific preset, but hopefully things like that will be part of some universes.

Quote from: SmileyMan

Just a quickie regarding the last devlog - will the new outdoor construction cleaning include roads?

Yeah, roads and bridges and anything else outdoors with a 'building' flag will be cleaned, though it still requires an idle dwarf in the area currently to get the job started.

In my experience, bar brawls generally end up with a quick succession of minor injuries, which are kind of fun (har, har got me lip torn off) and fatalities which aren't (like when someone starts pummeling the cheesemaker in the head repeatedly, including tearing off his headgear if he's wearing any). Quick intervention from the guard might help there.

Although once the off-duty military and elite wrestler visitors get involved there's not enough time for the guards to react unless they're already there. They'd just get to mop up the mess.

Poison. They used to do this long ago, and we never got back to coating weapons in full (though kobolds do it at their sites now). We hope to get back to it.

Is that true? I occasionally notice "slingers" from my mod (renamed blowgunners to simulate npcs using "thrown" weapons) have their ammo coated in things like bumblebee venom, atleast if they're bandits - or is that a separate thing?

They certainly coat the weapons in their weapon traps in poison. But never actually seen them wield poison coated weapons. Then again, it's very rarely you actually see a kobold manage to hit someone, so...

Would this be the case for specific location structures indicated for scenarios, like a dungeon/central jail to keep a prison colony functional and under control? (not meaning exactly a Australia format wherein just being very far away on a strange continent is detainment with nowhere to run to) - rather than just a jail & justice system we have currently.

It's quite possible. In that specific example, a location might be a tad small as you suggest; perhaps it's more likely the current fort jails might be considered a 'dungeon' location properly, whereas the prison colony is a more site-level consideration. But yeah, the space in between is interesting, and might work as a location complex with several zones, if the prison colony sufficiently restricts movement and has a sufficient administrative or civilian presence.

Isn't there an entire game about this already? (What's really amusing is that that game became a complex enough simulation that the developers started referring to it as "our Dwarf Fortress". Probably an exaggeration, but... )

But in that game your workers will do their job as soon as possible, you don't need to defend your place from armies of goblins and zombie hair,and you can't flood your place with magma if you are board so what's the point

Will certain magic practices be illegal or viewed with judgement? Like raising corpses right now, except it is hurling fireballs around or turning limbs into buckets of water because it came from some evil god or something

Is there an in-universe explanation for how/why an elf is able to devour 80 humans in a single sitting?

They're evil?

Logged

"Continue struggling, laws of physics be darned." - NRDL"What, are you stupid or something? Every one of the snake's bones is crushed! Its internal organs are torn apart! There is no way you can - (6) You resuscitate the snake."- Gatleos