I'm still not sure I even understand these in my opinion complicated parameters. :P Maybe I'm just dumb. ^^ I don't suppose anyone could be kind enough to make me a .cfg that's compatible with Doom Builder 2, one that compresses the blockmap in my vanilla map? Sorry if this me asking too much. Maybe I could do something in return, like playtest maps or so?

In any case, the reason to why I want to use ZokumBSP to mainly compress the blockmap is that as explained in this thread: https://www.doomworld.com/forum/topic/99867-please-help-i-have-a-strange-crash-bug-in-my-large-vanilla-map/ , I'm having trouble with a vanilla map I'm making, that has this strange crash bug, which crashes Chocolate Doom/renderlimits only. Basically, when I walk into one of three rooms that have this problem, Chocorenderlimits just bombs out to Windows without an error message at all. The program also crashes whenever I fire my weapon inside these rooms. In other parts of the map, there are no issues. The map is 1.33 mb large so I got advice that it may have something to do with the blockmap or sidedef limit.

So that's why I wanted to try ZokumBSP. Though according to the aforementioned Beginner's Guide to ZokumBSP, it states this: "When a vanilla mapper is really trying to push a map's size/complexity, they can bump into doom's blockmap limit and result in the engine crashing upon loading the map."

That doesn't rhyme with the problems I'm having though. My map works without problems in most parts, it just crashes ChocoDoom in these three rooms, it doesn't crash whenever I start the map. Is this true, that going over the blockmap limit in a map, always leads Chocodoom to crash at startup? In that case, it can't be the blockmap that's causing this.

Share this post

Link to post

The engine may crash when loading, or it may crash later when you fire, or something moves in an area "outside" the blockmap. The blockmap starts in the bottom left corner and ends in the top right. If the problematic rooms are at the top, it's highly likely it is a blockmap problem.

Just open up a cmd window and run zokumbsp manually. The most recent version gives you various information about how close you are to the limit etc.

Share this post

Link to post

The engine may crash when loading, or it may crash later when you fire, or something moves in an area "outside" the blockmap. The blockmap starts in the bottom left corner and ends in the top right. If the problematic rooms are at the top, it's highly likely it is a blockmap problem.

Just open up a cmd window and run zokumbsp manually. The most recent version gives you various information about how close you are to the limit etc.

Which parameters do you find hard to understand?

Hey! And thanks for the reply.

Ah ok. Indeed the problematic rooms are at the very top of the map.

The version of ZokumBSP I'm using is 1.0.9rc1. That is the latest official release, right? Will that show all the information needed about the blockmap?

I opened a cmd window and ran zokumbsp with these lines: "zokumbsp mapname.wad, was that correct?

I guess I find it all difficult to understand. I know I'm in a kinda stressed-out place in life at the moment so that could be why I find it all so hard to understand. I don't suppose you could help me out in editing a .cfg I have so it works in Doom Builder 2? If the problem indeed is about the blockmap then all I need is for this .cfg to always execute the command to compress the blockmap whenever I build nodes in my map in Doom Builder 2. Can you help me out with this please? I could do something Doom-related for you in return, like test your maps if your a mapper, etc. In the .cfg I'm using there are also some optimizations so that the engine compresses drawsegs and visplanes. Do you think it's possible to keep them in the .cfg, and make them work with ZokumBSP? Those optimizations are for Zennode.

I had two other questions about ZokumBSP as well. How about source port compatibility? Are there any programs that won't load my map once it's been blockmap compressed?

And how about demo compatibility? Can you just make regular -complevel 2 demos that will work on such a blockmap compressed map? And if not, is there a way to solve this so that demos can be recorded?

Share this post

Link to post

By looking at this, it sounds like "zokumbsp_optimize_heavy" or "zokumbsp - Optimize for vanilla (time consuming)" as I imagine is the line that will show up in Doom Builder 2, is the best bet. This setting compresses the blockmap, right? Does it also have parameters that compresses drawsegs and visplanes? I have this .cfg right now that Esselfortium made for BtSX, which have parameters that compresses drawsegs and visplanes. The cfg looks like this:

The output should be much easier to understand in that build. I talked to several people and redid it completely to make it easier to see how close you are to the vanilla limits. If it doesn't have the improved output, you'll have to compile your own from source. The stuff on GitHub is much more recent than that version.

If you still have problems, you might give some of some of the user-side optimizations a try. There are specific linedefs you can use to simply exclude those lines from the blockmap. Typically you'd do that on decorative lins in the ceiling, backside of doors that never close, etc. It's your map and you know it better than anyone else. The lines that need to be in the blockmap are the ones that can be hit with shots or projectiles or lines monsters or you can collide into. Experiment a bit and test it. If you're really desperate, you can usually remove the bottom step of stairs from the blockmap without people noticing it :)

Share this post

Link to post

About the -n2 parameter and some others. They are the old ZenNode parameters. I have retained them since I wanted mine to be a slot-in replacement for ZenNode. It's much easier to use the "full" parameters since it's more obvious what is going on and what is being changed.

I'd also say thanks for the offer about doing me any Doom favors. To be honest, play around with ZokumBSP, test it, give me feedback. It should be noted that Anotak's fork of DB2 comes with ZokumBSP support out of the box.

On a general note about segs and vpos, and we're talking VERY generically here. It's possible to tune for either a balanced "build" of a map or fewer nodes (possible less VPO) or fewer segs. You can't really compress SEGS in vanilla wads. Zokumbsp allows you to tune the node building project depending on what your needs are. In general, fewer nodes can give fewer visplanes. A visplane overflow is fatal and leads to the game stopping. Too many SEGS just give you flickering rendering. The exception is when you reach the maximum amount of segs. If you do that, I would definitely aim for fewer SEGS.

It's also possible to make the nodebuilder ignore that it is highly likely a slime trail will be visible. The division lines that lead to slime trails might also build a map with fewer vis planes and fewer segs. To be exact what the code does is find the best division line, but if that one has too high of a slime trail factor, it will go for the second best line etc. There might be room for some optimizations here, I might look into that.

Share this post

Link to post

About the -n2 parameter and some others. They are the old ZenNode parameters. I have retained them since I wanted mine to be a slot-in replacement for ZenNode. It's much easier to use the "full" parameters since it's more obvious what is going on and what is being changed.

I'd also say thanks for the offer about doing me any Doom favors. To be honest, play around with ZokumBSP, test it, give me feedback. It should be noted that Anotak's fork of DB2 comes with ZokumBSP support out of the box.

On a general note about segs and vpos, and we're talking VERY generically here. It's possible to tune for either a balanced "build" of a map or fewer nodes (possible less VPO) or fewer segs. You can't really compress SEGS in vanilla wads. Zokumbsp allows you to tune the node building project depending on what your needs are. In general, fewer nodes can give fewer visplanes. A visplane overflow is fatal and leads to the game stopping. Too many SEGS just give you flickering rendering. The exception is when you reach the maximum amount of segs. If you do that, I would definitely aim for fewer SEGS.

It's also possible to make the nodebuilder ignore that it is highly likely a slime trail will be visible. The division lines that lead to slime trails might also build a map with fewer vis planes and fewer segs. To be exact what the code does is find the best division line, but if that one has too high of a slime trail factor, it will go for the second best line etc. There might be room for some optimizations here, I might look into that.

First off, thanks to you both Vorpal and zokum for your help so far! :)

To zokum:

Late last night I tried this .cfg that Vorpal sent me containing this stuff: (full .cfg can be found above)

I tried out both these options, and both of them solved the issue with the crashing on my map in Chocolate Doom/renderlimits! And both these settings seemed to keep the drawsegs/visplanes on the exact (or very very close) level to the settings of my other cfg I've used for Zennode so far. So far so good.

But then when I try out the map in four different source ports, (GLBoom & PRBoom Plus v2.5.1.4, ZDoom 2.8pre-release as well as GZDoom v1.8.2), I get this bug that makes it possible to walk through and inside one-sided walls without using idclip. Only some walls can be walked through, but it seems that it's always the same walls that can be walked through, regardless of source port used. Is this a known bug? And do you think you can fix it anyways? Are there other potential problems in these mentioned source ports or other ports? In Chocolate Doom/renderlimits, this bug doesn't appear for me, the entire map now works flawlessly in those ports.

By the way, does the parameters in the zokumbsp cfg above do anything at all to compress the blockmap, visplanes or limit the drawsegs? It does say Optimize for vanilla so I assume so, just want it confirmed please. :)

I was also wondering, is it possible to record demos with -complevel 2 on maps with zokumbsp and these parameters? Or will they desync in Vanilla/choco/other ports?

Share this post

Link to post

Many years ago, Lee Killough added an optimization in MBF that wasn't all that well thought out, but since a lot of ports are based on MBF, they have all inherited this bug. Some of the newer ports have had this fixed. The option z=0 is the problem with these ports. The ports are not 100% vanilla compatible.

That is why the option is marked as "Optimize for vanilla". It works fine in vanilla doom, chocolate doom, recent releases of eternity etc. It might also be fixed in prboom+, there's a list in a thread on the forums about this. It should be noted that several ports have a workaround for the bug when playing back demos (lmps), since this port change also broke demo compability. The ports just have to turn off the faulty optimization when it detects highly optimized blockmaps.

It should be noted that the z=0 is a very efficient size optimization, your blockmap will grow a fair bit without this on. Some of the geometry optimizations and manual optimizations could possibly make it small enough if it goes over the limit with z=1 (the default).

I'm not an expert on what the complevels do, but if those settings would normally produce vanilla compatible demos, it should also do so with these optimizations turned on. I value vanilla-compability, ZokumBSP is one of the few, or only tool, that can rebuild blockmaps without demos desyncing. IdBSP / DoomBSP can of course also do that, but it doesn't really rebuild as it uses dwd files as input.

Share this post

Link to post

Many years ago, Lee Killough added an optimization in MBF that wasn't all that well thought out, but since a lot of ports are based on MBF, they have all inherited this bug. Some of the newer ports have had this fixed. The option z=0 is the problem with these ports. The ports are not 100% vanilla compatible.

That is why the option is marked as "Optimize for vanilla". It works fine in vanilla doom, chocolate doom, recent releases of eternity etc. It might also be fixed in prboom+, there's a list in a thread on the forums about this. It should be noted that several ports have a workaround for the bug when playing back demos (lmps), since this port change also broke demo compability. The ports just have to turn off the faulty optimization when it detects highly optimized blockmaps.

It should be noted that the z=0 is a very efficient size optimization, your blockmap will grow a fair bit without this on. Some of the geometry optimizations and manual optimizations could possibly make it small enough if it goes over the limit with z=1 (the default).

I'm not an expert on what the complevels do, but if those settings would normally produce vanilla compatible demos, it should also do so with these optimizations turned on. I value vanilla-compability, ZokumBSP is one of the few, or only tool, that can rebuild blockmaps without demos desyncing. IdBSP / DoomBSP can of course also do that, but it doesn't really rebuild as it uses dwd files as input.

Ah ok, thanks a lot for pointing this stuff out! :)

By the way I've now done some further testing, and I'd thought I'd report my findings.

Today I tested out the latest version of GZDoom (v3.2.5), coupled with ZokumBSP nodebuilding on my map and this version of GZD fixes that MDF bug. There is no need to even use vanilla 1.9 compatibility, the wall clipping bug is gone regardless. Good news! :) Also, when I use the final version of PR/GLBoom Plus 2.5.1.4, coupled with strict vanilla 1.9 compatibility, there's no wall-clipping here either. Though in this program, there's the need of using the 1.9 compatibility option or the wall-clipping will appear. Ideally, the PR/GLB maker should completely remove this bug regardless of compatibility mode, but I guess the available choice of 1.9 strict is good enough. You were saying other ports such as Eternity have this bug fixed as well? Do you know which other ports that have the bug fixed? It could be good to know, if it's not too much trouble, that is. :)

ZokumBSP is my own personal choice for vanilla mapping from now on! Besides this collaboration project I'm on right now, (two maps) I'm going to make some vanilla maps later on for my own project, so I'm going to be using ZBSP a lot. Do you want me to report further findings to you via PM or via your website's development page?

Ok I just had one more question for now. Please let me know if the .cfg mentioned above in the thread, also limits drawsegs as well as compresses visplanes?

Thanks a lot for your help! Really much appreciated! :D Hopefully I will be able to provide helpful information for you as well during my further testing.

Share this post

Link to post

Vanilla format nodes have no way that I know of to compress SEGS. The amount of segs per linedef varies, it depends on the amount of sides that are renderable. So as a starting point a linedef generates 0, 1 or 2 SEGS. 0 Segs when it's a double sided line in the middle of a sector, no texture to render. 1 on a one-sided linedef. And 2 on a two.sided linedef where both sides can possibly be rendered.

The amount of segs then increase as part of the node building of the map. The map is split into sections, and when these splits cross a seg, a new seg is generated for the "other" side of the split. This is algorithm is called binary space partitioning, BSP. A nodebuilder can pick between a lot of possible places to split, causing fewer or more splits. The first splits are often the more cruicial ones. Always going for the least splitting partition line tends to just move the problem further down in the tree, and will also give an unbalanced tree. In the beginning it's often better to have a good even division with roughly equally big parts on each side of the line so that later lines will not split as many lines.

So, if you tune your algorithm for fewer split segs, thus fewer segs, you will most likely have a more unbalanced, slower and deeper tree, with possibly more visplanes. ZokumBSP has tuning parameters that allow you to influence this.

The idea behind the multi-approach i am working on is to try several different division lines and pick the best one. Trying to avoid that "bad" pick in the beginning that led to so many more splits or subsectors later on. Subsectors are related to vis planes. Each visplane is one or more sub sectors.

Share this post

Link to post

Just post it here, it's nice to have a public discussion. Others can chime in and learn / comment / clarify / correct.

Hi again,

I've ran into a possible issue, either with ZokumBSP or it can also be something wrong with my computer, or some feature in Windows 10.

Basically when I run ZBSP 1.09 in Doom Builder 2, every time I try to build nodes I get the message from Windows saying, (roughly translated from swedish) "The publisher couldn't be verified. Do you want to run this program?" Further down it says "Unknown publisher". And that "The program doesn't have a valid digital signature that shows who's the publisher." I've seen this kind of message before, but it hasn't shown up every time I used the program. In addition, in ZBSP 1.0.10, AVG Antivirus Free does a check for "bad stuff" every time I node build. I don't have this issue when using Zennode. What could be causing this?

It should be noted that I do have issues with my computer at startup, there seems to be something's wrong with it. Don't have a clue if this could be causing issues in Windows as well. I don't have any bluescreens or so.

I had a another question about my map as well. I've now gone back to mapping it, it now has a few additional rooms than before. I got the idea to have a check in ZBSP 1.0.10 to see how close I am to the blockmap limit and I got this information after the check.

"Working on (mapfile.wad)

*P11 Old New %Change %Limit Time elapsed

Blockmap: 57062 => 66890 117.22% 102.07% 1.708 secs

Reject: 95.91% => 95.91% 100.00% - 3.642 secs

1 level processed in 8.364 secs - 2 levels needed updating.

Am I now over the blockmap limit of even what ZokumBSP can do? And if there's still room left, is there a way to tell how much sectors/linedefs I can add? The map is soon finished by the way.

UPDATE: Sorry for me being silly about the blockmap stuff. I think I get it now. When I checked the map I used this line in the cmd window: zokumbsp mapfile.wad.

If I do that way, it's without any parameters, more importantly without z=0. Thus in the working version of my map, built with ZBSP and z=0 using Doom Builder 2, I have 57000 in blockmap and 66000 afterwards when ZBSP builds without z=0 from the cmd windows. Correct, right? Luckily as mentioned above, the map is soon finished. :P

Share this post

Link to post

regarding the unknown publisher stuff, maybe it's a windows 10 feature, the \program files\ directories have some additional read/write access rules in win7/8 but I have no experience with 10. I dont experience this issue on 8 though.

Regarding blockmap size, you're not over what zokumbsp can do but what the doom engine can do. I don't think there's a (direct) correlation between linedefs/sectors to blockmap, you'll have to experiment with removing/reorienting stuff to get that last 2.07% under limit. Smarter folks than I will probably have strategy ideas on the most effective sorts of geometry to mess with to get under the vanilla limit.

Share this post

Link to post

regarding the unknown publisher stuff, maybe it's a windows 10 feature, the \program files\ directories have some additional read/write access rules in win7/8 but I have no experience with 10. I dont experience this issue on 8 though.

Regarding blockmap size, you're not over what zokumbsp can do but what the doom engine can do. I don't think there's a (direct) correlation between linedefs/sectors to blockmap, you'll have to experiment with removing/reorienting stuff to get that last 2.07% under limit. Smarter folks than I will probably have strategy ideas on the most effective sorts of geometry to mess with to get under the vanilla limit.

I edited my last post about at the same time as you made yours. ^^ I was actually at around 57000 in blockmap in the version node built with ZokumBSP using the .cfg and the parameter z=0. I only got over 65K because I checked the map in a command window using (zokumbsp mapfile.wad), without the z=0 parameter. So I don't have trouble now. The map is at 63500 in blockmap now but it's just a handful of sectors away from being finished so it won't be a problem with going below 65536.

I still have issues with the unknown publisher stuff though, I read on a webpage that this should be an "issue" with the program in question. But it's strange that the notification doesn't show up on your Windows 8 computer.

Share this post

Link to post

If you're above the limit, and don't want to use z=0 in order to ensure compatibility with buggy source ports. Why not try "-bo=3". The default, o=2 is only trying 36 variations, this will try several thousand. It's not guaranteed to improve anything, but it might. You could also go with subset compression but that might occasionally lead to buggy collission detection due to bugs in doom when moving slowly. The same error does happen with z=1 and I doubt you'll really notice it. I might even heuristically be able to determine if that happens. I might just have to one day look into that :).

There's more info about subset compression in the manual and web site.

Share this post

Link to post

Did you try relocating doombuilder out of the \program files\ directory?

Yeah I've only had Doom Builder inside C:\Games\Doom\Doom Builder 2

12 hours ago, zokum said:

If you're above the limit, and don't want to use z=0 in order to ensure compatibility with buggy source ports. Why not try "-bo=3". The default, o=2 is only trying 36 variations, this will try several thousand. It's not guaranteed to improve anything, but it might. You could also go with subset compression but that might occasionally lead to buggy collission detection due to bugs in doom when moving slowly. The same error does happen with z=1 and I doubt you'll really notice it. I might even heuristically be able to determine if that happens. I might just have to one day look into that :).

There's more info about subset compression in the manual and web site.

Right now I have no problems with the blockmap, it's at 63K and the map is just about finished, architecture-wise. Just have items left to add then. But since you say "bo=3" has better compatibility with buggy source ports, I'll try that out for sure. Up until now I've only tried "-bo=2".

And the "Optimize for vanilla (faster)" section looks the same except for "-bo=2". Should I remove "z=0" completely then? What function does z=0 have if -bo=3 is already present?

Also, do you know what could be causing the issue I'm having, copied from an above post, that every time I build nodes with zokumbsp inside DB2, I get a warning message:

"Basically when I run ZBSP 1.09 in Doom Builder 2, every time I try to build nodes I get the message from Windows 10 saying, (roughly translated from swedish) "The publisher couldn't be verified. Do you want to run this program?" Further down it says "Unknown publisher". And that "The program doesn't have a valid digital signature that shows who's the publisher." I've seen this kind of message before, but it hasn't shown up every time I used the program. In addition, in ZBSP 1.0.10, AVG Antivirus Free does a check for "bad stuff" every time I node build. I don't have this issue when using Zennode. What could be causing this?"

Share this post

Link to post

I'm not sure why Windows does this. It might have to do with a non-signed binary being under the program files umbrella. I keep all of my stuff in c:\games\doom\utils\ etc.

The z=0 removes an aantry from the blockmap lists which shouldn't really be there according to the way the engine works. It's generated by id's own tools and most user tools as a result of the original iwad data having this error.

o=3 just tries to build the blockmap with many more variations. ZenNode used the equivalent of o=0. DoomBSP and BSP uses o=1. Those two will build one blockmap. The default I set, o=2 is to build 36 blockmaps, and use the smallest one. o=3 will build thousands of blockmaps. In many cases these can be smaller than the 36 variations normally tried. o=4 tries all possible 65536 variations. My real world testing has shown to me that I have never found a blockmap that is smaller using this option over o=3. All the setting does is generate blockmaps with different offsets. In some cases the blockmap edges match up in such a manner that you get smaller lists. This usually happens when lines showing up in multiple blocks show up in fewer blocks when the edge of the block is moved.

This is really only useful when dealing with large maps, where there might be several different effects. Some areas of the map might get a smaller blockmap, others a bigger. What counts though is the end result, and I have no idea how to accuratly predict which blockmap offset will line up the data best. There might be some clever tricks one can do, but I haven't come up with a clean way of doing it. As it is now, doing a brute force seems to give decent results at processing time that is ok.

If you're developing a map and need faster build times, using -o=2 and z=0 and a compatible port might do the trick. Later when you want to ensure a high degree of compatibility, you can use a z=1 and o=3 to squeeze it below the limit. I would recommend to simply remove lines that you cannot collide with from the blockmap as optimization #1 if you're near the limit. This will help the -o=n algorithms produce smaller blockmaps.