A scenario can have each comment block be up to 8192 bytes including the invisible null terminator byte. How many bytes can an .hsc file be? Does it have the same 8 KB file size limit?

I'm adding some string usage types to facilitate data checking and I have these so far:

I am wondering if anyone can help by simply telling me if an .hsc file has a maximum size in bytes that it can be in order to be compiled into a scenario. The reason I am asking is because Blam scripting is the one area of Halo with which I have the very least experience. I have never compiled a working script into a scenario and never used a script that was compiled into a map file.

EmmanuelCDJoined: Jan 7, 2015I hate this comunity as much as hate my existence

You should check the Reclaimer library that mozz relies on. I've put all the hard limits in for all rawdata structs for both regular halo and open sauce(though I admit I somehow typed the max_size for the comment data as 2x larger).

Does the game engine recognize Unix line endings, or are CRLF Windows line endings necessary? I know that Guerilla does not interpret Unix line endings but interprets Windows line endings. If you could convert the line endings with a program such as Notepad++, you would use fewer bytes; this would be helpful if you want it to remain legible and if the Lua syntax that Blam uses does not need CRLF line endings.

Masters1337: Please test the file size of an .hsc script file or let me know how to do it: do I just add the script file to a "scripts" folder in the scenario's tags subdirectory and issue the "compile scripts" command from within Sapien? I'll be testing the limit as 1 << 18, which is 0x00040000, which is 262144 as maximum bytes, and I will be using Lua comments to fill the data.

EDIT: Kirby's scripting tutorial PDF says to use the corresponding subdirectory of data. I'll test this and report back, thanks for the help.Edited by sparky on Apr 18, 2017 at 11:17 AM

256 KB is the maximum .hsc script file size. (262144 bytes). If it is 262145 bytes, you get the error in Sapien, "Maximum source file size exceeded."Edited by sparky on Apr 19, 2017 at 08:46 AM

There are some caveats.

You can have several scripts (sapien and "a hobo" will compile up to 6 .hsc script files in a directory -- it will silently ignore additional script files after the first 6 files), and the total sum bytes of all scripts compiled must be at most 256KB (262144 bytes). The scenario will be able to be opened by Guerilla. Kornman's version of Guerilla will not open a scenario unless every script file used has at most 8192 bytes, because Kornman's version of Guerilla displays the script text in a text box similar to the comment text box and those text boxes only support showing at most 8192 bytes -- so Kornman's version of Guerilla will crash if you try to open a scenario tag that has a script file over 8192 bytes. But it will open fine in the default Guerilla.Edited by sparky on Apr 19, 2017 at 05:16 PM

Line endings don't matter to the syntax (although I haven't tested for situations where it might) and Kornman's version of Guerilla does not show line endings properly unless they are Windows CRLF format. Changing to Unix line endings will save some bytes.Edited by sparky on Apr 19, 2017 at 05:20 PM

---

OK so further testing reveals that MosesOfEgypt was correct: the maximum size of the Comments data in .scenario is 16384 bytes.Otherwise, you get:

tag data 'editor_comment_data_definition' too large.

When you compile using tool.exe.However, an additional limitation is imposed by Guerilla: Guerilla will crash if you try to open any (probably not exclusively .scenario) tag where it needs to load more than 8192 bytes in a text block. Since Kornman's version of Guerilla shows the script information in a text block, that means the same limitation is given for script data with that program.

Maximum Comment Data: 0x4000 (1<<14) (16384) bytes (16KB)

Maximum Script Data for Sum of All Scripts (that Sapien or a hobo will compile): 0x40000 (1<<18) (262144) bytes (256KB)

Maximum Script Files that Sapien or a hobo will recognize per directory: 6

So you can see that the interface imposes artificial limitations on the size of data. Another reason why there needs to be an improvement to the HEK.Edited by sparky on Apr 19, 2017 at 06:43 PM

Also note that Kornman's version of Guerilla "enables" 512 total entries in the Scripts block and 8 total entries in the Source Files block of a scenario tag.Edited by sparky on Apr 19, 2017 at 06:52 PM

String multi-line text fields (for comments in .scenario and -- in Korman's version -- for script text in .scenario) are limited to 8191 characters. Guerilla crashes if additional text is entered. These typically store many more bytes, as previously described.

String multi-line text fields (for .string_list tags) are limited to 30000 characters. Guerilla prevents additional text from being entered. These store up to 4096 bytes; Tool will not compile extra bytes and Guerilla will not save additional bytes.

Related tests:

"tool strings" will compile "ascii/multibyte" strings (ASCII or UTF-8) in this format:

hi this is string A###END-STRING###hi this is string B###END-STRING###

As long as each string entry is at most 8191 characters, which in this case translates to 8192 bytes. Actually, it will say that it compiled properly, but in reality, it will not compile the strings into the tag unless they are each at most 4095 characters, which in this case translates to 4096 bytes.

"tool unicode-strings" will compile "16-bit unicode" strings (UTF-16/UCS-2 with Little Endian or Big Endian Byte Order Markings -- they result in the same little endian byte patterns, essentially as 1-byte characters including the string terminator with 1-byte terminators) in the same format as mentioned -- and will actually compile the string entries into the tag if they are each at most 16383 characters, which in this case translates to 32768 bytes, even though Tool will say that it successfully compiled a string that is at most 32767 characters which would have translated into 65536 bytes. Modifying the data to be more than 32768 bytes results in it being unable to be opened in Guerilla, with the error that the string data is too large.

In sum, .string_list entries can be up to 4096 bytes (4095 characters) and .unicode_string_list entries can be up to 32768 bytes (16383 characters). Tool falsely claims to compile entries twice those lengths: .string_list entries up to 8192 bytes (8191 characters) and .unicode_string_list entries up to 65536 bytes (32767 characters).

...which means that the Reclaimer source code is accurate regarding these limits.Edited by sparky on Apr 19, 2017 at 09:12 PM

You should look at reclaimer for the max reflexive counts as well. I obtained all of my max sizes my either adding entries to guerilla till it capped, or if the limit was too high(around 128) I hex edited a reflexive count/rawdata size in a tag until it gave me an error about the size being too large. The nice thing is that if it decided the number was higher than the limit, it wouldnt even try to parse the rest of the file, so I could just keep tweaking the numbers without having to actually add to the tag. I even got the sbsp limits.

EDIT: https://github.com/LiquidLightning/infernal/commit/e14090e9f0115cf2e4febaeb4656da45af32c27dWhen I do my own tests, I'll let you know if any results differ from yours. Hopefully, there are no substantial inconsistencies among the other HEK programs. It's too bad that Guerilla's multi-line text field is limited to interpreting 8192 bytes and crashes if data is otherwise, even though more data is supported by the game engine. At least this is not the case with string_list tag text blocks but only with scenario tags' comment blocks and (with Kornman's Guerilla) also with script text blocks.

Here is what I have now with string types. I was thinking of doing what most other Halo editors have done, including Kornman with Prometheus, and specifying metadata groups like from/to. Kornman actually released the source code to Prometheus, but I've been waiting for him to say something on the Halo Maps Forum about this. Megasean shared the link on my Halo Discord server; I've been glancing through that to see what others have done in their approaches to reverse engineering. It seems like most of the text is generated from the HEK or game executables or files. I noticed several minor textual mistakes. Anyway...

Since these are new programs that are intended to replace Guerilla, it seems best for us to warn users about things that will prevent compatibility with Guerilla but to only provide limits according to Blam in general.Edited by sparky on Apr 20, 2017 at 09:10 AMEdited by sparky on Apr 20, 2017 at 10:56 AMEdited by sparky on Apr 20, 2017 at 11:02 AM

Do you think that it is worth doing a patch of Kornman's Guerilla, or doing a somewhat-inclusive patch of Guerilla to fix some of these interface problems? Or is that not worth it? It would seem to be a rather simple step for now.

Do you think that it is worth doing a patch of Kornman's Guerilla, or doing a somewhat-inclusive patch of Guerilla to fix some of these interface problems? Or is that not worth it? It would seem to be a rather simple step for now.

I'm not reverting the commit since I've already applied others after it, but I am overwriting the change it with a new commit. Also, I'm not sure it's worth it to tweak Korns Guerilla. I just don't really see a big benefit in doing so when Mozz exists and your editor will too. If you want to fix that stuff, go right ahead. I wouldn't even know where to begin.

Do you think that it is worth doing a patch of Kornman's Guerilla, or doing a somewhat-inclusive patch of Guerilla to fix some of these interface problems? Or is that not worth it? It would seem to be a rather simple step for now.