I'm familiar with Bison, I was just saying that you shouldn't be thrown off by: "save it for post 1.0". If you want to do it, go ahead.

That's exactly what I was hoping to hear

What about OBSE?
I think that TES4 implementation somehow should be support it. There are a bunch of mods (~1000 on Nexus) using this script extender.

The guys from silverlock who created OBSE already decoded many classes, greatly reducing the workload. However, there are still many unknown members inside the classes and structs/ classes of rendering/ physics components are completely untouched yet.
That being said, it would be possible to include OBSE right from the beginning, OBSE is catching Oblivions native scripting functions by function hooking. In this case, all the functions, including those non present in vanilla Oblivion, would be part of the games interpreter. OBSE can't be simply copied over though, due to the differences in the internal working of the engine.

As there is no way to properly render Cyrodiil atm, I can't truly verify the functionality of reversed functions. I will simply pretend as if Cyrodiil is already there. Means, for testing purposes, create a cmd line application which allows to test as many as possibly game mechanics and scripting functions without actually rendering anything.

it actually sets the proper values inside the targeted, reverse engineered, class and (as far as possible) calls other functions inside setEssential {...} and shows all the changes in classes in a second window (which can then also be compared with the original Oblivion in reclass)
This wont work perfect, if for example coordinate data is set inside the renderer/ physics engine, but it still makes it easier to reverse and verify game mechanics without actually rendering the game world. It can later be quickly integrated into OpenMW. I simply need a way to verify if the reversed stuff works as intended.

I am further reversing Oblivion scripting engine and related classes. So far, I implemented an unpolished and ugly bytecode interpreter recognizing opcodes of ingame functions (I will eventually add obse) and where possible, set the values in associated, thus far reversed, classes. If I actually want to call a game function I need to pass to the interpreter all the opcodes manually in an array, I have yet to build a parser so one can actually write stuff like "Setscale 3.0" without without writing the opcodes.I don't think there is an open sourced bytecode compiler available for Oblivion, is there ? Means I have to implemend one as well, I am really getting tired of writing opcodes. Worse than that, Bethesda's compiler seems to be directly integrated into the GUI oblivion construction kit, unlike in Skyrim.

Last edited by Deltaxus on 16 Sep 2017, 18:14, edited 2 times in total.

The foundations of the tes4 stack based vm have been implemented.
--> Every vanilla Opcode can be recognized, arithmetic logical instructions, move memory, var inits,... can be executed. Some of the game functions can be executed (They simply change the values of members in associated classes, there are however lots of placeholder functions).
--> Stack trace can be toggled on.

A small Lexer to tokenize tes4 scripts has been implemented.
--> I used the C++ 11 regex capabilities to get it done quickly. It's not fast and eventually needs to be handwritten.

To Do:
- Write the parser component and generate bytecode. The TES4 compiler is integrated into the construction kit, so this is truly annoying. Compiled scripts are not separated files, but part of an .esp or .esm. Shouldn't be hard to dump into an external file though.
- Replace C like Error Codes with C++ try- catch.

Ongoing:
- Reclassing and reversing the game logic.

After I am done with the compiler, I think it's time for an initial commit. I can't truly further proceed without linking the VM to the Engine and integrating the game logic. This part, I can't do on my own. I strongly suppose the engine also needs a major update to make HDR possible.

Future:
- Extend the VM so OBSE stuff can be recognized and executed.
- Determine how the compiler can be integrated into OpenMW and OpenMW-CS.
- Determine how the game logic can be integrated into OpenMW.

A fork may never merge back into the master, so as psi29a suggested, components necessary to run either TES3 or TES4 should be treated like plugins. At some point, I hope cc9cii returns to bring in his expertise with bullet. Another major problem for both Oblivion and Skyrim is facegen. We don't have a replacement, we do however have a potential new group of interested developers once we get to the newer games. Some may not be interested in Morrowind but if we could show some promising initial successes with Oblivion, it should attract new developers.

Atm, there are too many stupid comments, "to do's" and rants in my code . I usually update this thread every 1-2 weeks, so I will polish my code and hopefully also get the parser ready (not guarantee though) and upload it.

Regarding facegen, one possible work around would be to offer presets only, plus an ability to import faces. There's a Wrye Bash function to export the face from a save into an ESP, and presumably the esp face gets added to the preset list.

It doesn't help much for new players, but I expect it wouldn't be long before we saw some compilation mods appearing.