Lua is potentially sandboxable, but if, for example, we add file IO support (...) a mod could break out of the sandbox (e.g. by writing a DLL to the OpenMW directory so that next time it is started, some native code is run which could contain an arbitrary payload, or, even more simly, if the script just deleted everything in the documents folder).

Can't speak for Lua specifically on this, but allowing IO support could be done by giving each mod its own virtual storage. Which needn't be specifically an on-disk file or directory, but could be some kind of encoded or compressed database. Besides, even if it could somehow write a DLL to disk, it won't do anything if OpenMW doesn't try to load it. Given OpenMW's cross-platform nature, supporting user-provided DLLs isn't likely in the cards.

Just call it XInput.dll then OpenMW will load it (because SDL will load it) on like 90% of our users' machines. On Windows, programs always search for DLLs in their own directory first, no matter where they would usually be.

I thought Lua was designed to be sandboxed into a program? I am only peripherally familiar with this feature of it though, so correct me if I am wrong. I thought that's why lots of engines used it for scripting.

Lua is like MW's existing scripting in that a Lua script only has available to it the behaviors that the host application specifies. Embedded python etc. is the opposite: their full language is available by default. That's not sandboxing, that's not needing sandboxing. Sandboxing is a second line of defense, a defense against things that are already powerful enough or have enough behaviors given to them that they could do harm if maliciously crafted or maliciously abused.

On Windows, programs always search for DLLs in their own directory first, no matter where they would usually be.

A good reason for the program to be installed into Program Files, where it itself can't write to without admin privileges.

That's been known to cause people absolutely nightmarish headaches with mods whenever you fail to run with admin privileges with all sorts of games. In short, it breaks things left and right. It's actually almost certainly a greater headache for normal, non-programmer users with UAC enabled on Windows than taking reasonable care about viruses in downloaded mods. Everyone can easily run antivirus software. A lot fewer can resolve the problems installing in Program Files brings without fully nullifying all the benefits.

And while it is very apparent Zini feels strongly about this, I get the feeling that if OpenMW is perceived as less powerful in terms of scripting than vanilla + MWSE, a lot of people will just stick with the old style. It'll massively devalue the effort put in. Yeah, we could implement each script extender function and more individually, but even then, with the comment about config files, I think it's pretty much inevitable someone will need a function capable of escaping sandboxing anyway, in addition to it being an absurd amount of increased work and much less valuable to modders. Arbitrary scripting allowances may never happen with the official branch, but I strongly expect it to happen with a fork eventually. And if it does, that'll be the fork everyone uses. Count on it.

That's been known to cause people absolutely nightmarish headaches with mods whenever you fail to run with admin privileges with all sorts of games. In short, it breaks things left and right. It's actually almost certainly a greater headache for normal, non-programmer users with UAC enabled on Windows than taking reasonable care about viruses in downloaded mods. Everyone can easily run antivirus software. A lot fewer can resolve the problems installing in Program Files brings without fully nullifying all the benefits.

The headaches come from crappy game programmers incapable or unwilling to treat user/mod and game data separately. No need to replicate it in OpenMW.

As already mentioned, if file writing/reading from a script is a much requested feature, it could safely be added to mwscript by providing functions that can only access a specific base directory aka vfs. I don't think Zini would object to it.

Imho, if you need a full fledged programming language to implement a feature, just code in C++.

If an application without admin privileges tries to save anything in Program Files (like old games that stored saves in its directory), it is saved in C:\Users\<user>\AppData\Local\VirtualStore\. Its some sort of overlay on original Program Files (and also Windows) directories. It's more like compatibility fallback mechanism not intended to use it widely.
Pointing it just in case as there was something about writing in it.

It's not quite that simple. For example, Skyrim, even when running without admin privileges, can write to its own directory within Program Files as its installer sets the ACL for the directory to allow writes from any user. I imagine Morrowind's installer does the same, so if a user installs OpenMW to their Morrowind directory within Program Files, that protection doesn't exist.

If it turns out that the only non-sandbox-safe operation we need to support is file IO and we can restrict that to each mod's VFS folder, then that particular attack is something we can defend against anyway, so this protection is superfluous anyway.

Let's not forget that on a Mac, there isn't a Program Files directory. Instead, apps are self-contained. Modifying the app itself during runtime is a big no-no. You want an app to save data, it has to be in an external location. Luckily, there are standard places for apps to put their stuff.