i had the same problem with the empty "UltraVNC Security Editor" dialogue when using "MSLogonACL.exe" to import security settings for a silent installation on multiple machines.

The solution was to manually create the key "HKEY_LOCAL_MACHINE\SOFTWARE\ORL\WinVNC3" first and importing the security settings with MSLogonACL.exe afterwards.

Obviously MSLogonACL.exe is unable to create this key independently and when it doesn't find it it simply does nothing.
I think this is a relict of former times when UltraVNC stored it's settings in the registry.

Before you can read or write to a key, you must first obtain a handle to it. To do this, use either the RegOpenRegKeyEx or RegCreateKeyEx Microsoft Win32 functions. In practice, you will almost always use RegCreateKeyEx, which will open the key if it exists or create it if it does not. Here is a typical scenario:

I am having the same problem here on Win7 64bit with 1.1.9 x64. Weird thing is I don't have this problem on another same win7 machine. Even after I manually created the registry key HKLM:\Software\ORL\WinVNC3. This is driving me crazy. Any workaround? Any help would be appreciated.