Friday, April 4, 2014

vogl support for Unreal Engine 4

We're extremely excited that Epic is porting Unreal Engine 4 to Linux -- see the official announcement or some press here and here. Once we heard UE4 Linux was coming we pretty much dropped everything to ensure vogl can handle UE4 callstreams. The latest code on github now supports full-stream tracing/replaying and trimming of UE4 callstreams in either GL3 or GL4 mode. UI support for UE4 is still in the early stages, but now that we can snapshot/restore UE4 and continue to play back the callstream without diverging it's only matter of time before the UI comes up to speed.

UE4's OpenGL renderer is the most advanced we've worked with so far. It has provided us with valuable real-world test cases for several modern GPU features we've not had traces to validate our code against, such as compute shaders and cubemap arrays. We'll be making UE4 GL callstreams part of our regression test suite going forward.

Here are some shots of a trace of UE4's test game being replayed in voglreplay64's --interactive mode (which relies on state snapshotting/restoring):

Here's a trimmed trace loaded in the editor:

Known problems:

UI: Peter Lohrmann just added a dropdown that lets you select which context's state to view. This code is hot off the presses and is a bit fiddly at the moment. Also, UE4 uses several texture formats that the vogl UI can't display right now (LunarG is helping us fix this, see below.)

Snapshotting UE4 during tracing is currently unsupported (but snapshotting during replaying works), because the tracer can't snapshot state while any buffers are mapped. (We also have this problem with the Steam Big Picture renderer.) We have a fix in the works.

We're seeing several query related warnings/errors while snapshotting and replaying UE4 callstreams. (This problem is in vogl's replayer, not UE4.) These need to be investigated, but they don't seem to cause the replayer to diverge.

There are several "zombie" buffer objects that have been deleted on one context but remain bound on another, which causes the snapshot system to report handle remapping errors on these objects during snapshotting. These buffers don't appear to be actually referenced after they are deleted, so this doesn't cause the replay to diverge. We've got some ideas on how to improve vogl's handling of this scenario (which is unfortunately very easy to do by accident in GL).

Other news:

LunarG has provided us with the first drop of their universal OpenGL texture format converter/transformer module, which will be going open source soon. This module allows us to convert any type of OpenGL/KTX texture data to various canonical formats (such as 8-bit or float RGBA) in a driver independent manner, with the optional transforms we need to build a good texture/framebuffer viewer UI. The current vogl UI uses some temporary and very incomplete stand-in code to convert textures to formats Qt accepts, so we're really looking forward to switching to LunarG's solution.

Finally, John McDonald recently joined Valve and the SteamOS team and is currently getting up to speed on the vogl codebase.

6 comments:

We'll definitely be porting to Windows and then OSX, but we're not sure when yet. John is looking into the feasibility of doing a Windows port right now. He was a bit terrified of the codebase's size, and we've been Linux-only for a while, so it might be more work than 1 dev new to the codebase can handle..

About Me

Back in the day I worked for several years at Digital Illusions on things like the first shipping deferred shaded game ("Shrek" - 2001), software renderers, and game AI. Then, after working for Microsoft at Ensemble Studios for 5 years as engine lead on Halo Wars, I took a year off to create "crunch", an advanced DXTc texture compression library. I then worked 5 years at Valve, where I contributed to Portal 2, Dota 2, CS:GO, and the Linux versions of Valve's Source1 games. I was one of the original developers on the Steam Linux team, where I worked with a (somewhat enigmatic) multi-billionare on proving that OpenGL could still hold its own vs. Direct3D. I also started the vogl (Valve's OpenGL debugger) project from scratch, which I worked on for over a year. In my spare time I work on various open source lossless and texture compression projects: crunch, LZHAM, miniz, jpeg-compressor, and picojpeg.