I'm in the process of writing a quick paper on codec choice for a team I work with.

In the process of doing this, I dug into measured performance of H.264 vs HAP vs DXV3.

Now, I know the standard line is that DXV3 and HAP use the GPU to reduce the workload on the CPU, hence allowing more clips to be played, but at the cost of much greater file sizes.

What I'm finding is that H.264 on Macs is, in fact, hardware accelerated and has been since 2008. As such, straight up playback can be very lightweight (and obviously, uses up a lot less storage bandwidth).

The main difference between H.264 and DXV3 is in seeking (ie moving to a different position on a clip timeline). H.264 uses interframe (as well as intraframe) compression, so going to a specific frame means finding the nearest keyframe, and then decoding the intermediate frames until you get to the one you need. HAP and DXV3 just use intraframe compression, so they are quicker to find a frame.

As far as I can tell, Resolume has playback issues with H.264 because it tries to manage playback timing by itself, and is basically seeking to frames constantly, so DXV3 helps here. (As does MJPEG or any other codec that stores complete frames).

We found that in the tradeoff between playback performance and file size, we were starting to be more constrained by the latter. Since many times we are just playing back media, without a lot of seeking back and forth, a more compact codec would work best for us.

I'm wondering if there couldn't be a mode in Resolume where it just relies on the built-in timing in a file, and uses the system hardware acceleration to play H.264. I notice that people are finding that they can play a file in VLC and use Spout/Syphon to feed it to Resolume, with not a huge performance problem. Perhaps we can simulate this by having the option to use system playback features, at the cost of control over the timeline.

Not all hardware acceleration is created equal. There is still a huge performance difference between DXV (and codecs based on the same principle like HAP) and hardware accelerated h264, especially when it comes to higher resolutions and playing back many streams simultaneously.

The problems with continuously seeking were mostly related to micro stuttering and less to performance, and have also been resolved on Resolume 6. If all you need to do is play a single stream of h264 from beginning to end, it's perfectly fine to use an h264 encoded file.

We considered the option of giving control away completely, but it comes with so many unexpected side effects that are hard to anticipate and can completely blindside you if you're not expecting them (aside from bpm playmodes, direction and speed becoming unavailable, autopilot would become unavailable, animation modes like clip position become unavailable, but also loop points become unavailable, transitions become unreliable, audio playback would be completely bypassed by the OS defaults, etc etc) that we don't consider it valid to offer that as an official solution.

If you feel that routing via VLC is giving you better performance, you have all the tools at your disposal and no-one is stopping you from doing that.

I had noticed that H.264 playback in Resolume 6 is definitely far better. I do notice that if I play 4x 4k h.264 clips in Quicktime player vs the same 4x 4k clips in Resolume, the CPU load is higher in Resolume. I imagine this is because a lot more is going on than just playback (compositing, UI, etc).

How does DXV stack up vs HAP? We use a lot of Touch Designer patches as well, and that supports HAP natively. As far as I can tell, both DXV and HAP use the same DXT compression scheme. Resolume 6 works great with HAP, and seems to use hardware acceleration with it.

I'd be thoroughly disappointed in Apple if we'd outperform Quicktime Player on basic h264 playback. That's pretty much its core task and the one thing it needs to get right

DXV vs HAP is potahto - potayto. HAP uses a newer secondary decompressor, which should it make a fraction faster. The main GPU decompressor is extremely similar (suspiciously similar), so we've never bothered to do any A/B'ing. It's basically the same thing.