After reading a good chunk of the Opus RFC, I believe it may be possible to use Opus padding to embed lossless deltas for each packet. It states that this padding may be of any size, and that while an Opus encoder itself must set any padding to zero, the decoder must accept any value. It also states that when the decoder has finished reading bytes from a frame for decoding, it may not spill over into the padding for further input. Hrm... Sounds like an arbitrary extension field to me.

My only concern is about just how much padding decoders would be willing to accept before deciding that the packet is a DOS attack.

Nowhere in this thread has it been explained that this isn't possible. NullC has said that RFC6716 encoders are to set any padding to zero. But this isn't going to be an RFC6716 encoder. I aim to write a custom encoder that generates lossless streams that can still be played back by RFC6716 decoders.

Do DTS-ES encoders follow the specs for DTS encoders? Heck no. But they're still compatible.

EDIT: Now with that said, some good points have been raised and there are issues that need to be solved. Bit-exact inaccuracy is the main one.