yes you expect the bridged outputs to "fight" each other either current limiiting or blowing up

a way around this would be audio frequency output transformers to turn the bridged drive into (TRS)gnd referenced single ended - a little backwards from usual practice

of course you can get 2 orders of magnitude lower distortion with analog "chip amps" like the TPA6120 into even pretty inefficient headphones - with higher average efficiency too since you'd have much lower quiescient current running the many fewer transistors of the op amp - and no high frequency switching noise to filter out