Almost twice as much overhead is a much tougher pill to swallow. I
would
try to stay with IPv4 as long as I could. Even without adding shim6
into the picture you're taking a significant penalty.

Even standard IP headers are a pretty high overhead for VoIP,
particularly if you're doing very high compression to try to get the
samples to squeeze into a low bandwidth channel. Enter IP header
compression, which is shockingly effective at compressing IP headers of
all sorts... if you've dedicated 128 bits for the address, and it's
still just as static as it was in IPv4, it'll compress to just the same
amount. This is an easy technical problem to solve.