I think the right long term answer to all this is a way to get QEMU to
dump it's current machine configuration in glorious detail as a file
which can be reloaded as a machine configuration.

And then we'll have the same set of problems there.

We will, and the solution will be the same: options to create devices
as they were in older versions of QEMU. It only needs to cover device
features which matter to guests, not every bug fix.
However with a machine configuration which is generated by QEMU,
there's less worry about proliferation of obscure options, compared
with the command line. You don't necessarily have to document every
backward-compatibility option in any detail, you just have to make
sure it's written and read properly, which is much the same thing as
the snapshot code does.

This is a sensible plan, but I don't think we should mix these compat
options in with the VM manager supplied configuration.
There are two problems with that approach.
= Problem 1 - VM manager needs to parse qemu config =
Your proposal implies:
- VM manager supplies a basic configuration to qemu

- It then immediately asks qemu for a dump of the machine
configuration in all its glorious detail and retains that

config

- If the VM manager wishes to add a new device it needs to parse the
qemu config and add it, rather than just generate an entirely new
config

What's the problem with parsing the device config and modifying it? Is
it just complexity?

Yes, complexity is the issue.

If we provided a mechanism to simplify manipulating a device config,
would that eliminate the concern here?

In libvirt's case, a lot of the complexity would come from needing to
figure out what to change.

Right, libvirt wants to be able to easily say "add a scsi block device
to this VM". The way I see this working is that there would be a
default pc.dtc. We would still have a -drive file=foo.img,if=scsi
option that would really just be a wrapper around first searching for an
existing LSI controller, if one exists, attaching the lun, if not,
create one, etc.

libvirt could continue to use this sort of interface. However, as it
wants to do more advanced things, it may have to dive into the device
tree itself.

On live migration, QEMU will save a copy of the device tree somewhere
and libvirt needs to keep track of it. It can treat it as opaque. -M
/path/to/foo.dtc -drive file=foo.img,if=scsi should continue working as
expected IMHO.