It turns out that it's rhythmbox's fault. gstreamer grew a richer URI format for cdda:// URI's, and rhythmbox has been using a different format for a long time, which has happened to work, because gstreamer ignored the extra data in there.

From #gstreamer:
12:00:57 < soren> moch: Will that rhythmbox patch from yesterday make it into the upcoming gnome release?
12:08:58 < ~moch> soren: it made it into the release I put out about half an hour ago, yes

...so we should get the Rhythmbox side fixed with the next GNOME sync.