Upgrading Postgres for MusicBrainz Live Data Feed users

We’re slowly approaching that time of year: Schema change release time. After skipping our fall update to focus on some internal tasks, we’re ready to have another schema change release in the spring: May 16, 2016

We have started the process to collect features we wish to release for this schema change release and we’ll be publishing that list in the coming weeks. However, we’re contemplating the impact of one more change we’d like to make: Upgrading to a more recent version of Postgres.

Internally we are going upgrade to Postgres 9.5, which was recently released, so we expect that the Postgres team will have worked out the most significant kinks before we’re ready to move to it. However, even though we are moving to 9.5, we are considering the impact on our downstream users/customers who need to make the same or similar change.

While we are moving to version 9.5 of Postgres, we have the option of only adopting features from Postgres 9.4, which means that our downstream users may continue to use Postgres 9.4. However, Postgres 9.5 has some nice features we’d like to use (e.g. UPSERT), so we’re pondering if it is possible for us to require Postgres 9.5 from all of ours Live Data Feed users starting on May 16, 2016.

We have already informally queried a few of ours users and so far it seems that requiring Postgres 9.5 is feasible. If you are a Live Data Feed user and feel that this requirement of Postgres 9.5 is too much for your and your organization by May 16, 2016, please leave a comment to this blog post!

Will there be a migration path for existing VM users, rather than having to recreate everything from scratch? Because the current VM required a fair bit of monkey-wrenching to get running, and I don’t look forward to doing that again, not to mention a full database transfer will take most of a day on rust.

@ruaok not looking for Unicorns, but obviously there will be some sort of migration plan for MB infra, so sharing the details of any processes and pitfalls would be appreciated, rather than everyone repeating the fumbling.

I ended up resolving the rest of the problems I encountered by reading the production chef cookbooks and applying most of the differences by hand (the dev/VM cookbooks don’t appear to generate a particularly functional deployment, and it’s not clear to me why the two code-paths exist in the cookbook). Unfortunately I have not recollection of all the specifics at this stage.