Project Storky: Automate App And Data Migration Between Regions

While Scalingo is now available in multiple datacenters (called regions) and
as we’re planning to open more regions in the future, our customers felt the
need to have the tools to migrate their data and apps from one region to another.
This is why “Project Storky” was born. We are happy to announce
today that this project has come to fruition: you have now the possibility to
migrate apps from one region to another by yourself.

Storky in Action

Storky is the internal name of the service in charge of handling the
migration of applications and their data between regions. It is a
reference to the stork, the emblem animal
of Scalingo’s hometown region. This animal is known for its ability to deliver
new babies at the new parents’ doorstep (yes, this is how babies are made around
here!). This task requires great care as the package a stork carries is delicate!
This is how we see your applications’ migration: a tough task which requires great
care to handle a package made of many moving pieces.

With Storky’s release, your application migrations are in safe hands (or should
I say “beak”!). Follow us in this guide about how you can concretely migrate
your applications from one region to another.

Migrating an Application

Migrating an application is achieved using Scalingo's command line tool. Please install it before
proceeding.

Migrating an application with our CLI from one region to another is
completely automatic. The actual process is divided into 4 independent steps.
Each of them can be cancelled at any time.

Creation of the migration: where we check that the migration is doable; this
step returns a migration ID that will be required to run the next steps or to
abort the migration.

Preparing the migration: where we copy the skeleton (the app, the
collaborators, its configuration).

Migrating the data: this step is optionnal. All databases are supported except
InfluxDB.

Finalizing the migration: where we start the new app and redirect the traffic
from the old app.

During step 1 and 2 the source application will still work, will still receive
traffic . Only its settings will be frozen to avoid platform interactions that
could affect the migration.

It’s only during step 3 and 4 that the source application is stopped. That means
that the expected and unavoidable downtime for a migration is the duration
of the data backup from the source database and data restoration into the target
database.

import the build that is currently running on the source application (the
latest successful deployment)

import the environment variables

import the collaborators that have accepted the invitation (pending
collaborators are not imported)

import the SCM integration (GitHub, GitLab, etc.)

import the domains and their certificates (both from Let’s Encrypt and
manually added ones)

import the notifiers and the configured alerts

import the log drains

import the autoscalers

Storky can also migrate your data.

Here are the steps followed by Storky for each of your provisioned addons:

provision a database with the same plan and version

trigger a backup on the source database

once those two operations are done, it will restore the backup on the target
database

Shortcomings

While our initial goal with Project Storky was to provide an all-in-one,
completely automatic migration tool, it has a few shortcomings.

Log drains on databases and custom MySQL configuration won’t be migrated. If
you’re concerned, ask the support to enable them on your target application.

If you’re using a git remote repository on Scalingo (i.e. if you’re not using our
GitHub or GitLab integration), this repo won’t be migrated. That’s not really a
problem. You will be able to launch git push on the target repo.

Migration pre-flight checks will raise errors if you use any of the following:
TCP add-on, VPN add-on, Docker add-on or InfluxDB add-on. In that case you can
contact the support for further assistance.

Conclusion

Scalingo’s goal is to automate as many things as possible to ease a
developer’s life. The availability of this tool is a new mandatory, step in
this direction. With Scalingo’s plans to expand in new regions during the year
2020, this tool will be more and more important for our customers.