Migrate GitLab CI to GitLab CE or EE

Beginning with version 8.0 of GitLab Community Edition (CE) and Enterprise
Edition (EE), GitLab CI is no longer its own application, but is instead built
into the CE and EE applications.

This guide will detail the process of migrating your CI installation and data
into your GitLab CE or EE installation. You can only migrate CI data from
GitLab CI 8.0 to GitLab 8.0; migrating between other versions (e.g.7.14 to 8.1)
is not possible.

We recommend that you read through the entire migration process in this
document before beginning.

Overview

In this document we assume you have a GitLab server and a GitLab CI server. It
does not matter if these are the same machine.

The migration consists of three parts: updating GitLab and GitLab CI, moving
data, and redirecting traffic.

Please note that CI builds triggered on your GitLab server in the time between
updating to 8.0 and finishing the migration will be lost. Your GitLab server
can be online for most of the procedure; the only GitLab downtime (if any) is
during the upgrade to 8.0. Your CI service will be offline from the moment you
upgrade to 8.0 until you finish the migration procedure.

Before upgrading

If you have GitLab CI installed using omnibus-gitlab packages but you don't want to migrate your existing data:

2. Check source and target database types

Check what databases you use on your GitLab server and your CI server.
Look for the 'adapter:' line. If your CI server and your GitLab server use
the same database adapter no special care is needed. If your CI server uses
MySQL and your GitLab server uses PostgreSQL you need to pass a special option
during the 'Moving data' part. If your CI server uses PostgreSQL and your
GitLab server uses MySQL you cannot migrate your CI data to GitLab 8.0.

3. Storage planning

Decide where to store CI build traces on GitLab server. GitLab CI uses
files on disk to store CI build traces. The default path for these build
traces is /var/opt/gitlab/gitlab-ci/builds (Omnibus) or
/home/git/gitlab/builds (Source). If you are storing your repository data in
a special location, or if you are using NFS, you should make sure that you
store build traces on the same storage as your Git repositories.

I. Upgrading

From this point on, GitLab CI will be unavailable for your end users.

1. Upgrade GitLab to 8.0

2. Disable CI on the GitLab server during the migration

After you update, go to the admin panel and temporarily disable CI. As
an administrator, go to Admin Area -> Settings, and under
Continuous Integration uncheck Disable to prevent CI usage until rake
ci:migrate is run (8.0 only).

3. CI settings are now in GitLab

If you want to use custom CI settings (e.g. change where builds are
stored), please update /etc/gitlab/gitlab.rb (Omnibus) or
/home/git/gitlab/config/gitlab.yml (Source).

4. Upgrade GitLab CI to 8.0

Now upgrade GitLab CI to version 8.0. If you are using Omnibus packages,
this may have already happened when you upgraded GitLab to 8.0.

II. Moving data

1. Database encryption key

Move the database encryption key from your CI server to your GitLab
server. The command below will show you what you need to copy-paste to your
GitLab server. On Omnibus GitLab servers you will have to add a line to
/etc/gitlab/gitlab.rb. On GitLab servers installed from source you will have
to replace the contents of /home/git/gitlab/config/secrets.yml.

2. SQL data and build traces

Create your final CI data export. If you are converting from MySQL to
PostgreSQL, add MYSQL_TO_POSTGRESQL=1 to the end of the rake command. When
the command finishes it will print the path to your data export archive; you
will need this file later.

3. Copy data to the GitLab server

If you were running GitLab and GitLab CI on the same server you can skip this
step.

Copy your CI data archive to your GitLab server. There are many ways to do
this, below we use SSH agent forwarding and 'scp', which will be easy and fast
for most setups. You can also copy the data archive first from the CI server to
your laptop and then from your laptop to the GitLab server.

6. Restart GitLab

III. Redirecting traffic

If you were running GitLab CI with Omnibus packages and you were using the
internal NGINX configuration your CI service should now be available both at
ci.example.com (the old address) and gitlab.example.com/ci. You are done!

If you installed GitLab CI from source we now need to configure a redirect in
NGINX so that existing CI runners can keep using the old CI server address, and
so that existing links to your CI server keep working.

1. Update Nginx configuration

To ensure that your existing CI runners are able to communicate with the
migrated installation, and that existing build triggers still work, you'll need
to update your Nginx configuration to redirect requests for the old locations to
the new ones.