How Fast Can you Migrate Email Archives to Office 365?

This is a very frequently asked question for anyone wanting to switch to Microsoft's cloud.

The short answer is: If you’ve chosen the right software for your migration project, your migration ‘engine’ will never be your bottleneck.

These are the top 3 things that will have an impact on the time it takes to move your legacy archives to Office 365:

1 - Your data source

If your archive (e.g. EV) sits on sluggish hardware this might be your bottleneck as you’ll need to be careful not to compromise the performance of your archive service if it’s still in use. Some initial migration tests during the daytime will help you understand what can be tolerated. If your migration solution can access your archive stores directly rather than going via the archive database (API), you can usually get faster results, but this may not always be possible.

If you use compliant storage devices like EMC Centera you’ll be limited by the access nodes available for your migration activity. It’s worth checking whether your migration software is smart enough to migrate from both your primary storage and any replicated devices to double your throughput.

If you have a distributed archive where data needs to come over a slow WAN from multiple sites you could consider bringing a copy of the archive stores locally to your migration hub. This demands the ability to access the archive store directly (as you would typically not want to replicate your archive application servers).

2 – The target

For example, Office 365 has inbuilt throttling, so your archive migration solution should provide the optimal regimes for uploading.

The fastest migrations will use combination of multiple processing tasks, threads, batches and other techniques to optimise migration into Office 365. For example, our projects from EV to Office 365 have seen and range of between 10 and 25 messages per second per migration task, and we can have any number of these. You should expect that the migration can automatically tick over for 16+ hours per day and that you can really ramp up activity out of hours.

In some instances your migration will be limited by your physical network connection to Office 365. We have seen different results in different parts of the world.

To eliminate this as a bottleneck you may consider a 'double hop migration' whereby the raw archived data is shipped to the cloud first (e.g. Microsoft Azure) and the data can then be migrated to Office 365 from there. You might also use the new Microsoft Drive Shipping service, but then you may have a 'wait time' or lag before the data is uploaded by Microsoft. We do not as yet have experience of how long this takes.

Either way, if you're using a 'staged' rather than a direct 'end-to-end' migration you’ll need to ensure your archive is static before you start, so no more archiving. Also you will need to pay close attention to the process of handling shortcuts (making sure they are only removed once you know the corresponding item has been 'rehydrated' into Office 365). Finally you need make sure that your compliance team are happy with the handling of data because your chain of custody just got compromised.

3 – Your Project Plan

Having worked with over 100 clients to migrate their archives, the biggest risk to timescales slipping is your ‘preparation phase’, especially if you’ve got a lot of hoops to jump through internally before getting your project off the ground.

First, your overall Office 365 migration strategy will come into play. For example, unless you are planning a big-bang or ‘cut-over’ migration, it is likely you will move users across in batches over a period of time.

Also, if you are not planning to migrate using a Hybrid approach, you will need to migrate the users’ primary Exchange mailboxes first and then schedule the migration of their archived data to run directly after this.

There will be many ‘political’ and legal issues to consider around how you will ensure any journal (compliance) archives are treated (a subject of a future post) and whether you will take the opportunity to limit the amount of user data you move (e.g. just the last 5 years’). This does not address what you plan to do with leaver’s archives (another subject we plan to cover).

Getting agreement around these matters with the various stakeholders in your organisation can take a long time.

The migration procedure itself tends to be the easy bit and the quickest to get 'in flight'!