Data migrations are simple, right? Take some structured data and move it to a new structure. It's a simple concept, but when you take a closer look there are a lot of potential complications.

How well is the source system understood? If your data wranglers are new to it they will need time or support to get to grips with the underlying data structure. If the system's been around for a long time you can guarantee the older data is not as orderly as the newer, and there’s even less chance of it being well documented. If you need to keep that older data, plan for more effort there.

Have you identified all of your inputs and outputs for the current system, and if they need to be replicated? They will need to be properly included in your test plan. 

How well is the target system understood and is it on the same data platform as the source? Unless you've got some seasoned experts with the new system on tap, then complications are almost inevitable. The longer it takes to find those complications, the harder it will be to mop them up – the pressure will really be on if they are found after migration. Want to add in some data cleansing as you go? Depending on how far you need to take it, you may need some specialist skills added to the mix.

Then you need to decide on whether you're planning a one-and-done migration or a staggered approach. Both have their pros and cons, and it's a big risk to change that decision mid-flight. What's your roll-back plan, and what will it take for you to use it instead of committing to the migration?

Timelines for all of this work are hard to nail. It’s almost impossible to estimate a task like this with precision even before looking at the wider project of onboarding a new system. Hopefully everyone involved has the capacity for their workloads to shift around moving target dates. If there’s no slack planned in, it ratchets up the stress on your staff, a guaranteed killer of efficiency.

That's a lot of complications from just a cursory examination, and a lot of moving parts that could compromise each other. Best make sure that you are testing your migration process thoroughly, and re-testing as you iterate.

We’ve identified a good list of different competences here, and none of them can be neglected. A skilled data wrangler can build and iterate migrations quickly and mine unfamiliar data structures for meaning rapidly, an SME who knows the source or target systems can provide context to a data wrangler and aid building a test plan, a skilled test manager can build an efficient plan to ensure that issues are found early, and so on. Neglect any of those competences though, and you are inviting a compromised result, if not outright failure.

The biggest single risk in a data migration is underestimating the complexity it represents. Make sure all of these competences are properly represented, and don't forget that you're asking a lot of staff who are filling in multiple roles. Migrations are a big one-off purchase – worth investing in properly rather than short-changing, as you're going to be living with your purchase for a long time.