The process of transferring data from one data storage system to another and also between data formats and applications
Data migration is the process of transferring data from one data storage system to another and also between data formats and applications. It also involves data transfers between different data formats and applications.
The data migration process also includes data preparation, extraction, and transformation. It is usually conducted when introducing new systems and processes in an organization.
The following are some common scenarios that require data migration:
There are six types of data migration. A single data migration process can involve different types, including:
Storage migration is where a business migrates data from one storage location to another. It means moving data from one physical medium to another. A common reason for storage migration is the upgrading of storage equipment to more sophisticated modern storage equipment. Hence, it encompasses movement from paper to digital, tapes to hard disk drives (HDD), HDD to solid-state drives, and hardware-based storage to virtual (cloud) based storage.
The movement is not driven by a lack of space but rather a desire to upgrade storage technology. It normally does not alter the content or format of data. During storage migration, certain steps such as data validation, cloning, and data cleaning and redundancy can be carried out.
Databases are data storage media where data is structured in an organized way. Databases are managed through database management systems (DBMS). Hence, database migration involves moving from one DBMS to another or upgrading from the current version of a DBMS to the latest version of the same DBMS. The former is more challenging especially if the source system and the target system use different data structures.
Application migration occurs when an organization goes through a change in application software or changes an application vendor. This migration requires moving data from one computing environment to another. A new application platform may require radical transformation due to new application interactions after the migration.
The major challenge comes from the old and target infrastructures having distinctive data models and using different data formats. Application programming interfaces (APIs) can be provided by vendors to protect data integrity. Vendor web interfaces may be scripted to facilitate data migration.
Cloud migration concerns the movement of data or applications from an on-premises location to the cloud or from one cloud environment to another. It is, in essence, a specific storage migration. IT experts continue to witness an increase in cloud migration and forecast that the majority of major corporations will be operating on the cloud before the end of the decade ending 2030.
Business process migration requires the movement of business applications and data on business processes and metrics to a new environment. The metrics can include customer, product, and operational information. The migration is commonly instigated by business optimization and reorganization and mergers and acquisitions (M&A). Such business combinations are necessitated by the need to enter new markets and remain competitive.
Data center migration relates to the migration of data center infrastructure to a new physical location or the movement of data from the old data center infrastructure to new infrastructure equipment at the same physical location. A data center houses the data storage infrastructure, which maintains the organization’s critical applications. It consists of servers, network routers, switches, computers, storage devices, and related data equipment.
The data migration process should be well planned, seamless, and efficient to ensure it does not go over budget or result in a protracted process. It involves the following steps in the planning, migration, and post-migration phases:
The data migration process can also follow the ETL process:
ETL tools can manage the complexities of the data migration process from processing huge datasets, profiling, and integration of multiple application platforms.
The data migration process remains the same whether a big bang approach or a trickle approach is adopted. A brief overview of the two approaches is given as follows:
The big bang data migration approach moves all data in one single operation from the current environment to the target environment. It is fast and less complex, and also less costly. Its implementation will mean all systems will be down and unavailable for users during the migration. Hence, it should be conducted during public holidays or periods where users are not expected to use the system.
The advantages of the above approach are offset by the risk of an expensive failure due to big data, which can overwhelm the network during transmission. Because of such risk, the big bang approach is more suitable for small companies with smaller amounts of data or for operations or projects where the migration involves a small amount of data. Furthermore, it should not be used on systems that cannot sustain any downtime.
The trickle data migration approach is a phased approach to data migration. Trickle data migration breaks down the migration process into sub-processes where data is transferred in small increments. The old system remains operational and runs parallel with the migration. The advantage is that there is no downtime in the live system, and it is less susceptible to errors and unexpected failures.
However, on the downside, the iterative nature of the process makes it more complex, and it takes longer to complete. During the whole process, data should be synchronized between the old system and the new environment. The trickle migration process is ideal for big data organizations that cannot afford any downtime to their system.
There are some best practices that should be observed when conducting a data migration exercise to ensure the process is seamless with a high degree of success whilst avoiding costly delays.
Data migration risks include the following:
Examples of data migration tools include the following among others:
Thank you for reading CFI’s guide to Data Migration. To keep advancing your career, the additional CFI resources below will be useful: