What is Data Migration?

Data migration is moving data from one computing environment or storage system to another. Organizations collect and store data for analytics. They have to move data between systems for integrated visualization due to technology changes or because they want to move operations to the cloud. Data migration aims to move data efficiently and quickly and avoid disrupting business operations. It includes planning for considerations like network resources, data security, and time and transfer methods. Data migration may also involve storage architecture considerations for missing data values or changing data types.","id":"ams#c1","heading":"What Is Data Migration?","mediaShouldReset":"false"},"metadata":{"tags":[]}},{"fields":{"bodyContent":"

Organizational data resides in physical storage, on-premises servers or virtual servers, single machines, and different applications. Data is also stored in many different formats and types. Data migration becomes necessary to improve the usability of data while increasing efficiency in data management and storage. Cloud data migration unlocks many new benefits for organizations. \n

Implement modern application architecture \n

Moving data to the cloud lays the groundwork for deploying your applications in the cloud. You can modernize legacy systems and rebuild them into cloud-native applications that leverage microservices, containerization, serverless, and other modern designs. Application modernization increases agility and accelerates innovation. \n

Reduce storage costs \n

Migrating to the cloud helps organizations eliminate the high costs of maintaining on-premises infrastructure. With cloud storage, you only pay for what you use and can scale up or down based on demand. It is also highly cost-effective for use cases like data archiving or backup and recovery. \n

Improve data security \n

The cloud includes capabilities like encryption, access controls, logging, monitoring, and automated incident response. Organizations only have to configure cloud security controls and can avoid setting up the entire network infrastructure and security software, which is otherwise necessary to meet data regulations. You can also store sensitive data in specific geographies as required. \n

Enhance analytics \n

Cloud data warehouses allow you to integrate data from multiple sources into a centralized environment. You can quickly set up data quality checks and real-time data ingestion necessary for modern use cases. You also get native access to AI/ML services and can quickly add intelligence to analytics.","id":"ams#c2","heading":"Why is data migration important?","mediaShouldReset":"false"},"metadata":{"tags":[]}},{"fields":{"bodyContent":"

Data migrates from one source system to another target system. The architecture of these systems and the reason behind the data movement determines the type of your data migration project. \n

Storage migration \n

Storage migration involves transfers between data storage systems to improve performance, scalability, or cost-efficiency. A typical example is moving files from an on-premises NTFS-based file server to a cloud storage solution like Amazon S3. Storage migration is often one of the first steps in cloud adoption, as it allows organizations to move large volumes of documents, media files, backups, and logs to the cloud without reworking existing applications. Key benefits include: \n

\n
• Reduced hardware maintenance \n
• Easier access to data from anywhere \n
• Improved redundancy through built-in cloud replication and versioning features. \n \n

  Database migration \n

  Database migration refers to transferring data between database systems. It involves a move from on-premises databases to cloud-based solutions. For example, a company might migrate its SQL Server database from an in-house data center to a cloud relational database service like Amazon RDS. The process typically includes data conversion, replication, and performance tuning in the target environment. \n

  The main drivers for database migration are reliability and cost reduction. Cloud databases offer automated backups, high availability, and the ability to scale compute and storage independently. \n

  Application migration \n

  Application migration involves moving entire software applications and their associated data from an on-premises data center to the cloud. A good example would be migrating a legacy ERP database software to a cloud-based version of the same application or a new SaaS platform. This process may require re-platforming or even rebuilding the application to make use of cloud-native capabilities. \n

  The benefits of application migration include better performance, easier updates, and integration with other cloud services. Organizations can modernize their technology stack and improve user experiences through enhanced speed and availability. \n

  Business process migration \n

  Business process migration transfers the operational activities and workflows that support business functions to the cloud. For instance, a company might move its customer service operations from a mix of manual processes and on-premises CRM tools to a cloud-based CRM solution. This often requires migrating data, logic, and integration points that underpin core business functions. \n

  The motivation behind this type of migration is usually strategic—businesses seek greater agility, faster decision-making, and better customer experiences. By hosting these processes in the cloud environment, companies access real-time data and automate tasks more effectively.","id":"ams#c3","heading":"What are the types of data migration?","mediaShouldReset":"false"},"metadata":{"tags":[]}},{"fields":{"bodyContent":"

  Next, we give some strategies that you can use for data migration. \n

  Lift and shift \n

  Lift and shift is the easiest way to migrate data. You keep the data in the same format, without any transformation, and simply transport it and store it in another location. While it's an effective strategy, it can be less useful for cloud migration. Storing the data in the same format often won’t help effectively capitalize on the benefits of cloud storage. \n

  Use preexisting tools \n

  Many data migration software tools are available to help organizations complete a successful migration. These vendor and open-source tools facilitate enterprise data migrations from a management perspective. \n

  For example, AWS DataSync is an Amazon Web Services (AWS) offering. It helps organizations transfer their on-premises shared file systems, object storage, or Hadoop clusters to AWS cloud storage solutions. \n

  Move all at once or in phases \n

  Depending on the data, you can move everything all at once or shift the data in stages. For example, you can split up a large amount of data and perform chunked data migrations overnight over several weeks. While it’s easiest and fastest to migrate data all at once, sometimes it’s simply not possible. \n

  Enlist specialist help \n

  For complex migrations, it can be wise to enlist the help of outside experts. For successful data migration to AWS, you can choose to connect with one of our AWS Partners.","id":"ams#c4","heading":"What are some data migration strategies?","mediaShouldReset":"false"},"metadata":{"tags":[]}},{"fields":{"bodyContent":"

  A data migration solution requires planning every detail. Here are some factors to consider. \n

  Online or offline data migration \n

  It can be time-consuming and resource-intensive to migrate a considerable amount of data, even with modern networking solutions. Moving data by shipping physical storage devices can be more efficient and economical for some organizations. This strategy is also more secure than sending the data across the wider internet. \n

  Data format \n

  It's usually straightforward to migrate data in the same format from one location to another. For example, migrating databases from an on-site SQL Server to a cloud-based SQL Server requires no format or schema changes. However, you need an intermediary processing step to transform data into a new format before the data center migration. \n

  Operational outage \n

  System downtime or slowdowns are common during migration. You can schedule your migrations during off-peak hours to minimize impact. Many organizations put off data migration as they can't afford to have any system downtime. However, this approach may increase interruption in the future.","id":"ams#c5","heading":"What are the factors to consider before data migration?","mediaShouldReset":"false"},"metadata":{"tags":[]}},{"fields":{"bodyContent":"

  Every organization plans its data migration in ways customized to its requirements. We give a broad overview of steps you can follow to make the entire data migration process more efficient. \n

  Review the source data \n

  Before data migration, you must review and describe the existing data. First, look at the data storage format and its current environment. Following this, where applicable, examine the data in a viewer to determine its structure and attributes. You will need to map the structure to the new data system. \n

  Determine the destination \n

  Once the source data has been examined, choose a fitting destination data storage solution based on the source data’s structure and attributes. Sometimes, you must change the data's structure, attributes, or format to fit the new data storage solution. In the case of data integration, you will need to reorganize the source data to fit the specifications of the destination data. \n

  Outline the data migration strategy \n

  Once you define your needs and destination for data migration, you need a plan to execute it. The data migration plan is the roadmap to a successful migration. \n

  To figure out how the data migration process will work, you should make these determinations: \n

  \n
  • Systems and data migration tools you require \n
  • Security requirements \n
  • Any data transformation processes \n
  • Costs and human resource requirements \n
  • An approximate timeline of the data migration process \n \n

    The data migration strategy should also determine the potential impact of the data migration on users. This includes creating contingency plans for operation or a series of communications to alert users of planned outages. \n

    Implement the technical aspects \n

    Before running the data migration process, you must set up the destination environment, including security and permissions. If practical, create a data migration pipeline as code to provide an automated, reusable solution. You can use the code for future, similar migrations or keep it as a record for documented proof of the process. The codified pipeline serves as a living data migration plan. \n

    Test the solution \n

    Testing is essential to reduce risks associated with the data migration process. The type of testing is dependent on the data and solution. For example, you can choose a smaller chunk of the data to test with, dummy data, or even a copy of the live system data. For data integration, ensure that new test data and existing data match up. \n

    Run the data migration \n

    Once the tests are completed successfully, you can schedule and run the data migration. To troubleshoot in case of unexpected events, ensure the right team is available throughout the process—even if it’s running after hours. \n

    After the data migration, examine the live data in its new environment to check for correctness and ensure that the system works as intended. Once the new system is live and running as expected for a given amount of time, you can safely decommission the old environment.","id":"ams#c6","heading":"What are the steps in data migration?","mediaShouldReset":"false"},"metadata":{"tags":[]}},{"fields":{"bodyContent":"

    Here are some suggestions to make the data migration process more efficient and cost-effective. \n

    Clearly outline the business case \n

    The business case must be clear and warranted for a data migration project to succeed. For example, consider a scenario where users are already running queries on existing business databases. The organization has purchased a new data analysis solution, but only three people have been trained on it so far, with training to be rolled out over a year. The organization could face adverse business outcomes if it attempts a database migration before completing training. \n

    Carefully assess the solution space \n

    A new data solution may require more decision factors than a regular comparative purchasing decision. For instance, when an organization migrates applications to the cloud, they may want to consider containerizing their architectures before they lift and shift. Containerizing would maximize the benefits of cloud infrastructure. The target solutions for these two different strategies are also entirely different. \n

    Read about containerization » \n

    Clean the data \n

    While it’s not always necessary, cleaning the data before migration can be good practice. This includes tasks like deduplication, removing incomplete data, and removing incorrect data. \n

    Fully document the process \n

    Documenting the data migration project supports audit reporting for cases like acquisitions, mergers, and compliance activities. It's also helpful for capturing internal lessons learned and organizational knowledge.","id":"ams#c7","heading":"What are some data migration best practices?","mediaShouldReset":"false"},"metadata":{"tags":[]}},{"fields":{"bodyContent":"

    Given the criticality of data in an organization’s setup, data migration is complex and requires careful risk assessment. We give some common challenges next. \n

    Business continuity \n

    Data migrations should be carried out with as little service disruption as possible. When it isn’t possible to avoid downtimes or slowdowns, plan migration outside of regular business hours. Give users plenty of warning through channels like emails, in-application notifications, and pinned social media posts. \n

    Migration costs \n

    The tools, human resources, new data infrastructure, and cost of decommissioning old data infrastructure all add up when transferring data. Make sure you budget for all aspects before starting the process. It's also important to factor in any costs due to loss of productivity or revenue during application downtime. To keep migration impact costs to a minimum, try to limit outages and ensure all impacted users are aware of the migration in advance. \n

    Data security \n

    Keeping data secure both in transit and its new environment requires careful planning. You should perform complex encryption before transit and create virtual private networks for the transfer process. Thoroughly test and assess the security rules and permissions of the new environment before migration. \n

    New system failures and faults \n

    It's challenging to ensure the success of data migration for all scenarios. Sometimes, transferring data may fail or produce unexpected results. In the event of faults and failures, you need a contingency plan. Always have backups so that it’s possible to roll back to the old data systems if you need to.","id":"ams#c8","heading":"What are some data migration challenges?","mediaShouldReset":"false"},"metadata":{"tags":[]}},{"fields":{"bodyContent":"

    Amazon Web Services (AWS) provides an extensive range of solutions to help you in cloud data migration. We help you find and secure the right services and resources to match your requirements, as well as assist with running the process itself. \n

    For example, you can use these data migration services: \n

    \n
    • AWS DataSync to securely discover data and migrate to AWS with end-to-end security, simplified planning, and data movement management.  \n
    • AWS Direct Connect to create a dedicated network connection to AWS. This way, you can secure your data as it moves between your network and AWS with multiple encryption options. \n
    • Amazon Data Firehose to stream data. You can reliably load real-time streams into data lakes, warehouses, and analytics services. \n
    • AWS Snowcone to deploy edge computing devices. Snowcone devices are small, rugged, and secure. They offer edge computing, data storage, and physical data transfer on the go. They're good options in austere environments with little or no connectivity. \n
    • AWS Transfer Family to easily manage file transfers. You can also modernize your transfer workflows to Amazon Simple Storage Service (Amazon S3) or Amazon Elastic File System (Amazon EFS). You do this within hours and with your existing authentication systems. \n \n

      Get started with data migration on AWS by creating an account today.","id":"ams#c9","heading":"How can AWS help with your data migration requirements?","mediaShouldReset":"false"},"metadata":{"tags":[]}},{"fields":{"id":"ams#rt-grouped-questions406141665#pattern-data","patternDark":"false"},"metadata":{"tags":[{"name":"pattern-data","description":"Default pattern data","id":"ams#rt-grouped-questions406141665#pattern-data","namespaceId":"rt-grouped-questions"}]}}]},"metadata":{"auth":{},"testAttributes":{}},"context":{"page":{"pageUrl":"https://aws.amazon.com/what-is/data-migration/"},"contentType":"page","environment":{"stage":"prod","region":"us-west-2"},"sdkVersion":"2.0.26"},"refMap":{"manifest.js":"269d4ac6b1","rt-grouped-questions.rtl.css":"6041e37f19","rt-grouped-questions.js":"76731b0c78","rt-grouped-questions.css.js":"1d375453ec","rt-grouped-questions.css":"ec816aa39b","rt-grouped-questions.rtl.css.js":"590781e0f0"},"settings":{"templateMappings":{"patternDark":"patternDark","id":"id","heading":"heading","answerSummary":"answerSummary","title":"title","bodyContent":"bodyContent","hyperlinkText":"hyperlinkText","hyperlinkUrl":"hyperlinkUrl","mediaUrl":"mediaUrl","mediaAltText":"mediaAltText","mediaShouldReset":"mediaShouldReset","videoOverlayTitle":"videoOverlayTitle","videoThumbnailUrl":"videoThumbnailUrl"}}}

      Page topics

      • What Is Data Migration?
      • Why is data migration important?
      • What are the types of data migration?
      • What are some data migration strategies?
      • What are the factors to consider before data migration?
      • What are the steps in data migration?
      • What are some data migration best practices?
      • What are some data migration challenges?
      • How can AWS help with your data migration requirements?

      What Is Data Migration?

      Data migration is moving data from one computing environment or storage system to another. Organizations collect and store data for analytics. They have to move data between systems for integrated visualization due to technology changes or because they want to move operations to the cloud. Data migration aims to move data efficiently and quickly and avoid disrupting business operations. It includes planning for considerations like network resources, data security, and time and transfer methods. Data migration may also involve storage architecture considerations for missing data values or changing data types.

      Why is data migration important?

      Organizational data resides in physical storage, on-premises servers or virtual servers, single machines, and different applications. Data is also stored in many different formats and types. Data migration becomes necessary to improve the usability of data while increasing efficiency in data management and storage. Cloud data migration unlocks many new benefits for organizations.

      Implement modern application architecture

      Moving data to the cloud lays the groundwork for deploying your applications in the cloud. You can modernize legacy systems and rebuild them into cloud-native applications that leverage microservices, containerization, serverless, and other modern designs. Application modernization increases agility and accelerates innovation.

      Reduce storage costs

      Migrating to the cloud helps organizations eliminate the high costs of maintaining on-premises infrastructure. With cloud storage, you only pay for what you use and can scale up or down based on demand. It is also highly cost-effective for use cases like data archiving or backup and recovery.

      Improve data security

      The cloud includes capabilities like encryption, access controls, logging, monitoring, and automated incident response. Organizations only have to configure cloud security controls and can avoid setting up the entire network infrastructure and security software, which is otherwise necessary to meet data regulations. You can also store sensitive data in specific geographies as required.

      Enhance analytics

      Cloud data warehouses allow you to integrate data from multiple sources into a centralized environment. You can quickly set up data quality checks and real-time data ingestion necessary for modern use cases. You also get native access to AI/ML services and can quickly add intelligence to analytics.

      What are the types of data migration?

      Data migrates from one source system to another target system. The architecture of these systems and the reason behind the data movement determines the type of your data migration project.

      Storage migration

      Storage migration involves transfers between data storage systems to improve performance, scalability, or cost-efficiency. A typical example is moving files from an on-premises NTFS-based file server to a cloud storage solution like Amazon S3. Storage migration is often one of the first steps in cloud adoption, as it allows organizations to move large volumes of documents, media files, backups, and logs to the cloud without reworking existing applications. Key benefits include:

      • Reduced hardware maintenance
      • Easier access to data from anywhere
      • Improved redundancy through built-in cloud replication and versioning features.

      Database migration

      Database migration refers to transferring data between database systems. It involves a move from on-premises databases to cloud-based solutions. For example, a company might migrate its SQL Server database from an in-house data center to a cloud relational database service like Amazon RDS. The process typically includes data conversion, replication, and performance tuning in the target environment.

      The main drivers for database migration are reliability and cost reduction. Cloud databases offer automated backups, high availability, and the ability to scale compute and storage independently.

      Application migration

      Application migration involves moving entire software applications and their associated data from an on-premises data center to the cloud. A good example would be migrating a legacy ERP database software to a cloud-based version of the same application or a new SaaS platform. This process may require re-platforming or even rebuilding the application to make use of cloud-native capabilities.

      The benefits of application migration include better performance, easier updates, and integration with other cloud services. Organizations can modernize their technology stack and improve user experiences through enhanced speed and availability.

      Business process migration

      Business process migration transfers the operational activities and workflows that support business functions to the cloud. For instance, a company might move its customer service operations from a mix of manual processes and on-premises CRM tools to a cloud-based CRM solution. This often requires migrating data, logic, and integration points that underpin core business functions.

      The motivation behind this type of migration is usually strategic—businesses seek greater agility, faster decision-making, and better customer experiences. By hosting these processes in the cloud environment, companies access real-time data and automate tasks more effectively.

      What are some data migration strategies?

      Next, we give some strategies that you can use for data migration.

      Lift and shift

      Lift and shift is the easiest way to migrate data. You keep the data in the same format, without any transformation, and simply transport it and store it in another location. While it's an effective strategy, it can be less useful for cloud migration. Storing the data in the same format often won’t help effectively capitalize on the benefits of cloud storage.

      Use preexisting tools

      Many data migration software tools are available to help organizations complete a successful migration. These vendor and open-source tools facilitate enterprise data migrations from a management perspective.

      For example, AWS DataSync is an Amazon Web Services (AWS) offering. It helps organizations transfer their on-premises shared file systems, object storage, or Hadoop clusters to AWS cloud storage solutions.

      Move all at once or in phases

      Depending on the data, you can move everything all at once or shift the data in stages. For example, you can split up a large amount of data and perform chunked data migrations overnight over several weeks. While it’s easiest and fastest to migrate data all at once, sometimes it’s simply not possible.

      Enlist specialist help

      For complex migrations, it can be wise to enlist the help of outside experts. For successful data migration to AWS, you can choose to connect with one of our AWS Partners.

      What are the factors to consider before data migration?

      A data migration solution requires planning every detail. Here are some factors to consider.

      Online or offline data migration

      It can be time-consuming and resource-intensive to migrate a considerable amount of data, even with modern networking solutions. Moving data by shipping physical storage devices can be more efficient and economical for some organizations. This strategy is also more secure than sending the data across the wider internet.

      Data format

      It's usually straightforward to migrate data in the same format from one location to another. For example, migrating databases from an on-site SQL Server to a cloud-based SQL Server requires no format or schema changes. However, you need an intermediary processing step to transform data into a new format before the data center migration.

      Operational outage

      System downtime or slowdowns are common during migration. You can schedule your migrations during off-peak hours to minimize impact. Many organizations put off data migration as they can't afford to have any system downtime. However, this approach may increase interruption in the future.

      What are the steps in data migration?

      Every organization plans its data migration in ways customized to its requirements. We give a broad overview of steps you can follow to make the entire data migration process more efficient.

      Review the source data

      Before data migration, you must review and describe the existing data. First, look at the data storage format and its current environment. Following this, where applicable, examine the data in a viewer to determine its structure and attributes. You will need to map the structure to the new data system.

      Determine the destination

      Once the source data has been examined, choose a fitting destination data storage solution based on the source data’s structure and attributes. Sometimes, you must change the data's structure, attributes, or format to fit the new data storage solution. In the case of data integration, you will need to reorganize the source data to fit the specifications of the destination data.

      Outline the data migration strategy

      Once you define your needs and destination for data migration, you need a plan to execute it. The data migration plan is the roadmap to a successful migration.

      To figure out how the data migration process will work, you should make these determinations:

      • Systems and data migration tools you require
      • Security requirements
      • Any data transformation processes
      • Costs and human resource requirements
      • An approximate timeline of the data migration process

      The data migration strategy should also determine the potential impact of the data migration on users. This includes creating contingency plans for operation or a series of communications to alert users of planned outages.

      Implement the technical aspects

      Before running the data migration process, you must set up the destination environment, including security and permissions. If practical, create a data migration pipeline as code to provide an automated, reusable solution. You can use the code for future, similar migrations or keep it as a record for documented proof of the process. The codified pipeline serves as a living data migration plan.

      Test the solution

      Testing is essential to reduce risks associated with the data migration process. The type of testing is dependent on the data and solution. For example, you can choose a smaller chunk of the data to test with, dummy data, or even a copy of the live system data. For data integration, ensure that new test data and existing data match up.

      Run the data migration

      Once the tests are completed successfully, you can schedule and run the data migration. To troubleshoot in case of unexpected events, ensure the right team is available throughout the process—even if it’s running after hours.

      After the data migration, examine the live data in its new environment to check for correctness and ensure that the system works as intended. Once the new system is live and running as expected for a given amount of time, you can safely decommission the old environment.

      What are some data migration best practices?

      Here are some suggestions to make the data migration process more efficient and cost-effective.

      Clearly outline the business case

      The business case must be clear and warranted for a data migration project to succeed. For example, consider a scenario where users are already running queries on existing business databases. The organization has purchased a new data analysis solution, but only three people have been trained on it so far, with training to be rolled out over a year. The organization could face adverse business outcomes if it attempts a database migration before completing training.

      Carefully assess the solution space

      A new data solution may require more decision factors than a regular comparative purchasing decision. For instance, when an organization migrates applications to the cloud, they may want to consider containerizing their architectures before they lift and shift. Containerizing would maximize the benefits of cloud infrastructure. The target solutions for these two different strategies are also entirely different.

      Read about containerization »

      Clean the data

      While it’s not always necessary, cleaning the data before migration can be good practice. This includes tasks like deduplication, removing incomplete data, and removing incorrect data.

      Fully document the process

      Documenting the data migration project supports audit reporting for cases like acquisitions, mergers, and compliance activities. It's also helpful for capturing internal lessons learned and organizational knowledge.

      What are some data migration challenges?

      Given the criticality of data in an organization’s setup, data migration is complex and requires careful risk assessment. We give some common challenges next.

      Business continuity

      Data migrations should be carried out with as little service disruption as possible. When it isn’t possible to avoid downtimes or slowdowns, plan migration outside of regular business hours. Give users plenty of warning through channels like emails, in-application notifications, and pinned social media posts.

      Migration costs

      The tools, human resources, new data infrastructure, and cost of decommissioning old data infrastructure all add up when transferring data. Make sure you budget for all aspects before starting the process. It's also important to factor in any costs due to loss of productivity or revenue during application downtime. To keep migration impact costs to a minimum, try to limit outages and ensure all impacted users are aware of the migration in advance.

      Data security

      Keeping data secure both in transit and its new environment requires careful planning. You should perform complex encryption before transit and create virtual private networks for the transfer process. Thoroughly test and assess the security rules and permissions of the new environment before migration.

      New system failures and faults

      It's challenging to ensure the success of data migration for all scenarios. Sometimes, transferring data may fail or produce unexpected results. In the event of faults and failures, you need a contingency plan. Always have backups so that it’s possible to roll back to the old data systems if you need to.

      How can AWS help with your data migration requirements?

      Amazon Web Services (AWS) provides an extensive range of solutions to help you in cloud data migration. We help you find and secure the right services and resources to match your requirements, as well as assist with running the process itself.

      For example, you can use these data migration services:

      • AWS DataSync to securely discover data and migrate to AWS with end-to-end security, simplified planning, and data movement management. 
      • AWS Direct Connect to create a dedicated network connection to AWS. This way, you can secure your data as it moves between your network and AWS with multiple encryption options.
      • Amazon Data Firehose to stream data. You can reliably load real-time streams into data lakes, warehouses, and analytics services.
      • AWS Snowcone to deploy edge computing devices. Snowcone devices are small, rugged, and secure. They offer edge computing, data storage, and physical data transfer on the go. They're good options in austere environments with little or no connectivity.
      • AWS Transfer Family to easily manage file transfers. You can also modernize your transfer workflows to Amazon Simple Storage Service (Amazon S3) or Amazon Elastic File System (Amazon EFS). You do this within hours and with your existing authentication systems.

      Get started with data migration on AWS by creating an account today.