Implementing the Migration Plan for Azure
Updated: February 26, 2015
Azure is an internet-scale computing and services platform hosted in Microsoft data centers. With Azure, developers and administrators do not need to implement the underlying software and hardware infrastructure since all the underlying operating system, hardware, network, storage resources, and platform updates are taken care of by Microsoft automatically.
We strongly recommend that after you migrate your application to the cloud, run the functional and performance tests on your application just like you do for any newly deployed application. Since Azure is different than your on-premises platform, you must consider the following important issues when implementing the migration:
Setting up for validation tests
Synchronizing databases to minimize cut over time
Backup and restore
Cut over to Azure
Note that the focus of this topic is primarily the Azure Cloud Services. For preliminary guidance on migration with SQL Server in Azure Virtual Machines, see Getting Ready to Migrate to SQL Server in Azure Virtual Machines.
Authors: Kun Cheng, Steve Howard
Contributors: Selcin Turkarslan
While migrating your applications to the cloud, you must know how to test and debug your application so that you can be sure that it works as expected in the cloud. The following is a list of approaches that you can use to test your application:
Azure Tools for Microsoft Visual Studio: You can build your application and then you can run and debug this application locally using the compute and storage emulators that are provided as part of the Azure Tools. This allows you to develop your application locally before you publish it to Azure. Azure Tools for Microsoft Visual Studio extends Visual Studio 2010 and it enables you to test your application with compute and storage emulators, which provide most of Azure functions. We recommend that you perform this type of testing at the early stages of the functional testing. For more information, see Azure Tools for Microsoft Visual Studio.
SQL Server Data Tools: SQL Server Data Tools (SSDT) offers an integrated environment within Visual Studio 2010 that you can use to design databases, create or edit database objects and data, or execute queries for all supported SQL platforms; including off-premise Azure SQL Database and on-premises Microsoft SQL Server 2012. It enables you to test your database project solutions using either the local default database or Azure SQL Database by examining the relational data access part of the application. For more information, see SQL Server Data Tools. Note: Both Azure Tools for Microsoft Visual Studio and SSDT enable you to perform the basic functionality and compatibility tests on your application with offline and online data sources. In order to test all aspects of a real cloud application in terms of functionality, performance and scalability, you need to do a simulation testing on Azure where the application is deployed and running.
Automated Test Framework: Many existing applications already have an automated test framework that can be used to make sure all the components or functions of the application work as expected. When an application runs on Azure, the automated test framework may or may not work depending on how it is designed. If the automated test framework is required to run from on-premises but can connect to an application on Azure by using the defined end points, it may still work. Otherwise, we recommend that you host both the automated test framework and your application on Azure to mitigate potential connection-losses and latency problems.
Visual Studio Load Testing: If an application does not have an existing automated test framework, we recommend that you create a new automated test framework and use Visual Studio Load Testing to do a simulation testing with multiple concurrent users.
Between testing, staging, and production, you should try to minimize the actual cut over time as much as possible. It may take hours or days to copy a large database from on-premises to Azure. In addition, it is unlikely that you want your application down for the amount of time required to fully migrate the existing data. That’s why you must have a plan to minimize any downtime due to cut over. Note that cut over means the time required to execute the final move to Azure. Prior to cut over, look at your tables and decide which tables contain static data and which tables contain data that might change during cut over. For static data, you do not need to move any data at cut over time. However, if there is any doubt as to whether or not data in a particular table might change during cut over, you should include logic in your system to migrate all changes afterwards.We also recommend that you consider if all data from your on-premises systems need to be migrated to the cloud before your application goes live on Azure. If your application can go live and allow the data to catch up later, you can eliminate any down time.
However, if the data in Azure must be consistent with the data on-premises prior to going live on Azure, consider minimizing the amount of data that must migrate at cut over time as it helps to minimize the downtime required for the actual cut over. In some cases, it might be appropriate to move some of your data prior to cut over, and then to move the remaining data after the actual cut over. In such cases, your migration plan should clearly identify the data that must be migrated first and also the remaining data that can be migrated after cut over. This allows your application to go live on Azure with less downtime as your application can be on production while you migrate the remaining data. You can use the following data synchronization methods to perform the data migration before cut over:
Microsoft Azure SQL Data Sync (Preview) service provides data synchronization capabilities for SQL databases – SQL Server and SQL Database. The service currently has two main capabilities:
Synchronization of data between on-premises SQL Server databases and Microsoft Azure SQL Database instances, allowing on-premises and cloud-based applications to utilize identical data.
Synchronization of data between two or more Microsoft Azure SQL Database instances; the instances can be in the same data center, different data centers or different regions.
Microsoft Azure SQL Data Sync (Preview) is a good option for synchronizing on-premises databases and Microsoft Azure SQL Database instances in the following situations:
You need to do parallel testing of applications.
You need to run your application in parallel prior to the final move of all on-premises data operations to Microsoft Azure.
While migrating to Microsoft Azure, you need to run the application on-premises and at the same time minimal downtime is necessary.
You need to do continuous data synchronization as part of a hybrid solution between on-premises and Microsoft Azure application.
Note that in order to track incremental data changes, when the sync group is configured SQL Data Sync (Preview) adds change tracking table for each table that is being synchronized. When using SQL Data Sync (Preview), be sure your space plans account for the change tracking tables. In addition, you should not make changes to the table structures or primary keys of tables that are being synchronized after the synchronization is set up unless you re-initialize the synchronization group. SQL Data Sync (Preview) is not optimal for situations where intermediate or ongoing data synchronization will be required.
Warning: SQL Data Sync (Preview) is currently available as a Preview and is meant only for product feedback for future releases and should not be used in production environments.
For more information on SQL Data Sync (Preview) see SQL Data Sync.
You can use replication, mirroring, or log shipping to move data from on-premises SQL Server to other on-premises SQL Server or to an instance of SQL Server running in an Azure Virtual Machine. However, you cannot use them for moving data into or out of Azure SQL Database. For more information, see Replication and Log Shipping and Database Mirroring and Log Shipping.
In order to minimize the time needed to transfer data at cut over time, you should move as much data as possible to Azure Platform prior to the actual cut over. You can create a custom ETL job to move the changed data from your on-premises system to your Azure environment. When migrating from on-premises SQL Server 2008 or later, we recommend using Change Data Capture or Change Data Tracking to ensure that all the changed data, and no more than the changed data are actually moved from the on-premises database to the Azure SQL Database instance. For more information on these two features, see Track Data Changes in the SQL Server Books Online.For databases that do not use Change Data Capture or Change Data Tracking, you need to create a tracking system for your changes and the data that has been migrated. In all cases, having the minimal data to move at the time of actual cut over is the key to minimize the down time needed for the actual cut over.
Using DAC, you can export data from a SQL Server instance and place it into Azure Blob storage where it can be imported into an Azure SQL Database. With DAC, you can set up filters on which tables should be imported or exported. But you cannot set up the row level filters. That’s why DAC is appropriate where entire tables fit within a single database but does not work well for scaled out databases. DAC is not optimal for migrating data for applications where ongoing data synchronization will be required. For more information, see Export a Data-tier Application in the SQL Server Books Online.
The purpose of creating database backups is to enable you to recover from data loss caused by administrative errors, application errors, or the total loss of a data center. Backing up and restoring data is different in Azure SQL Database than an on-premise SQL Server and must work with the available resources and tools. Therefore, a reliable use of backup and restore for recovery requires an Azure SQL Database backup and restore strategy. There are three general categories of issues that an Azure SQL Database instance may need to recover from:
Infrastructure or hardware failures: Within a data center, hardware failures might occur. For example, the physical node running your Azure SQL Database instance may crash.
Application or user generated issues or failures: Users or applications can make unwanted changes to data. This may need a revert operation. For example, a user might modify some data that belongs to the wrong customer, and so on.
Loss of data center facilities: The current Azure SQL Database service level agreement specifically has an exception for factors outside Microsoft’s reasonable control, such as disasters. In the event of a disaster, the data center might be damaged in such a way that databases can’t be recovered from the replicas or the on-site backups.
You ultimately need to decide what level of risk you are willing to tolerate with respect to data stored within Azure SQL Database data centers. For detailed information on the available backup and restore tools and how to build disaster recovery strategies around them, see Azure SQL Database Business Continuity article in the MSDN library.
When you perform the actual migration of your application to Azure, you can follow the following approaches:
Run in parallel: With this approach, your application may run in parallel both on on-premises and Azure. This enables you to perform live tests on your application in the Azure before your application becomes fully dependent on the cloud. Your tests should include but not be limited to the following: functionality testing, performance testing, and scalability testing. After you finish testing your new system on the Azure completely, perform the final data migration and do shut down your on-premises system.
Pause and cut over: This approach is appropriate when all data needs to be synchronized prior to going fully live on Azure. Using this approach, all functional and performance testing should be complete on the Azure first. Then, set your system to replicate data to your Azure environment using one of the data synchronization methods specified above. We recommend that you keep the data as close to synchronized as possible by minimizing the amount of time required for the last synchronization or ETL operation prior to final cutover. When it is time to cut over to Azure, bring the on-premises system down, perform the last data synchronization, and bring your application up on Azure.
Other ResourcesMigrating Data-Centric Applications to Azure