Appendix A: Updating a Windows Azure Service

This appendix focuses on the scenarios for updating a Windows Azure™ technology platform service that you have previously deployed. It contains the following topics:

• "Options for Updating a Windows Azure Service"
• "Limitations When Updating a Windows Azure Service"
• "Design Considerations for Maximizing Upgradability"
• "About Upgrade Domains and Fault Domains"
• "Using Upgrade Domains for In-Place Upgrades"
• "Using the Windows Azure Service Management API"

Options for Updating a Windows Azure Service

You have three options for updating a deployed Windows Azure service:

• Redeploy the service. Suspend and then delete the service, and then deploy the new version.
• Perform a staged deployment. Upload the new package and swap it with the existing production version. This is referred to as a VIP swap because it simply swaps the visible IP addresses of the services.  
• Perform an in-place upgrade. The new package is uploaded and applied to the running instances of the service. There are options that allow you to update individual instances and individual roles within each instance.

The first of these three options, redeployment, requires downtime for your service, while the second two options allow you to perform updates without incurring service downtime. However, there are limitations on the changes you can make to a deployed service when using the second two options. For more information, see the following section, "Limitations When Updating a Windows Azure Service".

For information about how you can use in-place upgrades, see the sections, "About Upgrade Domains and Fault Domains" and "Using Upgrade Domains for In-Place Upgrades," later in this guidance.

For more information about updating a deployed Windows Azure Service, see "Upgrading a Service" on MSDN® (http://msdn.microsoft.com/en-us/library/ee517254.aspx).

Limitations When Updating a Windows Azure Service

The following are the general rules for choosing the appropriate type of update to perform for a deployed service:

• If you need to change the number or type of endpoints for existing roles, you must delete and redeploy the service.
• If you need to change the service definition, you must perform a staged (VIP swap) deployment (or you can redeploy the service).
• If there is no change to the service definition (.csdef) file, you can perform an in-place update.

The following table shows in more detail the types of changes you can make to a deployed service using the three update options.

Note:

The table here is accurate as of June 2010 however, for up-to-date information, see http://msdn.microsoft.com/en-us/library/ee517255.aspx. When you redeploy a service, you will be allocated new visible IP addresses for the endpoints, which can interrupt access to the service. To reduce the possibility of the need to redeploy your service, ensure that you define all the endpoints you will require for the service, including both HTTP and HTTPS endpoint types for each one.

Updating Only the Configuration of a Service

If you need to edit only the configuration information for the service, and you do not need to deploy new code, you can do this using the Windows Azure Developer Portal or the Windows Azure Service Management API. You can edit the service configuration (.cscfg) file on the portal or upload a new .cscfg file through the portal. However, you must ensure that the uploaded file schema matches the existing service definition. The service will detect and apply the new configuration settings after a preset period.

Redeploying a Service Without Incurring Downtime

If you need to redeploy your service (typically, this occurs when you need to change the number of roles or endpoints it contains), you can change the Domain Name System (DNS) entries at your domain registrar or in your own DNS server that contains the DNS records for your service domain. This allows you to redirect requests to the new version as the DNS change propagates through the world's DNS servers. After one or two days, you can remove the original service.

For example, if your service DNS name is myservice.contoso.com and you map www.contoso.com to this in your DNS, you could create a new service at newservice.contoso.com and then map www.contoso.com to it using a CNAME record. After the DNS change propagates, all requests will go to the new service and you can remove the old service.

Design Considerations for Maximizing Upgradability

Consider the following when you are designing your application to maximize the capabilities for upgrading it, and to make it easier to deploy updated versions:

• If you intend to use in-place upgrades, ensure that state held within the service is replicated or stored outside the roles, so that it is not lost when roles restart after the upgrade process completes. Roles must correctly re-initialize when they restart, and they should not assume that locally held state is available.
• Avoid creating tables within the run-time operations or business logic of a service. Use a separate initialization routine to generate any tables required by the service roles. This avoids performance degradation through exceptions thrown when a table already exists. For sample or demo applications, you can perform the initialization in the OnStart method in the role. However, instead of multiple role instances attempting to initialize the storage at the same time in a production environment, it is better to have a separate console application or script to perform the initialization. When deleting and recreating tables, ensure you wait for approximately one minute for the existing table to be fully removed or an exception will be thrown when creating a new table with the same name.
• Consider setting the IgnoreMissingProperties property to true for the DataContext you use to access tables. By default, this is property is set to false, which means that, when you add new properties to an existing table, existing clients that are not aware of the new property will receive an ADO.NET Data Services exception. When you set IgnoreMissingProperties to true, ADO.NET Data Services will not throw an exception when the client is not aware of the added properties. Although existing clients will not be able to use the added properties, this does allow you to extend the schema of a table and then upgrade clients later. This is particularly important in two specific scenarios:
  • For public services where you cannot be sure that all users will have upgraded versions of the client software
  • When you perform in-place upgrades, so that the roles in upgrade domains that have not yet been upgraded can still communicate with the upgraded roles and tables
• Properties in a table are identified by the combination of the name and the data type, so it is possible to have more than one property with the same name. If you need to change the data type of a property, you must add a new property with the required name and data type, copy the data from the old property to the new property for each row (converting it to the appropriate new type if required), and then delete the old property from the table. Ensure that you set the IgnoreMissingProperties property to true for the DataContext when performing these operations.
• Consider storing a table version number in each row of your tables. This allows newer clients to detect the schema version and take appropriate action, such as storing a default value in an added property when they update or add a row. After all clients are upgraded to understand the new schema, they can be updated again to make full use of the new schema and the added properties; and the row version number can be updated.

About Upgrade Domains and Fault Domains

When you deploy a service to Windows Azure, you specify the number of instances of each role you want to run. The Windows Azure Service Level Agreement guarantees the level of connectivity uptime for your service only if you deploy two or more instances of each role, which allows the service to continue to be available even if a server that hosts one instance of a role fails.

Fault Domains

The infrastructure of each Windows Azure data center is notionally divided into multiple sections known as fault domains. These sections of the infrastructure (which are not necessarily individual servers or server racks) are designed in such a way that a failure of one fault domain is extremely unlikely to affect any other fault domain. When you deploy a service, the Windows Azure Fabric Controller automatically locates the roles in at least two different fault domains so that a failure in one domain will not affect all instances of your service.

Note:
There is no guarantee that your role instances will be located in more than two fault domains, so there is no point in deploying more than two instances of a role unless this is necessary to meet the load anticipated for your service or for other operational reasons.

Upgrade Domains

Whereas a fault domain is a physical location within a data center, an upgrade domain is a logical unit of deployment, although it also affects physical role location. An upgrade domain is primarily a way of logically subdividing your role instances into groups to allow you to more closely control how updates to the instances occur when you perform an in-place upgrade to your service. However, as long as you define more than one upgrade domain (and you have more than one instance of each role), the Windows Azure Fabric Controller will ensure that the role instances within each domain are located in at least two different fault domains.

The main advantage of using upgrade domains and in-place upgrades to a service is that the Windows Azure Fabric Controller will update all the roles within the same upgrade domain as one transacted operation. Only the roles in that upgrade domain will be stopped, upgraded, and restarted. This means that the roles in other upgrade domains will continue to be available and can service requests. After the upgrade to one domain completes and the roles restart, the upgrade process automatically moves to the next upgrade domain.

If you are concerned that the performance of your application will degrade during the upgrade process when the roles in an upgrade domain are stopped, you should consider configuring additional role instances in an additional upgrade domain to maintain capacity.

You can also specify that updates be applied to only a single upgrade domain, and to only specific types of roles within that domain. This allows you, for example, to upgrade just the web roles without stopping and restarting the worker roles (which would result in the loss of any local state they may hold). If you specify only a single upgrade domain for a service, you will not benefit from the capability to maintain availability when performing an in-place upgrade.

You specify the number of upgrade domains for your service using the upgradeDomainCount attribute of the root ServiceDefinition element in the service definition (.csdef) file. The default is five upgrade domains for each service. For information about the ServiceDefinition element, see the relevant section of "Service Definition Schema" on MSDN (http://msdn.microsoft.com/en-us/library/ee758711.aspx#ServiceDefinition).

Within your code, you can query the UpdateDomain property of the RoleInstance class to discover which upgrade domain a role is located within (note the inconsistent naming of this property). The RoleInstance class also exposes the FaultDomain property, but this will only return the values 1 or 2 because a service is only guaranteed to run in two fault domains, even though it may be physically located in more than two.

Using Upgrade Domains for In-Place Upgrades

You can perform either an automatic or a manual upgrade to a service. If you specify an automatic upgrade, the Windows Azure Fabric Controller will automatically process all the roles in each domain. It works through the upgrade domains one by one, stopping, upgrading, and restarting the roles in that upgrade domain before moving on to the next upgrade domain.

If you specify a manual upgrade, the Windows Azure Fabric Controller will process only the roles in the specified upgrade domain. You can then check that the update succeeded and that the service is performing correctly before applying the upgrade to other domains. However, because the load balancer will distribute requests between all running instances of your service, you cannot verify correct behavior just by browsing the service. Instead, you should use instrumentation such as trace messages and performance counters within the service roles to create the appropriate information that identifies the performance of each role instance.

If you want to upgrade only a specific role within each domain, you can specify this role when performing a manual upgrade and it will apply to only that role. This means that you may be able to maintain state if it is held in a role that you do not need to upgrade.

You can initiate an in-place upgrade using the Upgrade button on the Hosted Service page of the Windows Azure portal. You specify the following settings before you start the upgrade:

• Upgrade Mode. Select Automatic to upgrade all domains or Manual to upgrade one domain at a time.
• Package. Select a service package to upload from your local file system or a service package located in BLOB storage.
• Configuration Settings. Select a configuration file to upload from your local file system or a configuration file located in BLOB storage.
• Service Deployment Name. Specify a name for the updated deployment.
• Service Upgrade. Specify if you want to upgrade all the roles within the service or a single role.

Alternatively, you can use third-party tools or the Windows Azure Service Management API to initiate an in-place upgrade. The next section describes the Windows Azure Service Management API.

Using the Windows Azure Service Management API

The Windows Azure Service Management API exposes operations that allow you to perform the tasks available through the Windows Azure portal by writing and executing code. You can use it to manage almost all aspects of your service except for accessing billing data, creating subscriptions, or uploading Management API certificates.

The Service Management API uses Representational State Transfer (REST) protocol and XML. You must submit requests to the core management address using HTTPS. For example, the following request returns a list of hosted services for a specified subscription ID.

https://management.core.windows.net/subscription-id/services/hostedservices

You can also submit XML documents that contain the information required to perform operations, such as creating a deployment, as the body of a posted request. For more information about the available operations, see "Windows Azure Service Management REST API Reference" on MSDN (http://msdn.microsoft.com/en-us/library/ee460799.aspx).

Accessing the Windows Azure Service Management API

You can access the Windows Azure Service Management API using a range of techniques, depending on your own requirements. The following are some examples:

• Use the csmanage.exe command line tool, which makes it easy to generate and send requests to the service using command line parameters.
• Use a custom client library and Windows Communication Foundation (WCF) bindings to send requests to the service management API.
• Use the REST protocol and XML documents in applications that do not run on or are not built using the .NET Framework.
• Use PowerShell scripts to generate and send requests.
• Use it in MSBuild tasks to automate and configure deployments

Note:
Microsoft® Visual Studio® development system uses the Windows Azure Service Management API to deploy and manage Windows Azure services.

For more information about the Windows Azure Service Management API, see "About the Service Management API" on MSDN (http://msdn.microsoft.com/en-us/library/ee460807.aspx).

The csmanage.exe command line tool, together with other useful resources to help you learn and use the Windows Azure Service Management API, is available from "Windows Azure Code Samples" in the MSDN Code Gallery (http://code.msdn.microsoft.com/windowsazuresamples).

A set of PowerShell scripts for accessing the Windows Azure Service Management API can be downloaded from "Windows Azure Service Management CmdLets" in the MSDN Code Gallery (http://code.msdn.microsoft.com/azurecmdlets).

Performing Service Upgrades Using the Service Management API

Unlike the Windows Azure portal, the Windows Azure Service Management API does not allow you to upload a new service package from your local file system. You must first upload the service package to the Windows Azure Blob service, and then specify the BLOB service address of the package when you initiate the upgrade. However, you can automate access to the BLOB service using the Service Management API. For more information, see "About the Storage Service API" on MSDN (http://msdn.microsoft.com/en-us/library/dd573356.aspx).

To upgrade a service using the Windows Azure Service Management API, you first call the Upgrade Deployment operation to initiate the upgrade; specifying the service identifier, the upgrade mode (automatic or manual), the BLOB storage address of the upgrade package, and the name of the role to upgrade (if you are upgrading a single role).

If you specify manual upgrade mode, you must also call the Walk Upgrade Domain operation on each upgrade domain in order, starting from the domain with ID zero, to complete the upgrade. If you call this operation out of order, the API will return an error.

Using the Windows Azure Service Management API to Manage Scaling

One possible use of the Windows Azure Service Management API is the opportunity to react to load on a service by automatically adding or removing role and service instances. For example, you may use a remote application and the Windows Azure Diagnostics mechanism to monitor a range of factors that indicate performance of the service, such as response time, processor load, or operational queue length. The remote application can then use the Windows Azure Service Management API to add or remove instances as the load on the service varies over time. For an example of this technique, see the post, "Auto-scaling in Azure," on the Windows Azure - Cloud computing blog (http://blogs.msdn.com/b/gonzalorc/archive/2010/02/07/auto-scaling-in-azure.aspx).