February 2012

Volume 27 Number 02

Microsoft Azure - Building a Massively Scalable Platform for Consumer Devices on Microsoft Azure

By Bruno Terkaly, Ricardo Villalobos | February 2012

This article is about scalability and interoperability, two characteristics that are required in architectures to support the diversity of today’s popular mobile platforms, which potentially have millions of users. Figure 1 depicts this diversity, a common—yet challenging—scenario for today’s developers. Supplying Web-based services to mobile devices is a daunting task, requiring distinct and diverse tooling, languages and IDEs. Beyond this diversity is the need for elastic scale—in terms of available Web services and for data that can reach terabytes in size.

Figure 1 A Diverse Set of Mobile Technologies Is a Challenge for Developers

Developers need to scale their Web applications in two different dimensions. The first dimension is compute, which simply boils down to the number of Web service instances made available by the hosting provider to respond to mobile Web requests. The second dimension is scalable data: Some cloud platforms offer scalable data through dedicated storage services, letting developers scale terabytes of data to millions of mobile users and effortlessly partition it across multiple servers, resulting in fast performance, redundancy and support for petabytes of capacity.

To support communication to as many diverse clients as possible, an interoperable approach is crucial. Everything from data formats to network protocols needs careful consideration. A solution must minimize custom coding and leverage open standards to the greatest extent possible.

We’re using RESTful Web services hosted in Azure—the Microsoft cloud platform—in this article to solve both the interoperability challenges and the elastic scale problem.

A reference architecture based on RESTful Web services is depicted in Figure 2. RESTful architectures are interoperable because they’re developed alongside HTTP/1.x and provide consistent communication across a vast array of clients. The architecture alternative to REST is SOAP. We chose not to use SOAP because it has larger, slower data payloads and additional complexities.

Figure 2 A Solution Based on Open Standards

Azure makes it easy to increase and decrease scale on demand. By simply changing a number—the “Instance Count”—through either the Azure Portal or a management API, you can scale RESTful Web services almost effortlessly to meet any level of demand.

Our implementation uses JSON (and not XML) as the data format because it’s compact and widely supported. XML suffers from larger payloads.

Although many vendors offer cloud hosting solutions for RESTful Web services, Azure has some advantages. For starters, you can choose from among six highly automated datacenters in Asia, Europe and North America, including support from 24 Content Delivery Networks (CDNs), making it possible to connect to users with low latency and data locality.

Azure offers an array of storage and computing options in addition to powerful developer tooling. A variety of storage mechanisms are available, from Binary Large Objects (BLOBs) to relational stores. Azure also provides identity management systems, secure messaging and hybrid cloud/on-premises connectivity capabilities.

Getting Started

The remainder of this article will divide the architecture and implementation into four parts:

1. Provision an account using the Azure Portal.
2. Create a Azure Cloud Project and write some code to define a RESTful Web service.
3. Deploy the cloud project to the account using the Azure Portal.
4. Build mobile applications for: Windows Phone, Android and iOS (iPhone/iPad).

Let’s go through these steps together. The first one takes place at the Azure Portal, which you can access at windows.azure.com if you have a subscription. (For more information, visit azure.com.)

Part 1: Provisioning the Web Service at the Azure Portal

The two key options at the Azure Portal are: New Hosted Service and New Storage Account.

Figure 3illustrates the workflow for provisioning a “hosted service.” This process will result in a URL that represents the endpoint in a Microsoft datacenter where the RESTful Web service will be deployed. Developers of Windows Phone, Android and iOS apps will need this URL to communicate with the service.

Figure 3 Provisioning the Azure RESTful Web Service

The workflow to get all this working is straightforward:

1. Log in to the Azure Portal.
2. Select “New Hosted Service.” Specify an account name, a URL and a region (location of datacenter).
3. Store the URL the Azure Portal generates; it will be used—along with the account name—when building both the RESTful Web service and the mobile clients. The account name will also be used in Part 3.

Note: The example in this article uses the account name “fastmotorcycleservice,” with the URL “https://fastmotorcycleservice.cloudapp.net.”

The second task at the Azure Portal is creating a Storage Account. Figure 4 illustrates this process, including the name and location of the Azure tables. Once again, it’s possible to choose from among the six datacenters. It makes sense to host both the Web service and the data in the same datacenter to reduce cost and improve performance.

Figure 4 Provisioning the Azure Storage Account

The workflow is similar to the “hosted service” explained previously:

1. Log in to the Azure Portal.
2. Create a new storage account and provide an account name and a region.
3. Store the access key the Azure Portal generates and provides, as well as the account name; they’ll be required when building the RESTful Web service.

Now that Part 1 is complete, the needed Azure Portal information can be used to write the RESTful Web service as well as the Windows Phone, Android and iOS applications.

Part 2: Building the Azure-Hosted RESTful Web Service

Building a RESTful Web service in Visual Studio is simple. Open Visual Studio as administrator from Start | All Programs | Microsoft Visual Studio 2010 by right-clicking the Microsoft Visual Studio 2010 shortcut and choosing “Run as administrator.” From the File menu, choose New | Project.

In the New Project dialog, expand the language of preference in the Installed Templates list and select Cloud. Choose the Azure Project template, set the name of the project to FastMotorcycleProject and set the location to anything convenient.

A video demonstrating these steps in detail can be found at bit.ly/VideoAzureRestfulService.

The Solution Explorer will look like Figure 5.

Figure 5 Creating a New Azure Project

Figure 6shows some basic steps not covered in this article (but which are covered in the referenced video).

Figure 6 Basic Steps Not Covered in This Article

These steps are common to almost all Azure projects. For example, it’s standard practice to use a Web Role to host RESTful Web services. A DataConnectionString is needed to access the storage account defined previously at the Azure Portal. Startup code is needed inside the Visual Studio project to read account names and access keys from the configuration files to use against the storage accounts.

Once the preliminary steps are complete, a RESTful Web service can be added using the WCF Service template in Visual Studio.

To add a WCF Service, right-click the FastMotorcycleProject_WebRole folder, select Add | New Item dialog and set the name of the class to FastMotorcycleService.

FastMotorcycleService.svc.cs will be generated. Replace the entire code of the class with the code shown in Figure 7.

The key to making this work is to know how to map different URIs and verbs to RESTful methods. For this, the WebGet and WebInvoke attributes must be added to the code in Figure 7.

Figure 7 FastMotorcycleListService.svc.cs

[ServiceContract]
public class FastMotorcycleListService
{
  private FastMotorcycleListDataProvider _data;
  
  public FastMotorcycleListService()
  {
    _data = new FastMotorcycleListDataProvider();
  }

  [OperationContract]
  [WebGet(UriTemplate = "/list/{owner}", ResponseFormat = 
    WebMessageFormat.Json)]
  public List<string> GetItems(string owner)
  {
    return _data.GetItems(owner);
  }

  [OperationContract]
  [WebInvoke(UriTemplate = "/list/{owner}", Method = "POST",
    RequestFormat = WebMessageFormat.Json)]
  public void AddItem(string owner, string item)
  { 
    _data.AddItem(owner, item);
  }

  [OperationContract]
  [WebInvoke(UriTemplate = "/list/{owner}/{item}", Method = "DELETE")]
  public void DeleteItem(string owner, string item)
  {
    _data.DeleteItem(owner, item);
  }
}

These attributes tell the framework that the method should respond to HTTP GET requests. WebInvoke is mapped to HTTP POST by default. Also by default, the URI is determined by the name of the method (added onto the base URI of the endpoint). Some experts or REST purists might argue that our method names should not be verbs but rather nouns.

The WCF REST programming model shown in Figure 8 allows customization of URIs for each method by using templates that can be set via the UriTemplate property on the WebInvoke and WebGet attributes. The model is explained in the following list, with numerals corresponding to those in Figure 8:

Figure 8 Workflow for Mobile Application Requesting RESTful Data

1. A mobile application uses standard HTTP to send a message request, which includes an HTTP verb plus a URL.
2. The RESTful Web service intercepts the mobile application message request (request for data) and makes a call to GetItems, passing “Bruno” as a parameter. GetItems queries for data using a LINQ query, using “Bruno” as part of the where clause.
3. Only the records in which the PartitionKey is equal to “Bruno” are returned from the Azure Table Service.
4. The data is converted to JSON format (automatically) and returned to the mobile device.
5. The data is available to the mobile application. The data is used to populate a ListBox and presented to the mobile application user.

The next three classes we discuss are helper objects, which are needed to interact with the Azure Table Service. Fast­MotorcycleListDataProvider, FastMotorcycleListItem and FastMotorcycleList are classes that abstract away storage and Azure Table-specific API details from the code in Figure 9, allowing the application to perform Create, Read, Update and Delete (CRUD) operations with the Azure Table Service.

In Visual Studio, add a new class module called FastMotorcycleListDataProvider.cs. Replace the code with the code in Figure 9.

Figure 9 The FastMotorcycleListDataProvider, FastMotorcycleListItem and FastMotorcycleList Classes

public class FastMotorcycleListDataProvider
{
  private FastMotorcycleList _list;

  public FastMotorcycleListDataProvider()
  {
    string configValue = RoleEnvironment.GetConfigurationSettingValue(
      "DataConnectionString");
    var account = CloudStorageAccount.Parse(configValue);

    _list = new FastMotorcycleList(account.TableEndpoint.ToString(),
                                   account.Credentials);
  }
 
  public List<string> GetItems(string owner)
  {
    var results = from entity in _list.Items
                  where entity.PartitionKey == owner
                  select entity;

    var list = new List<string>();
    foreach (var item in results)
    {
      list.Add(item.RowKey);
    }

    return list;
  }
 
  public void AddItem(string owner, string item)
  {
    _list.AddObject("FastBikes", new FastMotorcycleListItem(owner, item));
    _list.SaveChanges();
  }

  public void DeleteItem(string owner, string item)
  {
    var entity = (from i in _list.Items
                  where i.PartitionKey == owner
                  && i.RowKey == item
                  select i).Single();

    _list.DeleteObject(entity);
    _list.SaveChanges();
  }
}

 
public class FastMotorcycleListItem : TableServiceEntity
{
  public FastMotorcycleListItem()
  {
  }
 
  public FastMotorcycleListItem(string partitionKey, string rowKey)
    : base(partitionKey, rowKey)
  {
  }
}
 
public class FastMotorcycleList : TableServiceContext
{
  public FastMotorcycleList(string baseAddress,
    StorageCredentials storageCredentials)
    : base(baseAddress, storageCredentials)
  {
  }
 
  public DataServiceQuery<FastMotorcycleListItem> Items
  {
    get
    {
      return this.CreateQuery<FastMotorcycleListItem>("FastBikes");
    }
  }
}

Part 3: Deploying the RESTful Web Service

This is one of the areas where Azure really shines. It’s as simple to deploy 100 RESTful Web service instances as it is to deploy only one. Note the following list of steps:

1. In Visual Studio, right-click on FastMotorcycleProject and select Package.
2. Return back to the browser with the portal and select “Hosted Services, Storage Accounts & CDN.”
3. In the top pane, select “Hosted Services.”
4. In the middle pane, select the Hosted Service you previously created.
5. Right-click and select “New Production Deployment” and upload the files (FastMotorcycleProject.cspkg and ServiceConfiguration.Cloud.cscfg); these were generated in the first step.

Part 4: Consuming the RESTful Web Service from Mobile Applications

Now we’ll discuss consuming the RESTful Web services from various mobile applications. This section is meant to highlight the interoperability this approach provides.

The JSONKit (github.com/johnezang/JSONKit) makes interacting with the RESTful Web service from iOS devices easier. With a few lines of code, it’s possible to call the RESTful Web service, download the JSON-formatted data, convert it to a more usable format and attach the converted data to a Table View control, used by iPhone or iPad applications (see Figure 10).

Figure 10 Objective-C Code That Parses JSON Data

NSString *username = @"Bruno"; // Gets passed to the RESTful Web Service
  
NSString *serviceUri = "https://your_hosted_service_name.cloudapp.net/"+
  "FastMotorcycleListService.svc/list/";
// Build the service URI (will point to our RESTful Web service
NSString *url = [NSString stringWithFormat:@"%@%@", serviceUri, username];
      
// Retrieve the data in the form of a JSON array
NSData *json = [NSData dataWithContentsOfURL:[NSURL URLWithString:url]];
     
// Convert from JSON array to NSArray
// This allows us to populate the table view more easily
NSArray *itemArray = [json objectFromJSONData];
     
// Assign the array to the TableView
// fastbikes is the name of our TableView control
self.fastbikes = [[NSMutableArray alloc] initWithArray:itemArray];

Developing for Android involves the Java programming language, which has been around for a long time and can natively parse JSON data. Figure 11 shows an example. The Windows Phone SDK includes native support to call RESTful Web services and process the JSON-formatted data. The SDK makes it easy to process JSON data with DataContractJsonSerializer. Figure 12 shows an example. Finally, if you’d like to see a more robust toolkit for developing for Android and iOS, you can visit this Microsoft-sanctioned link: github.com/microsoft-dpe.

Figure 11 Android Code That Parses JSON Data

// HttpClient used to talk to Web service
HttpClient httpclient = new DefaultHttpClient();
              
String url = 
  "https://your_hosted_service_name.cloudapp.net/"+
  "FastMotorcycleListService.svc/list/Bruno";
// This will be the array we need to convert
// We get the data from the Web service
JSONArray listItems = null;
String jason = null;
              
// Set up the RESTful call to 'GET' the data
HttpGet request_http_get = new HttpGet(url);
  
              
// Read the JSON data and assign it to ListView
try 
{
  // Fill a response object using a request
  HttpResponse response_http_get = httpclient.execute(request_http_get);
            
  // Length represents the number of data items returned
  // by RESTful Web service
  long length = response_http_get.getEntity().getContentLength();

  // "entity" ends up being the data coming back from Web server
  HttpEntity entity = response_http_get.getEntity();
 
  // Read the bytes, one byte at a time
  InputStream stream = entity.getContent();
                 
  // Allocate a series of bytes
  byte[] buffer = new byte[(int) length];
                
  // Read bytes from RESTful Web service
  // After this loop, we end up with something like -> 
  // ["busa","gxr1000","ninja250"]
  for (int i = 0; i < length; i++) 
  {
    buffer[i] = (byte) stream.read();
  }
  // Create an array of strings
  jason = new String(buffer);
  // Convert to JSON array for Android ListBox
  // listItems ends up being a three-element JSON array (see "busa")
  listItems = new JSONArray(jason);
  } 
  catch (Exception e) 
  {
    System.out.println(e);
  }

Figure 12 C# Code That Parses JSON Data

private void LoadList()
{
  string uri =   
    @"https://your_hosted_service_name.cloudapp.net/"+
    "FastMotorcycleListService.svc/list/Bruno";
  var webRequest = (HttpWebRequest)WebRequest.Create(uri);
  webRequest.Method = "GET";
 
  try
  {
    webRequest.BeginGetResponse(new AsyncCallback((result) =>
    {
      var webResponse = 
        (HttpWebResponse)webRequest.EndGetResponse(result);
  
      if (webResponse.StatusCode == HttpStatusCode.OK)
      {
        var jsonDeserializer = 
          new DataContractJsonSerializer(typeof(List<string>));
        List<string> items = 
          (List<string>)jsonDeserializer.ReadObject(
          webResponse.GetResponseStream());                      

        shoppingListBox.Dispatcher.BeginInvoke(new Action(() =>
        {
          shoppingListBox.Items.Clear();
          foreach (var item in items)
          {
            shoppingListBox.Items.Add(item);
          }
        }));
      }

    }), null);
  }
  catch
  {
     // Ignored
  }
}

Access to Entire Spectrum of Devices

Because Azure-hosted RESTful Web services are based on HTTP, any client application that supports this protocol is capable of communicating with them. This opens up a wide spectrum of devices for developers, because the majority of devices fall into this category. Although we covered mobile platforms in this article, JavaScript implementations such as jQuery are also capable of consuming RESTful Web services. Regardless of the path mobile platforms take in regard to UI diversity, it will always make sense to build on simple, open, HTTP-based Web service architectures.

Bruno Terkaly works as a developer evangelist for Microsoft. His depth of knowledge comes from years of experience in the field, writing code using a multitude of platforms, languages, frameworks, SDKs, libraries and APIs. He spends time writing code, blogging and giving live presentations on building cloud-based applications, specifically using the Azure platform.

Ricardo Villalobos is a seasoned software architect with more than 15 years of experience designing and creating applications for companies in the supply chain management industry. Holding different Microsoft certifications, as well as an MBA in Supply Chain Management from the University of Dallas, he joined Microsoft in 2010 as an Azure architect evangelist.

Thanks to the following technical experts for reviewing this article: Reza Alizadeh and Wade Wegner