September 2011

Volume 26 Number 09

Open Data Protocol - Build Great Experiences on Any Device with OData

By Shayne Burgess | September 2011

In this article, I’ll show you what I believe is a serious data challenge facing many organizations today, and then I’ll show you how the Open Data Protocol (OData) and its ecosystem can help to alleviate that challenge. I’ll then go further and show how OData can be used to build a great experience on Windows Phone 7 using the new OData library for Windows Phone 7.1 beta (code-named “Mango”).

The Data Reach Challenge

An interesting thing happened at the end of 2010: for the first time in history, the number of smartphone shipments surpassed the number of PC shipments. A key thing to note is that this isn’t a one-horse race. There are many players (Apple Inc., Google Inc., Microsoft, Research In Motion Ltd. and so on) and platforms (desktop, Web, phones, tablets and more coming all the time). Many organizations want to target client experiences on all or most of these devices. And many organizations also have a variety of data and services that they want to make available to client devices. These may be on-premises, available through traditional Web services or built in the cloud.

The combination of a large and ever-expanding variety of client platforms with a large and ever-expanding variety of services creates a significant cost and complexity problem. When support for a new client platform is added, the data and services often have to be updated and modified to support that platform. And when a new service is added, all of the existing client platforms need to be modified to support that service. This is what I consider to be the data reach problem. How can we define a service that’s flexible enough to suit the needs of all the existing client experiences—and new client experiences that haven’t been invented yet? How can we define client libraries and applications that can work with a variety of services and data sources? These are some of the key questions that I hope to show can be partly addressed with OData.

The Open Data Protocol

OData is a Web protocol for querying and updating data that provides a uniform way to unlock your data. Simply put, OData provides a standard format for transferring data and a uniform interface for accessing that data. It’s based on ATOM and JSON feeds and the interface uses standard HTTP (REST) interfaces for exposing querying and updating capabilities. OData is a key component in bridging the data reach problem because, being a uniform, flexible interface, the API can be created once and used from a variety of client experiences.

The OData Ecosystem

The flexible OData interface that can be used from a variety of clients is most powerful when there are a variety of client experiences that can be built using existing OData-enabled clients. The OData client ecosystem has evolved over the last few years to the point where there are client libraries for the majority of client devices and platforms, with more coming all the time. Figure 1 lists just a sampling of the client and server platforms available at the time this article was written (a much more complete list of existing OData services is at www.odata.org/ecosystem/).

Figure 1 A Sampling of the OData Ecosystem

I can generally target any of the listed services in Figure 1 with any client. To help demonstrate this in practice, I’ll give examples of OData flexibility using a couple of different clients.

I’ll use the Northwind OData (read-only) sample service available on odata.org at https://services.odata.org/Northwind/Northwind.svc/. If you’re familiar with the Northwind database, the Northwind service shown in Figure 2 simply exposes the Northwind model as an OData service directly. Figure 2 shows the service document in the Northwind service from the browser; the service document exposes the entity sets in the service that are basically just the tables from the Northwind database.

Figure 2 The Northwind Service

Let’s look at a couple of examples of using the client libraries to consume and build experiences using the Northwind OData service. For the first example, let’s consider a client experience that doesn’t require any code to build. The PowerPivot add-in for Excel provides an in-memory analysis engine that can be used directly from within the Excel interface and supports importing data directly from an OData feed. The PowerPivot add-in also includes support for importing data directly from the Azure Marketplace DataMarket, as well as a number of other data sources.

Figure 3 shows a pivot chart created by importing the Products and Orders data from the Northwind OData service and joining the result. The price-per-unit and quantity fields of the order details are combined to produce a total value for each order, and then the average total value for each order is shown, grouped by product and then displayed on a simple pivot chart for analysis. All of this is done using the built-in interface and tools in PowerPivot and Excel.

Figure 3 PowerPivot Add-In

Let’s look next at another example of how to build an OData client experience on a different platform: iOS. The OData client library for iOS lets you build apps on that platform (such as for the iPad and iPhone) that consume existing OData services. The iOS library for OData includes a code-generation tool, odatagen, which will generate a set of proxy classes from the metadata on the service. The proxy classes provide facilities for generating OData URIs, deserializing a response into client-side objects and issuing create, read, update and delete (CRUD) operations against the service. The following code snippet shows an example of executing a request for the set of customers from the OData service:

// Create the client side proxy and specify the service URL.

 NorthwindCatalog *proxy =

  [[NorthwindCatalog alloc]initWithUri:@https://host/Northwind.svc];

 

// Create a query.

DataServiceQuery *query = [proxy Customers];

 

QueryOperationResponse *response = [query execute];

customers = [query getResults];

The DataServiceQuery and QueryOperationResponse are used to execute a query by generating a URI and executing it against the service.

The Windows Phone 7 Library

Next, I’ll demonstrate another example of an OData library on a third platform. I did a quick overview of building on OData on other platforms, but for this one I’ll do a detailed walk-through to give insight into what it takes to build an app. To do this, I’ll show you the basic steps needed to write a data-driven experience on Windows Phone 7 using the new Windows Phone 7.1 (Mango) tools. For the rest of the article, I’ll walk through the key considerations when building a Windows Phone 7 app and show how the OData library is used. The app I build will target the same Northwind sample I used in the previous two examples (the Excel PowerPivot add-in and the iOS library).

The latest release of the Windows Phone 7.1 (Mango) beta tools features a significant upgrade in the SDK support for OData. The new library supports a set of new features that makes creating OData-enabled Windows Phone 7 apps easier. The Visual Studio Tools for Windows Phone 7 were updated to support the generation of a custom client proxy based on a target OData service. For those familiar with the existing Microsoft .NET Framework and Silverlight OData clients, this, too, will be familiar. To use the Visual Studio tools to automatically generate the proxy in a Windows Phone 7 project, right-click the Service References node in the project tree and select Add Service Reference (note that this is only available if the new Mango tools are installed). Once the Add Service Reference dialog appears, enter the URI of the OData service you’re targeting and select Go. The dialog will display the entity sets exposed by the service; you can then enter a namespace for the service and select OK. The first time the code generation is performed using Add Service Reference, it will use default options, but it’s possible to further configure the code generation by clicking the Show All Files option in the solution explorer, opening the .datasvcmap file in the service reference and configuring the parameters. Figure4 shows the completed Add Service Reference dialog with the list of entity sets available.

Figure 4 Using the Add Service Reference Dialog

A command-line tool is also included in the Visual Studio Tools for Windows Phone 7 that’s able to perform the same code-generation step. In either case, the code generation will create a client proxy class (DataServiceContext) for interacting with the service, and a set of proxy classes that reflect the shape of the service.

LINQ Support

The client LINQ support in the latest version of the Windows Phone 7 tools—similar to the LINQ support currently available in the .NET client—allows you to write complex LINQ queries and have those LINQ queries translated into a request to the OData service via a URI. The library also supports executing pre-built URIs to an OData service directly, but the LINQ syntax provides a much cleaner experience and hides the complexity of building valid OData URIs. Figure 5 shows an example of using the LINQ support to create a LINQ expression that will query for all products in the Northwind database that have units in stock.

Figure 5 Executing a LINQ Query

public void LoadData()

{

  // Create a Northind Entities context and set the URL of the service.

  NorthwindEntities svcContext = new NorthwindEntities(

    new Uri("https://services.odata.org/Northwind/Northwind.svc")

  );

            

  // Create a DataServiceCollection to hold the results of the query.

  // This collection implements INotifyCollectionChanged and can be bound in XAML.

  DataServiceCollection<Product> ProductsCollection =

    new DataServiceCollection<Product>( svcContext );

           

  // Execute a LINQ query for all products with some units in stock and

  // include the supplier.

  var q = from p in svcContext.Products.Expand("Supplier")

    where p.UnitsInStock > 0

    select p;

 

  // Asynchronously execute the query and load the results.

  ProductsCollection.LoadAsync(q);

}

If you use the Add Service Reference tool described previously, a client proxy context will be generated for you that supports building LINQ queries.

The OData library includes an OData-specific Collection type, DataServiceCollection<T>, which implements INotifyCollectionChanged. If a DataServiceCollection is used to store entities on the client, the entities can be bound to UI elements in XAML and will automatically be change-tracked. Any updates to those tracked entities can be pushed to the service via a call to SaveChanges on the client context. Figure 6 shows some basic XAML binding using the DataServiceCollection class—the proxy classes generated using the tools implement INotifyPropertyChanged and will reflect the changes made in the UI.

Figure 6 XAML Binding

<ListBox x:Name="ProductListBox" Margin="0,0,-12,0"

  ItemsSource="{Binding Products}">

  <ListBox.ItemTemplate>

    <DataTemplate>

      <StackPanel Margin="0,0,0,17" Width="432" Height="78">

        <TextBlock Text="{Binding ProductName}" TextWrapping="NoWrap"

          Style="{StaticResource PhoneTextExtraLargeStyle}"/>

        <TextBlock Text="{Binding Supplier.CompanyName}"

          TextWrapping="Wrap" Margin="12,-6,12,0"

          Style="{StaticResource PhoneTextSubtleStyle}"/>

      </StackPanel>

    </DataTemplate>

  </ListBox.ItemTemplate>

</ListBox>

It’s possible to turn off the use of the DataServiceCollection in the code generation if you’re planning to manage the change tracking manually. This may slightly increase the performance of your app because the notification events won’t be raised if this feature isn’t used.

The process of executing a query against an OData service and materializing the results involves a call to the network stack on Windows Phone 7, which must always be an asynchronous call. Methods on the DataServiceContext, BeginExecute and EndExecute, asynchronously execute a request to the service using the IAsyncResult pattern. The IAsyncResult pattern can sometimes be cumbersome to use, so a method on the DataServiceCollection called LoadAsync hides most of the details of the asynchronous execution. The LoadAsync method will asynchronously add the result of the query to the DataServiceCollection and raise INotifyCollectionChanged events for the UI to repopulate itself. The collection also has a LoadCompleted event that can be subscribed to, which would be called when the data is done asynchronously loading.

Tombstoning

A Windows Phone 7 app will sometimes get a notification from the OS that it needs to deactivate so that the OS can activate another app. This can happen, for example, if the user selects the Home button on the phone or if the user gets an incoming call. The app won’t always be deactivated in the Mango beta release of Windows Phone 7 because the fast application-switching feature will often allow the OS to switch to another application without having to deactivate the current one first. When the app is told by the OS to deactivate, the app has the option to store its current state locally so that if it’s activated again—which can happen if the user subsequently selects the Back button, for example—the app can restore its previous state and the user can resume what he was doing before the app was deactivated. This is called tombstoning. This can be particularly useful in an app that consumes data from an external data source, because tombstoning saves the app from having to go to the network and use bandwidth to download the data from the external data source again.

Tombstoning in Windows Phone 7 is accomplished by serializing the state of the app into a set of strings in a dictionary. The OData client library for Windows Phone 7 has utilities to aid in serializing the current state by providing methods that can serialize or deserialize a DataServiceContext and a set of DataServiceCollections. The sample code in Figure 7 shows an example of using these methods to build a string that represents the current state. Windows Phone 7 applications must implement the Application_Activated and Application_Deactivated methods that are used by the OS to indicate when the app should activate or deactivate. The serialize and deserialize methods on the DataServiceContext can be used from these methods.

Figure 7 Serialization/Deserialization of DataServiceContext

// This method will serialize the local state to be stored

// when the app is deactivated.

public string Serialize()

{

  var saveItems = new Dictionary<string, object>();

  saveItems.Add("Products", ProductsCollection);

  return DataServiceState.Serialize(svcContext, saveItems);

}

 

// This method will deserialize the local state to be restored

// when the app is reactivated.

public void Deserialize (string state)

{

  DataServiceState dsState= DataServiceState.Deserialize(state);

  svcContext = dsState.Context as NorthwindEntities;

  if (dsState.RootCollections.ContainsKey("Products"))

  {

    ProductsCollection = dsState.RootCollections["Products"] as

     DataServiceCollection<Product>;

  }

}

Credentials

I’ll cover one final but important part of building an OData app on Windows Phone 7.1 (Mango): providing credentials to the OData service. In many cases, the service in question is behind an authorization scheme to protect the data from unauthorized use. Support has been added in the Mango tools to set the Credentials property on the HTTP stack when making calls to the OData service using the client library. The property takes an implementation of the ICredentials interface and passes that interface directly to the HTTP stack. The following code snippet shows the basics of creating the DataServiceContext and setting the Credentials property using a username, password and domain name:

// This method will serialize the local state to be stored

// when the app is deactivated.

public MainViewModel()

{

  // Create the data service context.

  NorthwindEntities serviceContext = new NorthwindEntities(

    new Uri("https://services.odata.org/Northwind/Northwind.svc"));

 

  // Specify the basic auth credentials.

  serviceContext.Credentials = new NetworkCredential(Username, Password, Domain);

}

Learning More

This article gives a summary of OData, the ecosystem that has been built around it and a walk-through using the OData client for Windows Phone 7.1 (Mango) beta to build great mobile app experiences. For more information on OData, visit odata.org. To get more information on WCF Data Services, see bit.ly/pX86x6 or the WCF Data Services blog at blogs.msdn.com/astoriateam.

Shayne Burgess is a program manager in the Business Platform Division at Microsoft, working specifically on WCF Data Services and the Open Data Protocol. He regularly blogs on the WCF Data Services blog at blogs.msdn.com/astoriateam.

Thanks to the following technical expert for reviewing this article: Glenn Gailey