Skip to main content

Open Data Protocol by Example

Chris Sells

March 2010

The purpose of the Open Data protocol[i] (hereafter referred to as OData) is to provide a REST-based protocol for CRUD-style operations (Create, Read, Update and Delete) against resources exposed as data services. A “data service” is an endpoint where there is data exposed from one or more “collections” each with zero or more “entries”, which consist of typed named-value pairs. OData is published by Microsoft under the Open Specification Promise so that anyone that wants to can build servers, clients or tools without royalties or restrictions.

Exposing data-based APIs is not something new. The ODBC (Open DataBase Connectivity) API is a cross-platform set of C language functions with data source provider implementations for data sources as wide ranging as SQL Server and Oracle to comma-separated values and Excel files. If you’re a Windows programmer, you may be familiar with OLEDB or ADO.NET, which are COM-based and.NET-based APIs respectively for doing the same thing. And if you’re a Java programmer, you’ll have heard of JDBC. All of these APIs are for doing CRUD across any number of data sources.

Since the world has chosen to keep a large percentage of its data in structured format, whether it’s on a mainframe, a mini or a PC, we have always needed standardized APIs for dealing with data in that format. If the data is relational, the Structured Query Language (SQL) provides a set of operations for querying data as well as updating it, but not all data is relational. Even data that is relational isn’t often exposed for use in processing SQL statements over intranets, let alone internets. The structured data of the world is the crown jewels of our businesses, so as technology moves forward, so must data access technologies. OData is the web-based equivalent of ODBC, OLEDB, ADO.NET and JDBC. And while it’s relatively new, it’s mature enough to be implemented by IBM’s WebSphere[ii], be the protocol of choice for the Open Government Data Initiative[iii] and is supported by Microsoft’s own SharePoint 2010 and WCF Data Services framework[iv]. In addition, it can be consumed by Excel’s PowerPivot, plain vanilla JavaScript and Microsoft’s own Visual Studio development tool.

In a web-based world, OData is the data access API you need to know.

Atom

OData is built as an application of a set of internet standards, most especially the Atom Publishing Protocol (aka “AtomPub” specified as RFC 5023), which itself is built on top of the Atom Syndication Format (aka “Atom” specified as RFC 4287). Because of this layering, Atom is the XML format in which OData data is published. AtomPub is the protocol to provide not only getting Atom data, but also creating it, updating it and deleting it, all of which OData relies upon. Finally, OData provides a set of features on top of Atom and AtomPub to make the data query and manipulation richer. So, before we get to AtomPub or OData, let’s look at the base: Atom.

Atom is a simple way of exposing a “feed” of data, each distinct piece of which is called an “entry.” Both the feed and the entry have a set of required and optional bits of metadata. Atom[v] was originally invented in the world of energetic bloggers as a way to keep readers up to date on who had something new to say. For example, the following single-entry Atom feed document example is pulled from the Atom specification itself:

<?xml version="1.0" encoding="utf-8"?> 
<feed xmlns="http://www.w3.org/2005/Atom">
  <title>Example Feed</title> 
  <link href="http://example.org/"/>
  <updated>2003-12-13T18:30:02Z</updated>
  <author> 
    <name>John Doe</name>
  </author> 
  <id>urn:uuid:60a76c80-d399-11d9-b93C-0003939e0af6</id>
  <entry>
    <title>Atom-Powered Robots Run Amok</title>
    <link href="http://example.org/2003/12/13/atom03"/>
    <id>urn:uuid:1225c695-cfb8-4ebb-aaaa-80da344efa6a</id>
    <updated>2003-12-13T18:30:02Z</updated>
    <summary>Some text.</summary>
  </entry>
</feed>

You’ll notice at the top level the “http://www.w3c.org/2005/Atom” namespace declaration. This is the namespace that means “Atom” and Atom may be legally extended by using elements from other namespaces (as per standard XML rules). We’ll see more of this later.

The top level element, feed, in this case has a title, link, updated, author and id child elements, all of which are required except link. Other elements, e.g. category, icon, subtitle, etc., can be provided if they are useful for the data entries contained. The feed element is really there to contain the set of entry child elements, which contain the important payload.

The ID associated with an entry must be unique for that feed. If the particular entry is updated or served up from a different address on the web, the ID must remain the same. This allows clients to do things like cache entries to find changes or notice new entries. The ID must be in a specific format[vi] and is usually universally unique (like the strings provided by the uuidgen.exe tool available to Windows developers). IDs are just identifiers and therefore you can only compare them – you can’t use them to get to anything. If you want to do that, you need to use the link attribute.

If a link attribute is provided, it can be absolute or relative. If it’s relative, you can use the xml:base attribute in scope to find the base to which the link is relative. For example, if our little entry sample, the link is absolute, but it could have been relative like so:

<feed xmlns="http://www.w3.org/2005/Atom" xml:base="http://example.org/">
  ...
  <entry>
    <link href="2003/12/13/atom03"/>
     ...
  </entry>
</feed>

In this case, the link to the item is the same, except you need to follow the parent elements ‘til you find an xml:base attribute to find the relative link base and therefore to be able to compose the absolute URL.

Also, in this example, the single entry is something you’d typically see on a blog, with a title, updated date and a summary, where the summary could be provided as a teaser of some great content to come if the link is followed. And while some blogs expose their data this way, as a means to get people to visit their site, many of them pack the interesting content right into the entry itself, e.g.

<entry>
    <title>Atom-Powered Robots Run Amok</title>
    <id>urn:uuid:1225c695-cfb8-4ebb-aaaa-80da344efa6a</id>
    <updated>2003-12-13T18:30:02Z</updated>
    <content type="xhtml">
     <div xmlns="http://www.w3.org/1999/xhtml">
        Those robots were <i>crazy</i>! ...
     </div>
    </content>
  </entry>

Notice the use of content element here. It’s been marked with the “xhtml” type, which indicates content following the XHTML format standardized by the W3C (World Wide Web Consortium) in 1999. If you’re a blog author, this is a useful format in which to be able to publish in, as are the “html” and “text” types that the content element of Atom also supports. However, the type of a content element can be any arbitrary MIME type, which means that you can also see data like the following:

<feed xml:base="http://localhost:53211/owind.svc/"
  xmlns:d="http://schemas.microsoft.com/ado/2007/08/dataservices"
  xmlns:m="http://schemas.microsoft.com/ado/2007/08/dataservices/metadata"
  xmlns="http://www.w3.org/2005/Atom">
  <title type="text">Categories</title> 
  <id>http://localhost:53211/owind.svc/Categories</id> 
  <updated>2009-12-26T02:27:02Z</updated> 
  <link rel="self" title="Categories" href="Categories" /> 
  <entry>
    <id>http://localhost:53211/owind.svc/Categories(1)</id> 
    <title type="text" /> 
    <updated>2009-12-26T02:27:02Z</updated> 
    <author>
      <name /> 
    </author>
    <link rel="edit" title="Category" href="Categories(1)" /> 
    <link rel="http://schemas.microsoft.com/ado/2007/08/dataservices/related/Products"
      type="application/atom+xml;type=feed" title="Products"
      href="Categories(1)/Products" /> 
    <category term="NorthwindModel.Category"
      scheme="http://schemas.microsoft.com/ado/2007/08/dataservices/scheme" /> 
    <content type="application/xml">
      <m:properties>
        <d:CategoryID m:type="Edm.Int32">1</d:CategoryID> 
        <d:CategoryName>Beverages</d:CategoryName> 
        <d:Description>Soft drinks, coffees, teas, beers, and ales</d:Description> 
        <d:Picture m:type="Edm.Binary">FRwvAA...</d:Picture> 
    </m:properties>
    </content>
  </entry>
  ...
</feed>

In this particular case, this is still a legal, plain ol’ Atom feed but with two new namespace declarations from the OData specification. Suffice to say that instead of the content element containing text or HTML meant to be read by a human, we’ve got XML data (as specified by the “application/xml” type) which contains a set of properties meant to be read by programs.

Notice also the use of the link elements, both relative to the xml:base at the root of the feed document. The rel attribute of “edit” indicates that if you’d like to update the entry, you can do so at the href provided. The “edit” value of the rel attribute isn’t defined in the Atom specification itself, but is in the AtomPub specification, which we’ll discuss later. The other link is very much part of OData and provides the relative link to the feed document where you can get to the Products (as defined by the title of the link) associated with this particular Category.

And not that it matters to the Atom protocol or the consumers of this Atom feed one whit, but the astute will notice the exact same data and schema as that of the Categories table of the Northwind sample database that Microsoft has been shipping since long before the web took hold. As part of this discussion, this sample database is exposed as an example of an OData service built to be used on the localhost and instructions on how to build it are found later in this article. If you refer to odata.org, you can find several publically available data services with which to experiment, including an online version of Microsoft’s Northwind sample database.

AtomPub

Atom is meant to be simple: point your feed reader software, e.g. RSS Bandit or Microsoft Outlook, at the URL of an Atom service and it will tell you when there’s something new. However, if you want to publish new data, you need a whole other API, like the Metaweblog API, the WordPress API, etc. Or you can use AtomPub.

Service Documents

As I mentioned, AtomPub (the Atom Publishing Protocol) is a set of services for doing more with Atom feed documents, specifically exposing one or more of them as “collections” and allowing the entries to be created, updated or deleted as well as read. Collections are gathered together into one or more workspaces (just another container) which themselves are gathered into a service document, like this abbreviated example from the AtomPub specification:

<service xmlns="http://www.w3.org/2007/app"
         xmlns:atom="http://www.w3.org/2005/Atom">
  <workspace>
    <atom:title>Main Site</atom:title>
    <collection href="http://example.org/blog/main" >
      <atom:title>My Blog Entries</atom:title>
    </collection>
    <collection href="http://example.org/blog/pic" >
      <atom:title>Pictures</atom:title>
      <accept>image/png</accept>
      <accept>image/jpeg</accept>
      <accept>image/gif</accept>
    </collection>
  </workspace>
  <workspace>
    <atom:title>Sidebar Blog</atom:title>
    <collection href="http://example.org/sidebar/list" >
      <atom:title>Remaindered Links</atom:title>
      <accept>application/atom+xml;type=entry</accept>
    </collection>
  </workspace>
</service>

In this case, the service document (notice the root element) has two workspaces (“Main Site” and “Sidebar Blog”) with three total collections (“My Blog Entries”, “Pictures” and “Remaindered Links”). Notice that the Atom namespace is pulled in and that the title element is reused to name the workspaces and collections. In addition, notice the AtomPub namespace of “http://www.w3.org/2007/app” where “app” stands for “Atom Publishing Protocol”.

Inside each collection, there can be zero or more “accept” elements, each of which describes the MIME type of the kind of content that the collection can accept as new or updated content. In this example, the “Pictures” collection can accept images in various formats and the “Remainered Links” collection can accept data in the “application/atom+xml;type=entry” format, which means an Atom entry document[vii]. In the absence of an “accept” element, as in the case of the “My Blog Entries” collection, Atom entries are assumed, so that particular piece of data will often be left off.

While it’s possible for the service document to describe more about each feed, like what categories each collection excepts, the most important thing is the href attribute of the collection element, which defines the URL for the Atom feed document where entries from each collection can be found. For example, an AtomPub service document for the Northwind sample we saw earlier could look like this:

<service xml:base="http://localhost:53211/owind.svc/"
  xmlns:atom="http://www.w3.org/2005/Atom"
  xmlns:app="http://www.w3.org/2007/app"
  xmlns="http://www.w3.org/2007/app">
  <workspace>
    <atom:title>Default</atom:title> 
    <collection href="Categories">
      <atom:title>Categories</atom:title> 
    </collection>
    ...
  </workspace>
</service>

If we were to follow the href of the Categories collection from this AtomPub service document, we’d expect to find ourselves at an Atom feed document which gives us a list of entries, one for each of the items in the collection as defined by the server’s notion of the complete set of Category entries. In our sample, that’s the items from the Northwind Categories table, but again, that’s merely an implementation detail and to the AtomPub client couldn’t matter less.

Creating, Reading, Updating and Deleting Entries

As I mentioned, in addition to grouping one or more feeds into a workspace and further into a service document, AtomPub also provides for the ability to create new entries and update and delete existing ones. While the HTTP method of GET is used to retrieve service documents containing collections and feed or entry documents containing entries (that’s the method used when you type a URL into your browser), the following are the set of methods used in AtomPub to manipulate the entries exposed by the collections:

  • GET: Get a collection of entries (as a feed document) or a single entry (as an entry document).
  • POST: Create a new entry from an entry document.
  • PUT: Update an existing entry with an entry document.
  • DELETE: Remove an entry.

Any platform that provides support for HTTP (and ideally XML) is enough to form HTTP requests to interact with AtomPub. For example, here’s the .NET code to create a new entry against our sample Categories collection:

using System;
using System.IO;
using System.Net;
using System.Xml;
using System.Xml.Linq;
...
static void CreateCategory() {
  // Create a new Category in XML to stash in the AtomPub content element
  // NOTE: this is specific to OData.
  XNamespace ds = "http://schemas.microsoft.com/ado/2007/08/dataservices";
  XNamespace dsmd = "http://schemas.microsoft.com/ado/2007/08/dataservices/metadata";
  var content =
    new XElement(dsmd + "properties",
      new XElement(ds + "CategoryName", "Dessert Topping"),
      new XElement(ds + "Description", "Toppings for desserts")
    );

  // Create the AtomPub entry. This is generic to AtomPub.
  XNamespace atom = "http://www.w3.org/2005/Atom";
  var entry =
    new XElement(atom + "entry",
      new XElement(atom + "title", "A new Northwind category"),
      new XElement(atom + "id", string.Format("urn:uuid:{0}", Guid.NewGuid())),
      new XElement(atom + "updated", DateTime.Now),
      new XElement(atom + "author",
      new XElement(atom + "name", "Chris Sells")),
      new XElement(atom + "content",
        new XAttribute("type", "application/xml"),
        content)
    );

  // Send the HTTP request
  HttpWebRequest request =
   (HttpWebRequest)HttpWebRequest.Create(@"http://localhost:8080/owind.svc/Categories/");
  request.Method = "POST";
  request.Accept = "application/atom+xml";
  request.ContentType = "application/atom+xml;type=entry";
  using (var writer = XmlWriter.Create(request.GetRequestStream())) {
    entry.WriteTo(writer);
  }

  WebResponse response = request.GetResponse();
  if (((HttpWebResponse)response).StatusCode == HttpStatusCode.Created /* 201 */ ) {
    Console.WriteLine(
      "Created category @ Location= {0}", response.Headers["Location"]);
  }
  else {
    Console.WriteLine("Category creation failed");
  }
}

In this little bit of code, we’re creating an HTTP POST request which creates an Atom entry document in XML. We’re using .NET LINQ to XML API to create the XML, first creating the XML for the Category content according to the rules of OData, then creating the rest of the enclosing document according to the AtomPub specification.

After the content element is created, we package it into an Atom entry element and submit the POST request, which looks like this on the wire (formatted a bit for easy reading):

POST /owind.svc/Categories/ HTTP/1.1
Accept: application/atom+xml
Content-Type: application/atom+xml;type=entry
Host: localhost:8080
Content-Length: 632
Expect: 100-continue
Connection: Keep-Alive

<?xml version="1.0" encoding="utf-8"?>
<entry xmlns="http://www.w3.org/2005/Atom">
  <title>A new Northwind category</title>
  <id>urn:uuid:25ab211e-cb1e-436b-b3c0-54ad960cff4d</id>
  <updated>2009-12-25T22:04:28.2682406-08:00</updated>
  <author><name>Chris Sells</name></author>
  <content type="application/xml">
    <properties
      xmlns="http://schemas.microsoft.com/ado/2007/08/dataservices/metadata">
      <CategoryName
        xmlns="http://schemas.microsoft.com/ado/2007/08/dataservices">Dessert Topping</CategoryName>
      <Description
        xmlns="http://schemas.microsoft.com/ado/2007/08/dataservices">Toppings for desserts</Description>
    </properties>
  </content>
</entry>

You’ll notice the POST method showing up as the first part of our HTTP request as well as the appropriate Content-Type header as specified by the AtomPub specification. Also, the entry element is specified as per the Atom specification, with the content as per the OData specification (as we’ll discuss later). In the event of a successful creation request, which maps to an HTTP status code of 201[viii], the output of our program is the location of the newly created Category, which is returned in the Location header of the response, again according to the AtomPub specification:

Created category @ Location= http://localhost:8080/owind.svc/Categories(25)

If you were to surf to the location returned by the server in response to a successfully created entry, you should see an entry that looks exactly like the one we put in the HTTP POST response (again, reformatted for readability):

HTTP/1.1 201 Created
Server: ASP.NET Development Server/10.0.0.0
Date: Sat, 26 Dec 2009 06:16:50 GMT
X-AspNet-Version: 4.0.21006
DataServiceVersion: 1.0;
Content-Length: 1137
Location: http://localhost:8080/owind.svc/Categories(25)
Cache-Control: no-cache
Content-Type: application/atom+xml;charset=utf-8
Connection: Close

<entry xml:base="http://localhost:8080/owind.svc/"
  xmlns:d="http://schemas.microsoft.com/ado/2007/08/dataservices"
  xmlns:m="http://schemas.microsoft.com/ado/2007/08/dataservices/metadata"
  xmlns="http://www.w3.org/2005/Atom">
  <id>http://localhost:8080/owind.svc/Categories(25)</id>
  <title type="text"></title>
  <updated>2009-12-26T06:16:50Z</updated>
  <author>
    <name />
  </author>
  <link rel="edit" title="Category" href="Categories(25)" />
  <link rel="http://schemas.microsoft.com/ado/2007/08/dataservices/related/Products"
    type="application/atom+xml;type=feed" title="Products"
    href="Categories(25)/Products" />
  <category term="NorthwindModel.Category"
    scheme="http://schemas.microsoft.com/ado/2007/08/dataservices/scheme" />
  <content type="application/xml">
    <m:properties>
      <d:CategoryID m:type="Edm.Int32">25</d:CategoryID>
      <d:CategoryName>Dessert Topping</d:CategoryName>
      <d:Description>Toppings for desserts</d:Description>
      <d:Picture m:type="Edm.Binary" m:null="true" />
    </m:properties>
  </content>
</entry>

In addition to the Location header, notice the Content-Type and that the root of the returned document is an Atom entry element, neither of which should surprise you. However, what may surprise you is that the Atom elements of id, title, updated and author under the entry root are all different, notable most of them are blank. That’s because our sample implementation doesn’t use them or keep them, so it tosses them. Likewise, while the updated field isn’t blank, it’s not the same as what we specified, picking up the updated date and time based on whatever the server decided it was. The AtomPub specification gives the server a great deal of latitude about how it implements the HTTP methods and this behavior is well within normal bounds.

Finally, notice that the content element has more information then what we sent in, most importantly the CategoryID, which is presumably the unique ID as far as the underlying database is concerned, although from an AtomPub point of view, we don’t care about that nearly as much as we do the id and link sub-elements of the entry root element, which is how we’ll come back to this entry later or update or delete it.

When you’re updating an element, the AtomPub specification says to use the PUT method against the link for the entry (the same one returned by the Location header of a POST request and in the link element with rel set to “edit” in the entry itself) and to provide all elements to the server, even the ones you haven’t changed or that you don’t understand.  In fact, AtomPub doesn’t guarantee that a feed document contains the entire set of data available for an entity unless the entity is retrieved using the “edit” link, so by far the easiest way to do that is to retrieve the entry first, make the changes you like and then send the whole thing back.

Deleting is a simple matter of using the HTTP method DELETE against the link for the entry.

OData

The Open Data Protocol (aka “OData”) is defined by the “Atom Publishing Protocol: Data Services URI and Payload Extensions” specification (catchy title, eh?)[ix], although you’ll be much happier starting with the OData overview documentation[x]. The OData specification defines how AtomPub is used to standardize a typed, resource-oriented CRUD interface for manipulating data sources. If you look at what we’ve seen in AtomPub so far, it’s clear it’s pretty close already, providing collections of entries which must have required elements and that can be extended. Also, AtomPub provides the idea of getting as well as updating and removing entries. What it doesn’t have is any idea of a data model for defining typed or untyped values on an entry (e.g. columns in a row), nor does it have any kind of a query language for getting just the entries and the data that we want[xi]. These are two of the extensions that OData adds on top of AtomPub.

Before we dive into the details on OData, I should mention that, as with any specification, different implementations support different features. I’ll be discussing OData from the protocol point-of-view and demonstrating many of the features, but your mileage may vary service to service depending on which parts of the protocol they implement.

Now, let’s get started on the OData data model.

Properties and Metadata

Earlier, when we constructed the XML that represented a new Category to be created, we split that work into two parts: the necessary set of OData properties (typed name/value pairs) and the AtomPub envelope. By now, you’re familiar with the AtomPub piece, but how did we determine what properties were needed inside the content element in the first place? If we had to just eyeball it, that’s going to make for a difficult programming experience. Luckily, we don’t have to do that – we can ask:

URL: http://localhost:53211/owind.svc/$metadata
Response:

<edmx:Edmx Version="1.0" xmlns:edmx="http://schemas.microsoft.com/ado/2007/06/edmx">
  <edmx:DataServices
    xmlns:m="http://schemas.microsoft.com/ado/2007/08/dataservices/metadata"
    m:DataServiceVersion="1.0">
    <Schema Namespace="NorthwindModel"
      xmlns:d="http://schemas.microsoft.com/ado/2007/08/dataservices"
      xmlns:m="http://schemas.microsoft.com/ado/2007/08/dataservices/metadata"
      xmlns="http://schemas.microsoft.com/ado/2008/09/edm">
      <EntityType Name="Category">
        <Key>
          <PropertyRef Name="CategoryID" />
        </Key>
        <Property Name="CategoryID" Type="Edm.Int32" Nullable="false"
          p8:StoreGeneratedPattern="Identity"
          xmlns:p8="http://schemas.microsoft.com/ado/2009/02/edm/annotation" /> 
        <Property Name="CategoryName" Type="Edm.String" Nullable="false"
          MaxLength="15" Unicode="true" FixedLength="false" /> 
        <Property Name="Description" Type="Edm.String" Nullable="true" MaxLength="Max"
          Unicode="true" FixedLength="false" /> 
        <Property Name="Picture" Type="Edm.Binary" Nullable="true" MaxLength="Max"
          FixedLength="false" /> 
        <NavigationProperty Name="Products"
          Relationship="NorthwindModel.FK_Products_Categories" FromRole="Categories"
          ToRole="Products" /> 
      </EntityType>
    ...
    </Schema>
  </edmx:DataServices>
</edmx:Edmx>

The document returned from the OData $metadata operation is defined by the “Entity Data Model for Data Services Packaging Format” specification[xii] (we’re just great at naming things here at Microsoft, aren’t we?), which is really just a small document that says it’s the Schema element under the Edmx and DataServices elements that you really care about. That Schema element and everything inside it is the “Conceptual Schema Definition File Format” specification, but we’ll just call it the “CSDL spec” (or Conceptual Schema Definition Language specification). CSDL defines Microsoft’s Entity Data Model (EDM), which is also the data model of OData. The CSDL specification tells us how we can interpret the result of the $metadata operation to see what kind of data is being exposed by the OData service. Here’s what it says we can tell about the NorthwindModel.Category type exposed by the Categories collection defined in the AtomPub service document:

  • The Category key is made up of a single property named CategoryID. The key is the thing we use to navigate to a specific entry, as we’ll see. The reason that the Key entry can contain multiple sub-entries is that some entities have multiple values that make up the key.
  • The CategoryID is a 32-bit integer and is non-null. However, because it’s marked with a store pattern of Identity, we know the service is going to create the ID for us and hand it back with the response (as we saw).
  • The CategoryName is a Unicode string of at most 15 characters and is required to be provided when a new Category is created.
  • The Description is a Unicode string of no maximum length except that enforced by the service. It is optional.
  • The Picture is a binary blob of data (provided in Base64 encoded format) and is also optional. We saw this when our .NET client provided no Picture property, but creation was still successful.
  • We can navigate from Categories to Produces as defined by the NorthwindModel.FK_Products_Categories relationship, which we’ll look at more later.

The CSDL instance documents are easy enough to read that you mostly won’t even need to dig through the CSDL specification, although it’s there for letting you what the range of data types is and other niggling details. CSDL defines the usual suspect of primitive types, as shown in Table 1.

TypeExampleDescription
Binary<d:foo m:type="Edm.Binary">FRwvAAI...</d:foo>Fixed or variable length binary data in Base64 encoded format.
Boolean<d:foo m:type="Edm.Boolean">true</d:foo>Represent the mathematical concept of binary valued logic.
Byte<d:foo m:type="Edm.Byte">1</d:foo>An unsigned 8-bit integer value
DateTime<d:foo m:type="Edm.DateTime">2010-02-26T17:08:53.0900752-08:00</d:foo>A date and time with value ranging from 12:00:00 midnight, January 1, 1753 A.D. through 11:59:59 P.M, December 9999 A.D.
DateTimeOffset<d:foo m:type="Edm.DateTimeOffset">2002-10-10T17:00:00Z</d:foo>Date and time as an Offset in minutes from GMT, with values ranging from 12:00:00 midnight, January 1, 1753 A.D. through 11:59:59 P.M, December 9999 A.D.
Time<d:foo m:type="Edm.Time">P120D</d:foo>Time duration.
Decimal<d:foo m:type="Edm.Decimal">3.3</d:foo>A numeric value with fixed precision and scale from -10^255 + 1 to 10^255 - 1.
Double<d:foo m:type="Edm.Double">4.4</d:foo>A floating point number with 15 digits precision that can represent values with approximate range of ± 2.23e -308 through ± 1.79e +308.
Single<d:foo m:type="Edm.Single">4.4</d:foo>A floating point number with 7 digits precision that can represent values with approximate range of ± 1.18e -38 through ± 3.40e +38.
Guid<d:foo m:type="Edm.Guid">223b00d9-e617-4a80-bf18-31bfd6e8e312</d:foo>A 16-byte (128-bit) unique identifier value
Int16<d:foo m:type="Edm.Int16">1</d:foo>Signed 16-bit integer value.
Int32<d:foo m:type="Edm.Int32">2</d:foo>Signed 32-bit integer value.
Int64<d:foo m:type="Edm.Int64">3</d:foo>Signed 64-bit integer value.
String

<d:foo>Beverages</d:foo>

<d:foo m:type="Edm.String">Beverages</d:foo>

Fixed or variable length character data. String is the default type.

Table 1: OData Data Types

A navigation property, however, is not a primitive and you’ll have to dig a little further into the CSDL document to figure out what it means:

<edmx:Edmx Version="1.0" xmlns:edmx="http://schemas.microsoft.com/ado/2007/06/edmx">
  <edmx:DataServices
    xmlns:m="http://schemas.microsoft.com/ado/2007/08/dataservices/metadata"
    m:DataServiceVersion="1.0">
    <Schema Namespace="NorthwindModel"
      xmlns:d="http://schemas.microsoft.com/ado/2007/08/dataservices"
      xmlns:m="http://schemas.microsoft.com/ado/2007/08/dataservices/metadata"
      xmlns="http://schemas.microsoft.com/ado/2008/09/edm">

      ...

      <Association Name="FK_Products_Categories">
        <End Role="Categories" Type="NorthwindModel.Category" Multiplicity="0..1" /> 
        <End Role="Products" Type="NorthwindModel.Product" Multiplicity="*" />
        <ReferentialConstraint>
          <Principal Role="Categories">
            <PropertyRef Name="CategoryID" /> 
          </Principal>
          <Dependent Role="Products">
            <PropertyRef Name="CategoryID" /> 
          </Dependent>
        </ReferentialConstraint>
     </Association>

     ...

    </Schema>
  </edmx:DataServices>
</edmx:Edmx>

In CSDL, a Navigation element references a corresponding Association, which defines the nature of the relationship between two entity types. In most cases, the important part is the multiplicity defined on both ends; in English, this example reads, “Each product has an optional category and each category has any number of products.” When we look at a Category entry, this gives us what we need to know in order to navigate:

URL: http://localhost:53211/owind.svc/Categories(4)
Response:

<entry xml:base="http://localhost:53211/owind.svc/"
    xmlns:d="http://schemas.microsoft.com/ado/2007/08/dataservices"
    xmlns:m="http://schemas.microsoft.com/ado/2007/08/dataservices/metadata"
    xmlns="http://www.w3.org/2005/Atom">
  <id>http://localhost:53211/owind.svc/Categories(4)</id> 
  <title type="text" /> 
  <updated>2009-12-27T01:24:50Z</updated> 
  <author><name /></author>
  <link rel="edit" title="Category" href="Categories(4)" /> 
  <link rel="http://schemas.microsoft.com/ado/2007/08/dataservices/related/Products"
     type="application/atom+xml;type=feed" title="Products"
     href="Categories(4)/Products" /> 
  <category term="NorthwindModel.Category"
    scheme="http://schemas.microsoft.com/ado/2007/08/dataservices/scheme" />
  <content type="application/xml">
    <m:properties>
      <d:CategoryID m:type="Edm.Int32">4</d:CategoryID> 
      <d:CategoryName>Dairy Products</d:CategoryName> 
      <d:Description>Cheeses</d:Description> 
      <d:Picture m:type="Edm.Binary">FRwvAAI...</d:Picture> 
      </m:properties>
    </content>
</entry>

With the CSDL in hand, we’ve got all the information we need to know to create and interpret entries of type Category:

  • To get to the related products, we follow the relative URL in the href to a feed document which can have any number of products in it. We know it’ll take us to products because of the rel=".../Products”.
  • We know that the properties in the content element map to properties on the Category entity type and what types each of the properties is. If it’s not specified, the default type is Edm.String.

And how do we know that the data in this entry is a Category and not something else? OData uses the category element with the OData scheme URI to mean “of entity type defined by the CSDL associated with this service” and the term itself, e.g. “NorthwindModel.Category”, to specify the specify type in that CSDL document.

However, while I have explained the link element to find related products on a given category, I still haven’t told you how I know to construct a URL to retrieve a specific category with which to work. For that, we’ll need to know about another major OData extension to AtomPub: the query language.

Query Navigation

Given the amount of time we’ve spent poring over XML documents defining services, feeds, entries and metadata, you may have noticed that the “edit” link of each entry in our Northwind sample contains the name of the database table and the unique identifier from that table, e.g.

http://localhost:8080/owind.svc/Categories(4)

In fact, OData provides an entire query language directly in the URL format you use to retrieve entries from a service. This is very different from AtomPub that makes no statement about the format of the entry links, promising only that you can follow an “edit” link to the full data representation of the entry suitable for editing. This guarantee is also true in OData, but OData allows you to form URLs based on what you know (and can discover) about the underlying data. For example, you can start at the top level service document and keep drilling, e.g.

URL: http://localhost:8080/owind.svc
Response:
<service xml:base="http://localhost:8080/owind.svc/" ...>
  <workspace>
    <atom:title>Default</atom:title> 
    <collection href="Categories">
      <atom:title>Categories</atom:title> 
    </collection>

    ...

  </workspace>
</service>

Using the href attribute (be careful – case matters), we can then go to:

URL: http://localhost:8080/owind.svc/Categories
Response:

<feed xml:base="http://localhost:8080/owind.svc/" ...>
  ...
  <entry>
    ...
    <link rel="edit" title="Category" href="Categories(1)" /> 
    ...
    <content type="application/xml">
      <m:properties>
        <d:CategoryID m:type="Edm.Int32">1</d:CategoryID> 
        <d:CategoryName>Beverages</d:CategoryName> 
        <d:Description>Soft drinks, coffees, teas, beers, and ales</d:Description> 
        <d:Picture m:type="Edm.Binary">FRwvAAI...</d:Picture> 
      </m:properties>
    </content>
  </entry>
  ...
</feed>

As in AtomPub, we can take the “edit” link and pull just the entry document for each of the entries in the feed document. However, if you know the identifier for any of the entries, you can compose that with the name of the collection inside parentheses and get yourself that entry, as we’ve already seen, as well as being able to get to the related Products:

URL: http://localhost:8080/owind.svc/Categories(2)/Products
Response:

<feed xml:base="http://localhost:8080/owind.svc/" ...>
  ...
  <entry>
  ...
  <link rel="edit" title="Product" href="Products(3)" /> 
  <link rel="http://schemas.microsoft.com/ado/2007/08/dataservices/related/Category"
    type="application/atom+xml;type=entry" title="Category"
    href="Products(3)/Category" /> 
  <link
    rel="http://schemas.microsoft.com/ado/2007/08/dataservices/related/Order_Details"
    type="application/atom+xml;type=feed" title="Order_Details"
    href="Products(3)/Order_Details" /> 
  <link rel="http://schemas.microsoft.com/ado/2007/08/dataservices/related/Supplier"
    type="application/atom+xml;type=entry" title="Supplier"
    href="Products(3)/Supplier" /> 
  ...
  <content type="application/xml">
    <m:properties>
      <d:ProductID m:type="Edm.Int32">3</d:ProductID> 
      <d:ProductName>Aniseed Syrup</d:ProductName> 
      ...
    </m:properties>
  </content>
  </entry>
  ...
</feed>

Again, we’re given a feed document that shows the entries associated with this relationship, in this case the products associated with the specified category. And from here we can see that we can get back to the category for each product, as well as to the order details and supplier associated with this product. Also, notice that the link title is singular Category, showing us the other end of our one-to-many relationship.

Backing up to the category, in addition to navigating through relationships, we can also pull out specific properties from the content:

URL: http://localhost:8080/owind.svc/Categories(2)/Description
Response:

<Description xmlns="http://schemas.microsoft.com/ado/2007/08/dataservices">Sweet and savory sauces, relishes, spreads, and seasonings</Description>

In this case, there is no Atom or AtomPub document to be seen – this is just the document that contains the specific piece of information we’re after. You can go ever further and just ask for the value itself:

URL: http://localhost:8080/owind.svc/Categories(2)/Description/$value
Response:

Sweet and savory sauces, relishes, spreads, and seasonings

Here, there’s no XML at all – just the raw value.

Likewise, if you’d like a count of entities in a collection instead of the entities themselves, you can use $count, e.g.

URL: http://localhost:8080/owind.svc/Categories/$count
Response:

8

Finally, if you’d like to get the set of navigation links between one entry and another without getting the set of related entries, you can do that with $links, e.g.

URL: http://localhost:53211/owind.svc/Categories(4)/$links/Products
Reponse:

<links xmlns="http://schemas.microsoft.com/ado/2007/08/dataservices">
  <uri>http://localhost:53211/owind.svc/Products(11)</uri> 
  <uri>http://localhost:53211/owind.svc/Products(12)</uri> 
  <uri>http://localhost:53211/owind.svc/Products(31)</uri> 
  <uri>http://localhost:53211/owind.svc/Products(32)</uri> 
  <uri>http://localhost:53211/owind.svc/Products(33)</uri> 
  <uri>http://localhost:53211/owind.svc/Products(59)</uri> 
  <uri>http://localhost:53211/owind.svc/Products(60)</uri> 
  <uri>http://localhost:53211/owind.svc/Products(69)</uri> 
  <uri>http://localhost:53211/owind.svc/Products(71)</uri> 
  <uri>http://localhost:53211/owind.svc/Products(72)</uri> 
</links>

Here, we’re not getting back a feed or an entry, but rather a set of links for the Product entries associated with a specific Category.

So, if you happen to know the unique identifiers of the things you’re after, you can navigate to them directly or anything related to them, whether it’s a set of associated entries in a feed, a property, the value of a property or even the count of entries, simply by using the appropriate URL format (and you can keep going arbitrarily deep).

Query Options

In addition to the value and the count, OData defines other URL options to customize the output of your query. For example, $orderby allows you to order a query result by one or more properties in ascending or descending order:

 http://localhost:8080/owind.svc/Categories?$orderby=CategoryName desc

In this case, we’re ordering by the CategoryName property of each category in descending order. If we want to combine query options, we can do so using the standard “&” query string operator:

 http://localhost:8080/owind.svc/Categories?$orderby=CategoryName desc&$top=4

In this case, we’re shorting and then pulling off the top 4. Table 2 shows the full set of query options.

Query OptionDescription
$expandExpand related data inline with the results, e.g. Categories/$expand=Products would expand Product data inline with each Category entry.
$filterA Boolean expression for whether a particular entry should be included in the feed, e.g. Categories?$filter=CategoryName eq 'Produce'. The Query Expression section describes OData expressions.
$formatOne of “atom” (the default), “json” or “xml” depending on how you’d like your data returned to you.
$inlinecountIncludes the number of entries without the filter applied as a count element on the feed itself, e.g. Categories?$top=4&$inlinecount=allpages will return 8 with the default Northwind database installed.
$orderbyOne or more comma-separated expressions with an optional “asc” (the default) or “desc” depending on the order you’d like the values sorted, e.g. Categories?$orderby=CategoryName desc.
$selectLimit the properties on each entry to just those requested, e.g. Categories?$select=CategoryName,Description.
$skipHow many entries you’d like to skip, e.g. Categories?$skip=4.
$skiptokenSkips entries up to one past the one that matches the unique key. Used for paged access into a feed. Best used with $orderby, e.g. Categories?$orderby=CategoryID&$skiptoken=5 will skip everything ‘til it reaches the CategoryID of 5, then return entries past that.
$topReturn entries from the top of the feed, e.g. Categories?$top=4

Table 2: OData Query Options

And as if that weren’t enough, you can ask an OData service to evaluation expressions for you.

Filter Expressions

OData packs a lot of power into the URL format for querying. We’ve already seen that it uses $-prefixed keywords for special operations and query options. The expression language will similarly look a little strange because characters common in other expression languages, like + and &, already have meanings in the URL syntax. To avoid stepping on URL toes, OData defines an expression language for filtering based on words instead of symbols, e.g.

http://localhost:53211/owind.svc/Categories?$filter=(CategoryID add 4) eq 8

This example returns a single entry feed where the CategoryID is 4. The filter expressions built into OData are listed in Table 3.

OperationExampleDescription
Grouping(x add 4) eq 3Used to make clear or override default operator precedence (which I can never remember anyway).
Member accessCategories(3)/DescriptionAccess of entity by ID or property by name.
Method callstartswith(Description, “Bread”)Call a built in method.
Negate-xChange numeric sign.
Notnot xLogical not.
Andx and yConditional and.
Orx or yConditional or.
Multiplyx mul yMultiplication.
Dividex div yDivision.
Modulox mod yRemainder.
Addx add yAddition.
Subtractx sub ySubtraction.
Less thanx lt yComparison.
Greater thanx gt yComparison.
Less than or equalx le yComparison.
Greater than or equalx ge yComparison.
Equalsx eq yComparison.
Not equalsx ne yComparison.
Type coercioncast(T), cast(x, T)Perform a type coercion if possible.
Type comparisonisof(T), isof(x, T)Whether targeted instance can be converted to the specified type.
String literal'a string'Instance of the String type.
DateTime literaldatetime'2009-12-26T21:23:38Z'Instance of the DateTime type.
Decimal literal

4.5m

4.5M

Instance of the Decimal type.
GUID literalguid'1225c695-cfb8-4ebb-aaaa-80da344efa6a'Instance of the GUID type.
Single literal

4.5

4.5f

4.5F

4.5e9

Instance of the Single type.
Double literal

4.5

4.5d

4.5D

4.5e21

Instance of the Double type.
Integer 16 literal

16

-16

Instance of the Integer16 type.
Integer 32 literal

32

-32

Instance of the Integer32 type.
Integer 64 literal

64

-64

Instance of the Integer64 type.
Binary literal

X'FRwvAAI...'

binary'FRwvAAI...'

Base64 encoded byte stream.
Null literalNullDistinguished value meaning the absence of a value.
Byte literalFFInstance of the Byte type.
Boolean literal

True

false

1

0

Instance of the Boolean type.
Time literaltime'P120D'Instance of the Time type.
DateTimeOffset literaldatetimeoffset' 2002-10-10T12:00:00-05:00'Instance of the DateTimeOffset type.

Table 3: OData Expressions

You’ll notice as part of the list of operations that there is one for method calls, which is also part of the OData query expression language for $filter.

Filter Methods

OData defines several methods to be used as part of your query strings, e.g.

http://localhost:53211/owind.svc/Categories?$filter=startswith(CategoryName, 'Sea')

As you might imagine, this example filters the results based on the first three letters of the CategoryName of each Category. The full list of methods is shown in Table 4.

MethodExampleDescription
endswithendswith(x, 'foo')Whether the end of the first parameter value matches the second parameter value.
indexofindexof(x, 'foo')Index of the first occurrence of the second parameter value in the first parameter value or -1 otherwise.
replacereplace(x, 'foo', 'bar')String value with all occurrences of the second parameter value replaced by the third parameter value in the first parameter value.
startswithstartswith(x, 'foo')Whether the beginning of the first parameter values matches the second parameter value.
tolowertolower(x)String value with the contents of the parameter value converted to lower case.
touppertoupper(x)String value with the contents of the parameter value converted to upper case.
trimtrim(x)String value with the contents of the parameter value with all leading and trailing white-space characters removed.
substring

substring(x)

substring(x, y)

String value starting at the character index specified by the second parameter value in the first parameter string value.
substringof

substringof(x)

substringof(x, y)

Whether the second parameter string value occurs in the first parameter string value.
concat

concat(x)

concat(y)

String value which is the first and second parameter values merged together with the first parameter value coming first in the result.
lengthlength(x)The number of characters in the specified parameter value.
yearyear(d)The year component value of the parameter value.
monthmonth(d)The month component value of the parameter value.
dayday(d)The day component value of the parameter value.
hourhour(d)The hour component value of the parameter value.
minuteminute(d)The minute component value of the parameter value.
secondsecond(d)The second component value of the parameter value.
roundround(x)The nearest integral value to the parameter value, following the rules defined in IEEE754-2008.
floorfloor(x)The largest integral value less than or equal to the parameter value, following the rules defined in IEEE754-2008.
ceilingceiling(x)The smallest integral value greater than or equal to the parameter value, following the rules defined in IEEE754-2008.

Table 4: OData QueryMethods

Functions

In addition to the entities and query methods, both of which you can use as part of $filter queries, you can also form requests to functions exposed from a service endpoint. If you look at the CSDL (via $metadata), you can see that functions are denoted differently than entities:

<edmx:Edmx ...>
  <edmx:DataServices ...>
    ...
    <Schema Namespace="owind" ...>
      <EntityContainer Name="NorthwindEntities" ...>
        <FunctionImport Name="ResetCategories" m:HttpMethod="GET" />
        <EntitySet Name="Categories" EntityType="NorthwindModel.Category" />
        ...
      </EntityContainer>
    </Schema>
  </edmx:DataServices>
</edmx:Edmx>

The difference between a function and an entity is that an entity has up to four operations available to it, depending on how you call it, i.e. create, read, update and delete (aka CRUD). A function, on the other hand, is just one operation to be invoked.

In this case, our function is named ResetCategories and is invoked with the HTTP method GET. It could also be a POST, depending on what the service supports, which really only matters if you’re sending arguments along. In this case, we aren’t. In fact, there’s not even any return type associated with this method, so a request like the following will yield a result with no content:

URL: http://localhost:53211/owind.svc/ResetCategories
Response:

HTTP/1.1 204 No Content
Server: ASP.NET Development Server/10.0.0.0
Date: Thu, 04 Feb 2010 04:17:13 GMT
X-AspNet-Version: 4.0.21006
DataServiceVersion: 1.0;
Cache-Control: no-cache
Content-Length: 0
Connection: Close

Notice that there is a return code in the 200s, so it was a successful call, but it’s 204, indicating that there was no content returned. The following slightly more complicated function declaration returns a value:

<FunctionImport Name="CategoryUniqueNameCount"
                ReturnType="Edm.Int32"
                m:HttpMethod="GET" />

In this case, the return type is a 32-bit integer, so in the event of a successful result, we’ll see a response like this:

URL: http://localhost:53211/owind.svc/CategoryUniqueNameCount
Response content:

<CategoryUniqueNameCount
  p1:type="Edm.Int32"
  xmlns:p1="http://schemas.microsoft.com/ado/2007/08/dataservices/metadata"
  xmlns="http://schemas.microsoft.com/ado/2007/08/dataservices">
  9
</CategoryUniqueNameCount>

Notice that instead of returning the content in Atom, the result is wrapped in a custom XML element with the same name as the function and decorated with the appropriate namespaces so that the type of the result is evident. If you’d just like the raw value, you can filter the results of a function just like the results of a query, e.g.

URL: http://localhost:53211/owind.svc/CategoryUniqueNameCount/$value
Response content:

9

Functions can return collections of things as well:

<FunctionImport Name="CategoryUniqueNames"
                ReturnType="Collection(Edm.String)"
                m:HttpMethod="GET" />

In this case, the function returns a collection of strings. However, that collection is not returned in Atom format like an AtomPub collection:

URL: http://localhost:53211/owind.svc/CategoryUniqueNames
Response content:

<CategoryUniqueNames xmlns="http://schemas.microsoft.com/ado/2007/08/dataservices">
  <element>Beverages</element> 
  <element>Condiments</element> 
  <element>Confections</element> 
  <element>Dairy Products</element> 
  <element>Dessert Topping</element> 
  <element>Grains/Cereals</element> 
  <element>Meat/Poultry</element> 
  <element>Produce</element> 
  <element>Seafood</element> 
</CategoryUniqueNames>

Again, the result is wrapped in an XML element with a custom name, with each collection item wrapped in an element. Since the result is a collection of strings, the type is left off (recall that the default type is string). If it wasn’t a collection of strings, each element would be marked with a type. In addition to the primitive types, you can also return entity types and collections of entity types:

<FunctionImport Name="CategoriesLikeName" EntitySet="Categories"
                ReturnType="Collection(NorthwindModel.Category)"
                m:HttpMethod="GET" />

Here the return type is a collection of entity types and the content body looks like so:

URL: http://localhost:53211/owind.svc/CategoriesLikeName

Response content:

<feed ...>
  <title type="text">CategoriesLikeName</title>
  <id>http://localhost:53211/owind.svc/CategoriesLikeName</id>
  <updated>2010-02-04T04:54:10Z</updated>
  <link rel="self" title="CategoriesLikeName" href="CategoriesLikeName" />
  <entry>
    <id>http://localhost:53211/owind.svc/Categories(1)</id>
    <title type="text"></title>
    <updated>2010-02-04T04:54:10Z</updated>
    <author>
      <name />
    </author>
    <link rel="edit" title="Category" href="Categories(1)" />
    <link rel="http://schemas.microsoft.com/ado/2007/08/dataservices/related/Products"
      type="application/atom+xml;type=feed" title="Products"
      href="Categories(1)/Products" />
    <category term="NorthwindModel.Category"
      scheme="http://schemas.microsoft.com/ado/2007/08/dataservices/scheme" />
    <content type="application/xml">
      <m:properties>
        <d:CategoryID m:type="Edm.Int32">1</d:CategoryID>
        <d:CategoryName>Beverages</d:CategoryName>
        <d:Description>Soft drinks, coffees, teas, beers, and ales</d:Description>
        <d:Picture m:type="Edm.Binary">...</d:Picture>
      </m:properties>
    </content>
  </entry>
  ...
</feed>

In this case, notice we’re back to the Atom format for our output because that’s the mapping the OData specification mandates. Again, function results can be further filtered (depending on what support the service provides), so if you wanted, you could something like this to narrow down the results, just as if it were a query over a collection of entities:

URL: http://.../owind.svc/CategoriesLikeName?$filter=startswith(CategoryName, 'Sea')

However, there are lots of operations that OData doesn’t mandate as part of the specification, which is one of the reasons why functions are handy – they can do things on the server-side that aren’t part of the spec. In some cases, just like the built in query operations, you’ll want to provide arguments to these operations, which are specified in the CSDL like so:

<FunctionImport Name="CategoriesLikeName" EntitySet="Categories"
                ReturnType="Collection(NorthwindModel.Category)"
                m:HttpMethod="GET">
  <Parameter Name="name" Type="Edm.String" Mode="In" />
</FunctionImport>

Calling a function with parameters can be done using GET or POST, but the parameters are always packed into the query string syntax like so:

http://localhost:53211/owind.svc/CategoriesLikeName?name='Sea'

At this point, it’s clear why functions look like queries, even including the ability to pass parameters and filter the results. And, like queries, function results come back as XML by default. If that isn’t the format you want, you can switch it to something else, like perhaps JSON.

JSON Format

As you may recall, one of the query options, $format, brings up a whole interesting topic: result format. By default, data comes back as Atom, but if you like you can switch the output format using the $format option into a non-Atom dialect of XML (boring) or JSON, the JavaScript Object Notation (fun!). The JSON format is especially interesting if you’re using JScript from inside the browser.

Modern browsers today provide a something called the “XMLHttpRequest” object which allows you to make a request of data from a data service. By default, the results will come back as XML, but if you set the Accept type to “application/json”, it will come back in the native format of JavaScript, making it very easy to use. Even better, if you’re using a modern JavaScript library, like jQuery, you can just form a request and the Accept header will be set and the JSON data parsed automatically, integrating the results of an OData request seamlessly into your JavaScript program, like so:

<html>
<head>
    <title>OData JSON Test</title>
    <script src="/Scripts/jquery-1.3.2.js" type="text/javascript"></script>
</head>
<body>
    <script type="text/javascript">
        $(document).ready(function () {
            $("#foo").text("fetching...");
            var url = "http://localhost:8080/owind.svc/Categories/";
            $.getJSON(url, function (result) {
              $("#foo").text(result.d[0].CategoryName);
            });
        });
    </script>
    <p id="foo">loading page...</p>
</body>
</html>

Here, we’re using jQuery’s getJSON function on a URL from our sample data service. When the results come back, our function will be called and we’ll set the text of an HTML <p> element to the name of the first Category that’s returned[xiii]. The results are similarly simple, as Figure 1 shows.

Figure 1: OData in JSon from the browser

The call to getJSON translates into a call on the browser’s underlying XMLHttpRequest object, which yields an HTTP request/response pair like this one:

Request:
GET /owind.svc/Categories/ HTTP/1.1
x-requested-with: XMLHttpRequest
Accept-Language: en-us
Referer: http://localhost:54087/
Accept: application/json, text/javascript, */*
Accept-Encoding: gzip, deflate
User-Agent: Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; WOW64; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; .NET4.0C; .NET4.0E; InfoPath.3; MS-RTC LM 8)
Host: localhost:8080
Connection: Keep-Alive
Cookie: ...
Response:

HTTP/1.1 200 OK
Server: ASP.NET Development Server/10.0.0.0
Date: Sat, 27 Feb 2010 19:49:45 GMT
X-AspNet-Version: 4.0.21006
DataServiceVersion: 1.0;
Content-Length: 120605
Cache-Control: no-cache
Content-Type: application/json;charset=utf-8
Connection: Close
{
  "d" : [
    {
      "__metadata": {
        "uri": "http://localhost:8080/owind.svc/Categories(1)",
        "type": "NorthwindModel.Category"
      },
      "CategoryID": 1,
      "CategoryName": "Beverages",
      "Description": "Soft drinks, coffees, teas, beers, and ales",
      "Picture": "FRwvAAI...",
      "Products": {
        "__deferred": {
          "uri": "http://localhost:8080/owind.svc/Categories(1)/Products"
        }
      }
    }, ...
  ]
}

If you’re unfamiliar with the JSON format, it’s just the JavaScript data initialization format. In this case, the HTTP response content is parsed and turned into data accessible directly from JavaScript. Notice that the top-level root of the data returned in this OData response is “d” (and it always will be, according to the OData specification), so I can just reach into the array of entities returned and dig through the properties as you’d expect, e.g. CategoryID, CategoryName, Description, etc.

You’ll also notice two things of interest that you won’t see in normal JSON data. The first is the __metadata property, which provides the name and URL to the type data for the entity in case you’d like access to it in your JavaScript program (most JavaScript programmers won’t want it, which is why it’s got the double underscore in front of the name). Also, because we didn’t use the $expand query option to bring down the associated products with the category, instead of the data under the Products property, we have the URL to the associated products inside the __deferred property.

As you can see, the jQuery getJSON function does everything you need to get results from a data service without any muss or fuss. Similarly, the OData community has built a number of client libraries for making it just as easy to get to OData results as in JavaScript.

Client Libraries

While it’s true that OData is accessible from any environment that supports HTTP and XML (or JSON), there are a number of OData client libraries across various platforms to make you even more productive:

I won’t bother you by showing you examples of every OData client library or tool, especially as more are being developed all the time, but let’s take a look at what it takes to create the same Category we did using XML and HTTP using the .NET Data Services client library built into .NET 4.0.

To start, in a console application (new or used, it doesn’t matter to me), right-click on the name of the project in the Solution Explorer and choose Add Service Reference, as shown in Figure 2.

Figure 2: Add Service Reference dialog

Notice in Figure 2that we’ve entered the URL to the OData service example we’ve been playing with. After pressing Go, Visual Studio 2010 will dig out the service document to be able to populate the list of collections exposed by that service. In the Namespace box, you’re free to enter whatever you like, but for this example, I chose “NorthwindService”.

Pressing OK will cause VS2010 to dig out the CSDL metadata associated with the service and generate a file called “Reference.cs” (or “Reference.vb” depending on the language you’re using), which you can find under the service you generated in the Service References folder in the Solution Explorer, as shown in Figure 3[xiv].

Figure 3: An OData service reference inside a VS2010 project

The top-level class generated by the Add Service Reference project item template is named after the namespace found in the metadata. It derives from the DataServiceContext base class and represents the container for each collection found in the service document. It doesn’t maintain a constant connection between the client and the service because HTTP doesn’t work like that, but each client-side service context instance is going to keep track of the changes you make, like adding new entries, updating them, deleting them, etc. until you signal it’s time to push those changes to the service. In fact, under the covers, it will generate XML and send it via HTTP just like we did in our .NET code previously, but what we have to write to get this to happen is far simpler:

using System;
using System.Data.Services.Client;
using System.Linq;
using System.Net;
using DataServicesClient.NorthwindService.NorthwindModel;
...
static Category CreateCategory() {
  // Point the service context at the correct URL
  NorthwindEntities nwind =
    new NorthwindEntities(new Uri(@"http://localhost:53211/owind.svc/"));

  // Create the new category -- this will eventually turn into an entry document
  var category = new Category() {
    CategoryName = "Dessert Topping", Description = "Toppings for desserts" };

  // Add the new category to the service context so it knows to track it
  nwind.AddToCategories(category);

  // Compose the XML and POST the new entry via HTTP
  DataServiceResponse response = nwind.SaveChanges();

  // Check for completion. It could be a batch operation handled all at once,
  // which is why the response exposes a set of operation responses for each
  // service response
  if (response.First().StatusCode == (int)HttpStatusCode.Created) {
    Console.WriteLine("Created category w/ CategoryID= {0}", category.CategoryID);
    return category;
  }
  else {
    Console.WriteLine("Category creation failed");
    return null;
  }
}

In this code, we create an instance of the service context we generated from the service metadata, create a new category and add it to the collection of categories exposed from the service context and save our changes. This code is far less than what we had to write before, but does the same thing on the wire and the Northwind service can’t tell the difference one way or the other.

Each client library is built to fit into the platform as nicely as possible. The .NET client library for OData, in addition to supporting the standard CRUD operations and data binding for GUI frameworks like the Windows Presentation Foundation or ASP.NET, also translates LINQ into the OData query language. For example, if you were to write the following code:

var categories =
  from c in nwind.Categories where c.CategoryName.StartsWith("C") select c;
foreach (var category in categories) { Console.WriteLine(category.CategoryName); }

You might expect that the .NET StartsWith function would throw things off and force LINQ to do the filtering on the client side after all categories have been fetched. But that’s not what we want. We want a filtered URL to be generated so that the query can be performed on the service side like so:

http://localhost:53211/owind.svc/Categories?$filter=startswith(CategoryName, 'C')

In fact, that’s exactly what happens. If you like, you can watch the communication between the client and the server on the wire using tools I mentioned before (Fiddler or TcpTrace). On the other hand, if you’re lazy like I am and you’re debugging your .NET code, you can hang your cursor over the “categories” various, which represents our LINQ results and see the URL that the Data Services library created for you even before it has been executed, as shown in Figure 4.

Figure 4: You can see the generated OData query URL while debugging in VS2010

This is exactly the kind of thing I want a platform-specific OData client library to do for me. However, as of this writing, the Data Services client doesn’t support calling OData functions from the generated code. Luckily, .NET has great support for HTTP and XML, so it’s not hard to form the queries to call OData functions (in fact, it looks just like the .NET code I showed earlier).

Server Implementations

In the same way that there are several client toolkits for consuming OData, there are several server-side implementations as well:

  • OData Test Service: A read-write service to let you get a feel for OData. Live Endpoint.
  • Read-Only Northwind Service: A read-only service that exposes the Northwind database via OData. Live Endpoint.
  • Vancouver Street Parking: A data service that exposes Vancouver street parking information. Live Endpoint.
  • Open Government Data Initiative: A growing variety of data from US government agencies. Live Endpoint.
  • The City of Edmonton Open Data Catalogue: Public data from the city of Edmonton. Live Endpoint.
  • Microsoft .NET Framework 3.51: the WCF Data Services framework is available as a separate download for .NET 3.x.
  • Microsoft .NET Framework 4.0: the WCF Data Services framework built into .NET 4.0 (in release candidate as of this writing).
  • SharePoint 2010: any data you’ve got on SharePoint as of version 2010 can be manipulated via the OData protocol, which makes the SharePoint developer API considerably simpler.
  • IBM WebSphere: the WebSphere eXtreme Scale REST data service supports OData.

Again, I won’t make you look at every one of the server implementations of OData, but let’s take a quick look at exposing a database using WCF Data Services. In fact, let me show you how I got Northwind exposed as an OData service.

As a start, you need an ASP.NET web application. Next, right-click on the project in the Solution Explorer in VS2010 and choose to add an ADO.NET Entity Data Model. Figure 5 shows you how.

Figure 5: Adding an EDMX file to model the entities you'd like to expose as OData

You may pick the name of your choice for the resulting EDMX file and press the Add button, after which you’ll be asked whether you’d like an empty model or whether you’d like to generate the model from an existing database, as Figure 6 shows.

Figure 6: Choosing the model contents

Choosing “Generate from database” and pressing Next lets you choose or create the connection to the database, as shown in Figure 7.

Figure 7: Choosing your data connection

You’ll notice, in the midst of a very involved connection string, the mention of a .csdl file. This is the exact same CSDL that is the core of EDM. The .ssdl file is the Storage Schema Definition Language and the .msl file the Mapping Schema Language, which describe the storage layer and the mapping to the storage layer from the conceptual model that the CSDL defines[xv]. All of this is generated from the database objects you choose on the next screen: Figure 8.

Figure 8: Choosing your database object

Here we’ve chosen all the tables, but we could bring in views and/or storage procedures as well. Finally, pressing Finish gets you an .edmx file and a .Designer.cs file, which provides the .NET object context class used as the root for accessing your data in .NET. This is just like the Data Services service context class that we generated using a URL to a service document, except that instead of talking HTTP to an OData service, the EDMX-generated data context class talks to the ADO.NET provider, which in turn talks whatever protocol it needs to get to the database server, e.g. SQL Server exposes its data via TDS (Microsoft’s Tabular Data Stream protocol, the communication format to talk to SQL Server).

In our case, we get a generated NorthwindEntities class which derives from the Data Services ObjectContext base class, as Figure 9 shows.

Figure 9: The Data Services generated ObjectContent class

If we wanted, we could use the generated class to write data access code against the database from which we just generated our model, but we don’t want to do that. Instead, we want to expose it as an OData service itself. To do that, right-click on the project in the Solution Explorer again and choose Add | New Item, then WCF Data Service, choose a name you like and press Add (Figure 10).

Figure 10: Adding a Data Services endpoint

The generated .svc (service) file will have a C# or VB file that you need to edit, providing your generated ObjectContext-derived class along with the permissions you’d like to use on the entities you choose to expose from your OData service. Here, I’ve chosen to expose them all with read/write access (although I’ll understand if you want to be more choosey in your production software):

using System.Data.Services;
using System.Data.Services.Common;

namespace owind {
  public class owind : DataService<NorthwindEntities> {
    public static void InitializeService(DataServiceConfiguration config) {
      config.SetEntitySetAccessRule("*", EntitySetRights.All);
      config.DataServiceBehavior.MaxProtocolVersion = DataServiceProtocolVersion.V2;
    }
  }
}

That’s it. When you surf to the .svc file exposes from your web application, you’ll see a service document as you’d expect and poking at the service expecting it to act like an OData service should not disappoint you, as Figure 11 shows.

Figure 11: The OData service document for the Northwind database

As you can see, WCF Data Services makes a handy server and client-side library for managing and manipulating your OData services.

Where are we?

As you’ve seen, the Open Data protocol (OData) is an application of Atom and AtomPub for typed CRUD operations and rich queries in a standardized way that enables a large number of services to expose their data, relational or not, and a large number of clients to consume it. While this technology has its roots in Microsoft technology, all the specifications are released under the Open Source Promise to enable anyone to play in this space, which includes Microsoft tools like Visual Studio, Excel and SharePoint, as well as non-Microsoft solutions like WebSphere, JavaScript and the US government. Data on the web wants to be open. OData is here to help us make it so.

 


[i] The Open Data protocol specifications are available at http://odata.org.

[ii] WebSphere’s support for OData: http://ibm.com/developerworks/websphere/downloads/xs_rest_service.html

[iii] The Open Government Data Initiative: http://www.microsoft.com/industry/government/opengovdata/

[iv] WCF Data Services: http://msdn.microsoft.com/en-us/data/bb931106.aspx

[v] Atom was inspired by RSS (the Really Simple Syndication protocol) as a more flexible and more rigorous standardized replacement. Most blog-based tools support both RSS and Atom, although AtomPub and OData only support Atom itself.

[vi] Specifically, it must be in the form of an IRI (International Resource Identifier), which is an international version of the URI (Uniform Resource Identifier), which is like a URL (Uniform Resource Locator) that you see in the address bar of your browser, but a URI doesn’t have to reference a resource on the network.

[vii] An entry element can be at the root – it doesn’t have to be a child under a feed element.

[viii] The AtomPub specification says that anything in the 200s indicates a successful request, so a more robust client will take greater care with HTTP status codes.

[ix] The main Open Data protocol specification: http://odata.org/docs/[MC-APDSU].htm.

[x] The OData protocol overview: http://odata.org/developers/protocols/overview.aspx.

[xi] To be fair, AtomPub does define the idea of partial feeds and a mechanism for navigating between sets to get the full set of entries. However, it does not allow for filtering or projection, forcing the client to handle those chores.

[xii] The OData metadata specification: http:// odata.org/docs/[MC-CSDL].htm

[xiii] In a real-world application, if we’re only using one property on one entity, we should form our URL to include filtering and projection, but this makes for a nice demo of the data returned from an OData query.

[xiv]  You may have to press the Show All Files button at the top of the Solution Explorer to be allowed to look at your own files, but I won’t judge that particular user experience decision made by my corporate brothers and sisters.

Microsoft is conducting an online survey to understand your opinion of the MSDN Web site. If you choose to participate, the online survey will be presented to you when you leave the MSDN Web site.

Would you like to participate?