• Article

March 2013

Volume 28 Number 03

Modern Apps - Data Access and Storage Options in Windows Store Apps

By Rachel Appel | March 2013

Rachel AppelManaging data is a critical part of app development. Whether it’s a game, news, travel or fashion app, it’s always all about the data. Modern apps often need to manage data scattered throughout multiple, disparate locations and in countless formats. I’ll discuss the various data storage options and data access APIs available for building Windows Store apps, in all languages, as well as data management strategies for both content and configuration.

Data Management and Storage Considerations

As an app developer, you need to determine your app’s data requirements prior to starting your project, because changing the underlying architecture causes a lot of rework. You might have an existing data source, in which case the decision is made for you, but with a greenfield project, you must think about where to store the data. Your two options are on the device or at a remote location:

  • Local: Usually this data is in a file or local database, but in Windows 8, you can now treat other apps as sources for data by using the built-in File Picker or contracts. In JavaScript apps, Web Storage and the Indexed Database (IndexedDB) API are also available as local data sources.  
  • Remote: This data could be in the cloud using Azure SkyDrive or any remote HTTP endpoint that can serve JSON or XML data, including public APIs such as Facebook or Flickr.

The size of the data often determines whether the data is local or remote; however, most modern apps will use data from both sources. This is because smaller, more mobile devices such as slates, tablets and phones are the norm, and they don’t usually have much storage space. Despite that, they still need data to function correctly when offline. For example, the Surface, as with many portable devices, comes in 32GB and 64GB models. Simple text-based data such as JSON isn’t usually large, but relational databases and media data (such as images, audio and video) can fill up a device quickly.

Let’s take a look at the various local and remote options for storing app content data.

Web Storage

It might sound like Web Storage (bit.ly/lml0Ul) is simply storage on the Web, but it isn’t. Instead, Web Storage, an HTML5 standard, is a great way to keep app data on the client, locally. Both Windows Store apps as well as plain old HTML pages support Web Storage. There’s no database setup required and no files to copy, as Web Storage is an in-memory database.

Web Storage is accessible via JavaScript through one of the two following properties of the window object:

  1. localStorage: Local data that’s persistent after the app terminates and is available to future app instances.
  2. sessionStorage: Also local data; however, sessionStorage is destroyed when a Windows Store app terminates execution.

You can store data from simple types to complex objects in Web Storage by attaching dynamic properties to either the sessionStorage or localStorage variables. The dynamic properties are a key/value pair with syntax similar to this:

sessionStorage.lastPage = 5;
WinJS.xhr({ url: "data/data.json" }).then(function (xhr) {
  localStorage.data = xhr.responseText;
};

The lastPage property exists until the app terminates because it’s part of sessionStorage, while the data property of localStorage persists past the lifetime of the app.

Being able to persist data locally between app sessions makes Web Storage an excellent choice for supporting offline scenarios. Small data is also more suited for offline support. Because JSON data is compact, it’s easy to stuff entire JSON datasets into the 5MB of space provided by Web Storage and have plenty of space left over for some media data.

Because Web Storage is an HTML5 standard, it’s only available in Windows Store projects built with JavaScript.

IndexedDB

Another standard in the HTML5 family is IndexedDB (bit.ly/TT3btM), which is a local data store for large, searchable and persistent data sets. As a component of HTML5, you can use IndexedDB in client Web apps for browsers as well as Windows Store apps. IndexedDB stores items in an object database and is extremely flexible because you can store any kind of data from text to Binary Large Objects (BLOBs). For example, multimedia files tend to be quite large, so storing audio and video in IndexedDB is a good choice.

Because IndexedDB is an object database, it doesn’t use SQL statements, so you must access data through an object-oriented-­style syntax. Interaction with an IndexedDB data store is through transactions and cursors, as shown here:

var dataStore = "Datastore";
var trn = db.transaction(dataStore, IDBTransaction.READ_ONLY);
var store = trn.objectStore(dataStore);
trans.oncomplete = function(evt) { // transaction complete };
var request = store.openCursor();
request.onsuccess = function(evt) {
  var cursor = evt.openCursor();
};
request.onerror = function(error) { // error handling };

Because IndexedDB specializes in really big data, using it for small item storage causes it to behave inefficiently, making Web Storage a much better choice for bite- (or byte-) sized local data. IndexedDB is also well-matched for content data, but ill-suited for app configuration data.

SQLite

SQLite (bit.ly/65FUBZ) is a self-contained, transactional, relational and file-based database, which requires no configuration and doesn’t need a database administrator to maintain it. You can use SQLite with any Windows Runtime (WinRT) language, and it’s available as a Visual Studio extension. While SQLite works well in JavaScript apps, you need to obtain the SQLite3 WinRT wrapper (bit.ly/J4zzPN) from GitHub before using it.

Developers with backgrounds in ASP.NET or Windows Forms gravitate to relational databases, but a relational database management system (RDBMS) isn’t always the best choice when writing modern apps, due to space issues on mobile devices as well as the varied types and formats of data, especially multimedia. Because SQLite is a relational database, it makes sense for those apps that need relational and transactional behaviors. This means SQLite is great for line-of-business (LOB) apps or data-entry apps, and can also be a repository for local, offline data originally obtained from an online source.

If the SQLite database becomes too large for portable devices, you can move it to a server or cloud location. The code won’t change much, because the SQLite3 library uses a traditional connection and Create/Read/Update/Delete (CRUD) objects similar to the following code:

// C# code that opens an async connection.
var path =
  Windows.Storage.ApplicationData.Current.LocalFolder.Path + @"\data.db";
var db = new SQLiteAsyncConnection(path);
// JavaScript code that opens an async connection.
var dbPath =
  Windows.Storage.ApplicationData.current.localFolder.path + '\\data.db;
SQLite3JS.openAsync(dbPath).then(function (db) {
  // Code to process data.
});

As you can see, using SQLite is just like using other SQL databases. Limits on SQLite databases go as high as 140TB. Keep in mind that very large data often warrants professional database administration for the best possible data integrity, performance and security. Most DBAs and developers who work with relational databases prefer a GUI tool to create and manage database objects or run ad hoc queries on the data, and the Sqliteman (bit.ly/9LrB1o) admin utility is ideal for all basic SQLite operations.

If you’re porting existing Windows desktop apps written in Windows Forms, Windows Presentation Foundation (WPF) or Silverlight, you might already be using SQL Server Compact (SQL CE). If this is the case, you can migrate SQL CE databases to SQLite with the ExportSqlCE (bit.ly/dwVaR3) utility and then use Sqliteman to administer them.

Files as Data and the File API

Why bother with a database at all, especially if users of your app want to stick with the files they already have? This is especially true of photos and documents. The File API goes beyond simply providing a navigator to directories and files; it gives the user the ability to choose an app as a file location. This means apps can talk to each other and exchange data. A File Open Picker can interact with Bing, camera, or photo apps as well as regular directories and named locations such as My Documents, Videos, or Music. Sharing data so easily between apps, services and personal files is a first-rate feature of Windows.

Many OSes have a mechanism for registering types of files that apps intend to use alongside the ability to launch those apps when a user interacts with an icon in the OS. In Windows, this is called a file association. Windows 8 takes this concept further by allowing apps to talk to each other through a system-wide feature called contracts. One way to implement a contract is through a File Picker. Notice that the following code is similar to the File Dialog APIs from desktop apps or earlier Windows versions (note: some code has been left out of this example for brevity; for a more thorough examination of the FileOpenPicker class, see bit.ly/UztmDv):

fileOpen: function () {
  var openPicker = new Windows.Storage.Pickers.FileOpenPicker();
  openPicker.viewMode = Windows.Storage.Pickers.PickerViewMode.thumbnail;
  openPicker.suggestedStartLocation =
    Windows.Storage.Pickers.PickerLocationId.picturesLibrary;
  openPicker.fileTypeFilter.replaceAll([".png", ".jpg", ".jpeg"]);
  openPicker.pickSingleFileAsync().done(function (file) {
  // ...
  });
}

Choosing an app from a Windows 8 picker launches that app. For example, if the user selects Bing, the picker will launch the Bing app and then return the user’s image selection to your app.

Now that you’ve seen the local options, let’s look at the options for remote data.

Web Services and the ASP.NET Web API

Most developers are familiar with consuming and modifying data with XML Web services because they date back to the days of the Microsoft .NET Framework 1.x. The main advantage in using Web services is that the data lives at a central remote location and the apps from multiple devices can access the data any time while connected. Access to the underlying database is piped through a set of HTTP endpoints that exchanges JSON or XML data. Many public APIs such as Twitter or Flickr expose JSON or XML data that you can consume in Windows Store apps.

Web services come in a variety of flavors:

  • ASMX services: Use traditional ASP.NET to deliver data across HTTP.
  • Windows Communication Foundation (WCF) and Rich Internet Application (RIA) services: A message-based way to send data between HTTP endpoints.
  • OData: Open Data protocol, another API for transporting data over HTTP.
  • ASP.NET Web API: A new ASP.NET MVC 4 framework that makes it easy to build RESTful HTTP services that deliver JSON or XML data to apps or Web sites.

If you’re building back-end services from the ground up, the ASP.NET Web API streamlines the development process because it makes it easy to build services in a consistent RESTful fashion, so they can be readily consumed by apps. 

Regardless of the back-end service, if it’s over HTTP, you can use the HttpWebRequest and HttpWebResponse objects to communicate with a Web service via C#. In JavaScript apps, a WinJS XMLHttpRequest wrapper is fitted for asynchronous operations, which looks like the following code:

WinJS.xhr({ url: "data.json" }).then(function (xhr) {
  var items = JSON.parse(xhr.responseText);
  items.forEach(function (item) {
    list.push(item);
  })
});

Storage space isn’t usually an issue with Web services because a Web service is just the software that transports the data between endpoints. This means its underlying database can live anywhere, such as on a remote server, Web host or Azure instance.

You can host any of the previously noted Web services such as an ASP.NET Web API instance or a WCF service on Azure, along with the data itself.

SkyDrive

Don’t forget that SkyDrive (bit.ly/HYB7iw) is not only an option for data storage, but an excellent option. Think of SkyDrive as a non-storage storage option. Users can choose SkyDrive as a location in the cloud and access documents through File Open or Save pickers. Allowing users to save files to SkyDrive means zero worries about database management, and with 7GB of space per user, there’s plenty of room. SkyDrive users can also purchase more storage space.

The Live API (bit.ly/mPNb03) contains a fully featured set of RESTful SkyDrive APIs for reading and writing to SkyDrive. A call to SkyDrive to retrieve the list of shared items looks like this:

GET https://apis.live.netv5.0/me/skydrive/shared?access_token=ACCESS_TOKEN

A Microsoft.Live namespace allows C# developers to access the Live and SkyDrive APIs, while JavaScript developers can make RESTful calls with HTTP verbs POST or PUT. SkyDrive isn’t recommended for app configuration data.

Azure Mobile Services

Azure Mobile Services is a great option for those building cross- and multi-platform apps. Powered by Azure, this option offers more than just scalable storage; it offers push notifications, business logic management, an authentication API and a complete SDK. In addition to these features, an easy-to-use, Web-based administrative tool is provided. 

The Mobile Services SDK integrates with Windows Store, Windows Phone 8, iOS and Android apps. All of the major platforms are supported, and with the Mobile Services SDK, you can build out a working prototype in just minutes and be on your way to delivering data to multi-platform apps in no time.

Among the many items in the SDK, you can use the query object to query data from tables, similar to this:

var query = table.orderBy('column').read({ success: function(results) { ... }});

As you can see, the code is quite the same as any other API, so the learning curve is the same as the other options discussed here. You can access the SDK and Mobile Services using any Windows Store app language.

Application Data Management and Storage Options

All of the previously noted data storage and access options are for storing content, but as a developer, you also need to deal with configuration data for the app. This is the metadata that describes your app or the capabilities of its device, not the user’s data. Modern apps make use of both persistent application data (such as user preferences) as well as temporary metadata (such as the user’s last scroll position or trending search terms). These small yet effective conveniences ensure the best-possible experience for the user, so it’s important to build them into your app.

While some of this data belongs on the device, consider the fact that many apps work across multiple platforms and devices, so centralizing and synchronizing application data in Azure with the content data often makes sense.

A specific set of APIs for managing application data exists in an aptly named ApplicationData class in the Windows.Storage namespace, and the code looks similar to this:

var localSettings = Windows.Storage.ApplicationData.current.localSettings;
var roamingSettings = Windows.Storage.ApplicationData.current.roamingSettings;

You can store simple or complex objects in either the localSettings or roamingSettings properties.

Using either local or roaming settings is the preferred way to work with configuration data. Technologies such as IndexedDB, files or SkyDrive ordinarily aren’t good options for app configuration data, and SQLite makes sense here only if the app is already using it to store content. You must also consider what to do when the app is offline or disconnected from the Internet. In other words, some of your data needs to be cached, but without consuming too much disk space.

Content and Configuration

In summary, to implement a proper data architecture, you shouldn’t depend on the limited space on portable devices, so hosting data in the cloud is usually a good bet. But, if you have a legacy database, reusing it might be a requirement. Windows Store apps support a variety of structured and BLOB storage needs for both content and configuration.

If you’re building a Window Store app and would like assistance with choosing a data access strategy, I highly recommend that you check out the Generation App (bit.ly/W8GenAppDev) program. Generation App guides you through the process of building a Windows Store (or Windows Phone) app in 30 days by offering free technical and design consultations and assistance, along with exclusive tips and resources.

Get help building your Windows Store app!

Rachel Appel is a developer evangelist at Microsoft New York City. Reach her via her Web site at rachelappel.com or by e-mail at rachel.appel@microsoft.com. You can also follow her latest updates on Twitter at twitter.com/rachelappel.

Thanks to the following technical experts for reviewing this article: Scott Klein and Miriam Wallace
Miriam Wallace has been writing developer documentation for Windows for five years.

Scott Klein is an Azure technical evangelist for Microsoft. His career has focused on SQL Server for the past 20-plus years and thus focuses on Microsoft’s PaaS services for Azure. He lives in Redmond, blogs at scottlklein.com, and you can follow him on Twitter at twitter.com/sqlscott.