June 2011

Volume 26 Number 06

Windows Phone 7 - Sterling for Isolated Storage on Windows Phone 7

By Jeremy Likness | June 2011

The launch of Windows Phone 7 provided an estimated 1 million Silverlight developers with the opportunity to become mobile coders practically overnight.

Applications for Windows Phone 7 are written in the same language (C# or Visual Basic) and on a framework that’s nearly identical to the browser version of Silverlight 3, which includes the ability to lay out screens using XAML and edit them with Expression Blend. Developing for the phone provides its own unique challenges, however, including special management required when the user switches applications (called “tombstoning”) combined with limited support for state management.

Sterling is an open source database project based on isolated storage that will help you manage local objects and resources in your Windows Phone 7 application as well as simplify the tombstoning process. The object-oriented database is designed to be lightweight, fast and easy to use, solving problems such as persistence, cache and state management. It’s non-intrusive and works with your existing type definitions without requiring that you alter or map them.

In this article, Windows Phone 7 developers will learn how to leverage the Sterling library to persist and query data locally on the phone with minimal effort, along with a simple strategy for managing state when an application is deactivated during tombstoning.

Tombstoning Basics

Silverlight in the browser and on the phone runs in a special “security sandbox” that isolates the host system from the application’s runtime environment. The phone environment adds a layer of complexity because multiple applications can co-exist on the platform. While the underlying OS of the phone supports multitasking, third-party applications aren’t provided with access to this layer. 
Instead, applications have the opportunity to run “in the foreground” but can be quickly swapped out to make way for other applications, incoming phone calls or hardware features such as the Back button and search. When the application is deactivated, there’s a chance it may be “killed,” with the opportunity to be revived if and when the user navigates back. This is the process known as tombstoning.

The “Mango” update to Windows Phone 7 will limit tombstone scenarios by providing “fast application switching.” Applications will no longer be automatically tombstoned. While this functionality is already present in the Windows Phone 7 Developer Tools, it’s important to recognize that it doesn’t eliminate the tombstone scenario. Factors such as other running applications and available memory will influence whether the application is tombstoned.

The problem with tombstoning is that when the application is revived, new instances of the pages that are part of the application are created. Therefore, anything the user was working on when the event occurred—such as selecting an item from a pick list or entering text—is lost. It’s up to you, the developer, to maintain this state and restore it when the application returns to provide a seamless experience to the user. Imagine the confusion of a typical user who’s in the middle of filling out a form and clicks on search to look up a term, only to return and find the application has navigated to a separate, blank page!

Saving State on the Phone

Fortunately, Windows Phone 7 does provide several mechanisms for saving state. To work with applications on the OS, you’ll need to become familiar with these methods. The options include SQL CE (with the Mango update), a page state dictionary, the application state dictionary and isolated storage. I’ll focus on the last option, isolated storage, but a quick review of the first three will help shed light on why Sterling is useful on the phone.

SQL CE The Mango update will provide SQL CE, a compact version of the popular SQL Server database. The difference between this database and the other options listed here is that SQL is a relational database. It relies on special table formats that must be mapped from your classes and controls. Object-oriented databases can take the existing class structures—even when they include nested classes, lists and other types—and serialize them without any additional mapping or modifications.

Page State Each Page object provides a State property that exposes a dictionary that’s defined by key names and related objects. Any key or object may be used, but the object must be serializable, and then only shallow (top level) serialization occurs. In addition, the page state only allows for up to 2MB of data per page (4MB for the entire application) and can only be used after the start of the OnNavigatedTo and before the OnNavigatedFrom methods on the page. This limits the practical use to simple value types. The fact that it can only function within the page navigation methods makes it a poor choice for patterns such as Model-View-ViewModel (MVVM) that synchronize the view state using a separate ViewModel.

Application State This is also a dictionary. Like the page state, it takes a string key and an object value, and the object must be serializable. The application object on the phone has a Deactivated event called when tombstoning occurs, and an Activated event called when the application returns from the tombstone state. The application state can be accessed anytime between activation and deactivation. It’s accessed using the static PhoneApplicationService class (there’s a Current property that references a single application-wide instance). There’s no documented limit to the size of the application state dictionary, but at some point attempting to store too many items will result in an unhandled COM exception being thrown.

Isolated Storage This is by far the most flexible option for maintaining state. Isolated storage isn’t unique to the phone, and in fact works almost exactly the same way in Silverlight and in the Microsoft .NET Framework runtime. Isolated storage provides a layer of abstraction from the host file system, so instead of dealing with direct file storage, you interface with an indirect mechanism that provides folders and files in an isolated sandbox. In Windows Phone 7, that sandbox is isolated to the level of your phone application. You may access the storage anywhere from within the application, but not from any other application on the phone.

Isolated storage also has some powerful advantages. While it does provide a settings dictionary similar to the page and application settings described earlier, it also allows you to organize data in folders and as files. In fact, any type of file—XML, binary or text—can be created and accessed in isolated storage. There’s no quota for the size of isolated storage on the phone, so you’re effectively limited by the amount of memory and storage available on the phone. The only drawback is that the process of writing to storage and retrieving from storage is somewhat slower than the other methods that store the lists in active memory.

Serialization Options

Another benefit of isolated storage is that you can choose multiple serialization strategies. Unlike the settings that allow you to assign a key and an object, the isolated storage mechanism provides a file stream that you can write to using text, XML, JSON or even binary code. This allows for easy and straightforward serialization of many types of objects, including “deep” serialization that can handle a complex object graph and serialize the children and grandchildren and so on of the instance you’re working with.

JSON and XML These are the two main serialization strategies available in Silverlight. XML is available using either the DataContractSerializer (which emits XML) or the XMLSerializer. The first is familiar to developers who use Windows Communication Foundation (WCF) and is what the framework uses to hydrate and dehydrate messages sent via Web services. It requires that data is marked using the DataContract and DataMember attributes. This is essentially an “opt-in” approach because you explicitly flag the fields you want serialized. The XmlSerializer serializes all public properties with getters and setters and you can change the behavior by using special XML-specific attributes. For more information on the DataContractSerializer, see bit.ly/fUDPha. For more information on the XmlSerializer, see bit.ly/fCIa6q.

JSON stands for JavaScript Object Notation and is popular on the Web because it’s a format that’s easily translated into JavaScript objects. Some developers prefer this method because it provides readable text in the serialized object in a more compact form than XML. JSON serialization is achieved using a special version of the data contract serializer called the DataContractJsonSerializer. The output it produces takes up significantly less space on disk than XML. For more information on the DataContractJsonSerializer, see bit.ly/8cFyjV.

Binary Silverlight and Windows Phone 7 can also use binary serialization. There’s no BinaryFormatter available in Silverlight (this is the class that helps automatically serialize objects to binary) so you must handle the serialization yourself. To use binary serialization, you simply create a binary writer and write to the stream. The writer handles many primitives, so most of the time you can write out the properties on your class to serialize the instance, then create an instance and populate the properties using the reader. For projects with a lot of classes, however, this can become quite cumbersome. This is where Sterling enters the picture.

Sterling Sterling was designed specifically to ease the pain of serializing and deserializing objects in Silverlight. Originally created for Silverlight 4 in the browser, Sterling evolved to a nearly identical codebase that supports Windows Phone 7. Under the covers, Sterling uses binary serialization, which results in a very compact format on disk. Sterling can serialize almost any class and organizes instances using keys that you provide (any property on the instance may be designated as the key). Sterling also provides indexes that can be queried in memory for speed before loading the entire 
instance from disk. Sterling gives you full control over the underlying serialization stream; you can encrypt, compress or even override the binary stream to serialize types exactly how you want.

The advantage Sterling provides is a quick and easy way to serialize objects along with the ability to query keys and indexes with lightning speed using LINQ to Objects. It handles foreign keys and relationships well. All of these benefits come at only a slight speed cost when serializing and deserializing.

Figure 1 compares various serialization strategies and the relative size on disk.

Figure 1 Size-on-Disk Comparison of Various Serialization Strategies

To generate these figures, a collection of 2,000 contacts was created using random names and addresses. The contact records each contain a full name, address and a unique ID (see Figure 2).

Figure 2 The Contact Class

These were then saved using the various serializers, including Sterling. The size computed is the total size on disk, including the additional files that Sterling requires to keep track of indexes and keys.

Full serialization to and from disk is only slightly slower for Sterling due to the overhead of walking the object graph and handling indexes and keys. The chart in Figure 3 compares how quickly each option can save and load all 2,000 contacts.

Figure 3 Comparison of Speed

The final query statistics show where Sterling provides the most leverage in scanning the collection for contacts with names that begin with an “L” and then loading those full contacts. In the example run, the query produced 65 contacts out of the 2,000. Sterling filtered and loaded them in just 110 milliseconds compared to more than 2 seconds for the other formats.

The Recipe Application

To demonstrate the use of Sterling on Windows Phone 7, I’ve written a small recipe application that allows you to browse, edit and add new recipes. Each recipe consists of a name, a set of instructions, a category (such as lunch or dinner) and a set of ingredients. Ingredients can have an amount, a measure and a food item. Food can be added on the fly. Figure 4 shows a sample page from the application.

Figure 4 Edit Recipe Screen

For this application, Sterling serves three distinct purposes. First, it helps front-load the reference data used by the application, such as categories, measurements, initial foods and sample recipes. Second, it preserves the data users enter when they add their own custom recipes and foods. Finally, it facilitates tombstoning by serializing the state of the application when it’s deactivated. The sample application uses the MVVM pattern and demonstrates how to tombstone from within a ViewModel.

The first step, of course, is to add Sterling to the project. This can be done using NuGet (just search for Sterling) or by downloading the binaries and full source from CodePlex at sterling.codeplex.com.

Setting up the Database

The next step is to set up the database. The database configuration defines what types are going to be persisted and what keys and indexes will be used. The Sterling database is a class created by deriving from BaseDatabaseInstance. There are two overloads you must provide: a unique name for the database (the name is unique per application—you can host multiple databases to segregate data or manage different versions) and a list of table definitions. The base class provides helper methods to define tables, keys and indexes.

Keys and Indexes This application defines a table of type FoodModel using the object Id as the key with an index over the FoodName property. This creates an in-memory list of objects that can be quickly queried and filtered.

The following code defines the food “table” with an integer key and a string index:

CreateTableDefinition<FoodModel, int>(f => f.Id)
.WithIndex<FoodModel, string, int>(IDX_FOOD_NAME, f=>f.FoodName)

The call to create the table definition takes the class type and the key type and is passed a lambda expression used to resolve the key. You can use any unique non-null value on the class as your key. The index extension method requires the table type, the index type and the key type. Here, a constant defines the index name and the lambda expression provides the value that the index will use.

Identity Using TriggersSterling supports any type of key, so there’s no built-in facility to automatically generate new keys. Instead, Sterling allows you to specify how you would like keys to be generated by using triggers. Triggers are registered with the Sterling database and are called before a save, after a save and before a delete. The call before a save allows you to inspect the instance and generate a key if one doesn’t already exist.

In the sample application in the code download accompanying this article, all entities use an integer key. Therefore, it’s possible to create a generic trigger based on the instance type to generate the key. When the trigger is first instantiated, it queries the database for existing keys and finds the highest one. If no records exist, it will set the starting key to 1. Each time an instance is saved that has a key less than or equal to zero, it will assign the next number and increment the key. The code for this base trigger is in Figure 5.

Figure 5 Base Trigger for Auto-Key Generation

public class IdentityTrigger<T> : BaseSterlingTrigger<T,int>  
  where T: class, IBaseModel, new()
{
  private static int _idx = 1;

  public IdentityTrigger(ISterlingDatabaseInstance database)
  {
    // If a record exists, set it to the highest value plus 1
    if (database.Query<T,int>().Any())
    {
      _idx = database.Query<T, int>().Max(key => key.Key) + 1;
    }
  }

  public override bool BeforeSave(T instance)
  {
    if (instance.Id < 1)
    {
      instance.Id = _idx++;
    }

    return true;
  }

  public override void AfterSave(T instance)
  {
    return;
  }

  public override bool BeforeDelete(int key)
  {
    return true;
  }
}

Notice that the query can use standard LINQ expressions such as Any and Max. Also, the developer is responsible for providing thread safety within the trigger mechanism.

Using a trigger is easy: You simply register the trigger with the database and pass an instance (this allows you to pass any constructor parameters that are necessary). You can use a similar call to unregister a trigger.

Custom Serializer Sterling supports a variety of types out of the box. When classes and structs are encountered, the public properties and fields of those classes are iterated to serialize the content. Sub-classes and structs are recursively iterated as well. Sterling can’t serialize some basic types 
directly. For example, System.Type is defined as an abstract class that has many possible derived classes. Sterling can’t directly serialize or deserialize this type. To support tombstoning, a special class will be created that stores ViewModel properties and uses the type of the ViewModel as the key. In order to handle the type, Sterling lets you create a custom serializer.

To create a custom serializer, derive from the BaseSerializer class and handle the overloads. For the custom TypeSerializer class, any class that 
derives from System.Type is supported, and the serialization simply writes out the assembly qualified type name. Deserialization uses the static GetType method on the Type class to return the type from the assembly qualified name. The result is shown in Figure 6. Note that it explicitly supports any type that derives from (is assignable to) System.Type.

Figure 6 TypeSerializer

public class TypeSerializer : BaseSerializer 
{
  /// <summary>
  ///     Return true if this serializer can handle the object, 
  ///     that is, if it can be cast to type
    /// </summary>
  /// <param name="targetType">The target</param>
  /// <returns>True if it can be serialized</returns>
  public override bool CanSerialize(Type targetType)
  {
    return typeof (Type).IsAssignableFrom(targetType);
  }

  /// <summary>
  ///     Serialize the object
  /// </summary>
  /// <param name="target">The target</param>
  /// <param name="writer">The writer</param>
  public override void Serialize(object target, 
    BinaryWriter writer)
  {
    var type = target as Type;
    if (type == null)
    {
      throw new SterlingSerializerException(
        this, target.GetType());
    }
    writer.Write(type.AssemblyQualifiedName);
  }

  /// <summary>
  ///     Deserialize the object
  /// </summary>
  /// <param name="type">The type of the object</param>
  /// <param name="reader">A reader to deserialize from</param>
  /// <returns>The deserialized object</returns>
  public override object Deserialize(
    Type type, BinaryReader reader)
  {
    return Type.GetType(reader.ReadString());
  }
}

Any custom serializers are registered with the Sterling engine before it’s activated.

Seeding and Saving Data

Once the database is defined, it’s common to provide “seed data.” In the sample application, a list of categories, standard measures and foods are provided to help the user get started—an example recipe is also included. There are several ways to embed the data, but the easiest is to include the data as a resource in the XAP file. The data can then be parsed as a resource stream the first time the application is run and stored in the database.

In order to work with the tombstone process, the Sterling database engine is activated when the application itself is activated, and deactivated when it’s tombstoned or exited. This ensures the database keys and indexes are flushed to disk and the database is in a stable state when used again. In the App.xaml.cs file, these events can be connected to the phone lifecycle. To set up the database, only a few lines of code are needed, as shown here:

_engine = new SterlingEngine();
_engine.SterlingDatabase.RegisterSerializer<TypeSerializer>();
_engine.Activate();
Database =  
  _engine.SterlingDatabase.RegisterDatabase<RecipeDatabase>();

The preceeding code snippet demonstrates the steps that create the engine, register the custom serializer and then activate the engine and prepare the database for use. The following code shows how to shut down the engine and database when the application is tombstoned or exited:

Database.Flush();
_engine.Dispose();
Database = null;
_engine = null;

Once the database has been activated, it’s ready to receive data. The most common way to package data is to include it as an embedded resource in a readable format, such as XML, JSON or CSV. A query to the database can determine whether data already exists, and if it doesn’t, the data is loaded. Saving data in Sterling is straightforward: You simply pass the instance to save, and Sterling handles the rest. Figure 7 shows a query that checks for categories. If no categories exist, the data is read from embedded resource files to seed the database. Note the use of the truncate operation to clear the tables first.

Figure 7 Seeding Data

if (database.Query<CategoryModel, int>().Any()) return;

// Get rid of old data
database.Truncate(typeof(MeasureModel));
database.Truncate(typeof(FoodModel));
                
var idx = 0;

foreach(var measure in ParseFromResource(FILE_MEASURES,
  line =>
  new MeasureModel
  { Id = ++idx, Abbreviation = line[0], FullMeasure = line[1]} ))
{
  database.Save(measure);
}

// Sample foods auto-generate the id
foreach (var food in
  ParseFromResource(FILE_FOOD, line 
    => new FoodModel { FoodName = line[0] })
    .Where(food => !string.IsNullOrEmpty(food.FoodName)))
{
  database.Save(food);
}

var idx1 = 0;

foreach (var category in ParseFromResource(FILE_CATEGORIES,
  line =>
  new CategoryModel { Id = ++idx1, CategoryName = line[0] }))
{
  database.Save(category);
}

The Main ViewModel: Categories and Recipes

With the seeded database parsed and loaded, the rest of the application can use the existing data to provide lists and prompts for the user, as well as save any information entered by the user. Here’s an example of how the seeded data is made available to the application. The following code snippet loads all categories into an observable collection on the main ViewModel:

Categories = new ObservableCollection<CategoryModel>();
foreach(var category in App.Database.Query<CategoryModel,int>())
{
  Categories.Add(category.LazyValue.Value);
}

The Categories collection can now be bound directly to a Pivot control. A common use of the Pivot control is to provide filtered views over large data sets. The categories indicate the type of meal (whether it’s a breakfast meal, lunch meal or other category) and the Pivot displays the relevant recipes when a category is selected. The recipes for each category are exposed by a query that filters based on the currently selected category.

The following XAML snippet shows how the control is directly bound to the collection and selected category:

<controls:Pivot        
  x:Name="pivotMain"
  Title="Sterling Recipes"
  ItemsSource="{Binding Categories}"
  SelectedItem="{Binding CurrentCategory,Mode=TwoWay}">

Editing Ingredients: Foreign Keys

The key to recipes is, of course, their ingredients. The application contains a “master list” of food items and measurements. Each recipe can then have a list of “ingredients” that include the amount, type of measurement and food item. The ingredients button (back in Figure 4) takes the user to an ingredients list, where the user can add, delete or edit existing ingredients.

This functionality is possible due to an important feature of Sterling: the support for navigation properties that act like foreign keys. Figure 8 shows the hierarchy of models used in the application.

Figure 8 Recipe Class Hierarchy

Recipes contain lists of ingredients that have a circular reference back to the parent recipe. Ingredients also contain an “amount” model that includes units and measure.

When you save a recipe, Sterling will automatically recognize each ingredient as a separate entry in a separate table definition. Instead of serializing the ingredient with the recipe object, Sterling will serialize the key for the index and then save the ingredient separately. It will also recognize the circular reference with the recipe and stop recursion on the object graph. Including the recipe with the ingredient allows queries directly on the ingredient that can then load the corresponding recipe.

When you modify or add an ingredient, the save operation will automatically save the related tables as well. When loaded, foreign tables are always pulled from disk, ensuring they’re always in sync with the latest version.

Food Search: Querying with Indexes

Sterling uses in-memory keys and indexes to facilitate queries and filters. The food search provides an example of filtering and querying. The text box is bound to the ViewModel and will update the text being typed as the user enters it. Users will see the results (foods that contain the text they’re typing in the name) instantly as they’re typing. This makes it easy for the user to narrow the search and either select an existing food or enter a new one. You can see the food search page in Figure 9, with selected items that contain the letters “pe.”

Figure 9 Food Search for Items that Include the Letters “pe”

Whenever the user types, the property setter on the ViewModel is updated with the search text. This setter in turn raises the property changed event for the food list. The food list executes a new query against the food database and returns the results using LINQ to Objects. Figure 10 shows the query, which uses an index to filter and order the data. To access the query, the database is called with the class type, the index type and the key type, and then is passed the name of the index. Notice that a new food model is created based on the key and index value returned from the index.

Figure 10 Food Query

public IEnumerable<FoodModel> Food
{
  get
  {
    if (string.IsNullOrEmpty(_foodText))
    {
      return Enumerable.Empty<FoodModel>();
    }
    var foodTextLower = _foodText.ToLower();
    return from f in App.Database.Query<FoodModel, 
      string, int>(RecipeDatabase.IDX_FOOD_NAME)
      where f.Index.ToLower().Contains(foodTextLower)
      orderby f.Index
      select new FoodModel { Id = f.Key, FoodName = f.Index };
  }
}

Practical Tombstoning

For tombstoning to work, the current state of the application must be saved and restored when the user returns to the application. Sometimes this requirement can be complex because the user may also navigate backward through the application after it has returned from the tombstoned state. Therefore, all pages must be able to retain their values for the experience to remain seamless.

The MVVM pattern takes the responsibility for tombstoning out of the view-centric XAML and codebehind because the state of the view is synchronized with the ViewModel through data binding. Therefore, each ViewModel can be responsible for saving and restoring its own state. The bindings will update the view accordingly. To facilitate tombstoning, a class called TombstoneModel was created.

It’s keyed based on a type (which will be the interface for the ViewModel being saved) and contains a dictionary of keys and objects. This provides the ultimate flexibility for storing types or classes as needed to preserve the ViewModel state.

Sterling supports this because regardless of how a property is defined—whether as a generic object, an interface or an abstract base class—the serialization will write out the object as the implemented type. This isn’t possible with the built-in serializers. To provide a common interface, an ITombstoneFriendly interface is provided by the lightweight MVVM framework. This interface defines activate and deactivate methods that are called when the bound view is navigated to and from (tombstoning will trigger a “from” navigation, for example).

Tombstoning then becomes as simple as creating the model, setting the type and then setting the values that must be persisted. Returning from the tombstone event involves loading the model, reading the values and restoring the state on the ViewModel. Figure 11 illustrates these steps in the text editor ViewModel that must preserve the title that was passed in and the text the user has entered.

Figure 11 Tombstoning

/// <summary>
///     Tombstone
/// </summary>
public void Deactivate()
{
  var tombstone = new TombstoneModel 
    {SyncType = typeof (ITextEditorViewModel)};
  tombstone.State.Add(ExtractPropertyName(()=>Title), Title);
  tombstone.State.Add(ExtractPropertyName(() =>Text), Text);
  App.Database.Save(tombstone);
}

/// <summary>
///     Returned from tombstone
/// </summary>
public void Activate()
{
  var tombstone = App.Database.Load<TombstoneModel>
    (typeof(ITextEditorViewModel));
  if (tombstone == null) return;
  Title = tombstone.TryGet(ExtractPropertyName(() => 
    Title), string.Empty);
  Text = tombstone.TryGet(ExtractPropertyName(() => 
    Text), string.Empty);
}

When the view is closed via normal means (not tombstoning) the record can easily be deleted. When the application is closed, the table is truncated to remove all records, because the state shouldn’t be preserved when the application is relaunched.

Lightweight and Flexible

As you can see, Sterling removes the burden of complex serialization strategies from the developer. To persist entities is as simple as defining a class type and a lambda expression that returns a unique key. Lightweight (less than a 100KB DLL as of this writing) and flexible (with hooks for triggers, encryption, compression and custom serialization), it should address most needs related to local, embedded databases, caching and tombstoning. The recipe application demonstrates how Sterling integrates with Windows Phone 7 to easily and effectively address these needs.

Jeremy Likness is a senior consultant and project manager at Wintellect LLC in Atlanta. He’s a Microsoft Silverlight MVP and a certified MCTS Silverlight developer. Likness frequently presents topics related to Silverlight line-of-business applications at conferences and users groups. He also regularly updates his Silverlight blog at csharperimage.jeremylikness.com.

Thanks to the following technical experts for reviewing this article: Rob Cameron, John Garland and Jeff Prosise