• Article

October 2012

Volume 27 Number 10

Windows Phone - Tombstoning and the Zen of Async for Windows Phone

By Ben Day | October 2012

Have you ever written an application and almost as soon as you’re finished, wish you hadn’t written it the way you did? It’s that gut feeling that something isn’t quite right with the architecture. Changes that should be simple feel next to impossible, or at least take much longer than they should. And then there are the bugs. Oh, there are bugs! You’re a decent programmer. How’d you manage to write something with so many bugs? 

Sound familiar? Well, it happened to me when I wrote my first Windows Phone application, NPR Listener. NPR Listener talks to National Public Radio Web services (npr.org/api/index.php) to get the list of available stories for its programs, and then lets users listen to those stories on their Windows Phone devices. When I first wrote it, I’d been doing lots of Silverlight development and I was very happy at how well my knowledge and skills ported to Windows Phone. I got the first version done fairly quickly and submitted it to the Marketplace certification process. The entire time I was thinking, “Well, that was easy.” And then I failed certification. Here’s the case that failed: 

Step 1: Run your application.  

Step 2: Press the Start button to go to the main page of your phone.

Step 3: Press the Back button to return to your application.

When you press the Back button, your application should resume without error and, ideally, should put the user right back at the screen where he exited your application. In my case, the tester navigated to a National Public Radio program (such as “All Things Considered”), clicked into one of the current stories and then pressed the Start button to go to the home screen on the device. When the tester pressed the Back button to return to my app, the application came back up, and it was a festival of NullReferenceExceptions. Not good.

Now, I’ll let you know a bit about how I design my XAML-based applications. For me, it’s all about the Model-View-ViewModel pattern, and I aim for a nearly fanatical separation between the XAML pages and the logic of my app. If there’s going to be any code in the codebehinds (*.xaml.cs) of my pages, there had better be an extremely good reason. A lot of this is driven by my near-pathological need for unit testability. Unit tests are vital because they help you know when your application is working and, more importantly, they make it easy to refactor your code and change how the application works.

So, if I’m so fanatical about unit tests, why did I get all those NullReferenceExceptions? The problem is that I wrote my Windows Phone application like a Silverlight application. Sure, Windows Phone is Silverlight, but the lifecycle of a Windows Phone app and a Silverlight app are completely different. In Silverlight, the user opens the application, interacts with it until she’s done and then closes the app. In Windows Phone, in contrast, the user opens the application, works with it and bounces back and forth—into the OS or any other application—whenever she wants. When she moves away from your application, the application is deactivated, or “tombstoned.” When your app has been tombstoned, it’s no longer running, but its navigation “back stack”—the pages in the application in the order they were visited—is still available on the device.

You might have noticed on your Windows Phone device that you can navigate through a number of applications and then press the Back button repeatedly to go back through those applications in reverse order. That’s the navigation back stack in action, and each time you go into a different application, that application is reactivated from persisted tombstone data. When your application is going to be tombstoned, it gets a notification from the OS that it’s about to be deactivated and should save its application state so it can be reactivated later. Figure 1 shows some simple code for activating and deactivating your application in App.xaml.cs.

Figure 1 Simple Tombstone Implementation in App.xaml.cs

// Code to execute when the application is deactivated (sent to background).
// This code will not execute when the application is closing.
private void Application_Deactivated(object sender, DeactivatedEventArgs e)
{
  // Tombstone your application.
  IDictionary<string, object> stateCollection =
    PhoneApplicationService.Current.State;
  stateCollection.Add("VALUE_1", "the value");
}
// Code to execute when the application is activated
// (brought to foreground).
// This code will not execute when the application is first launched.
private void Application_Activated(object sender, ActivatedEventArgs e)
{
  // Un-tombstone your application.
  IDictionary<string, object> stateCollection =
    PhoneApplicationService.Current.State;
  var value = stateCollection["VALUE_1"];
}

My NullReferenceException problem was caused by a complete lack of planning—and coding—to handle those tombstoning events. That, plus my comprehensive, rich and complex ViewModel implementation, was a recipe for disaster. Think about what happens when a user clicks the Back button to reenter your application. That user doesn’t end up at some start page but instead lands at the last page he visited in your app. In the case of the Windows Phone tester, when the user reactivated my application, she entered the application in the middle and the UI assumed that the ViewModel was populated and could support that screen. Because the ViewModel wasn’t responding to tombstone events, almost every object reference was null. Oops. I didn’t unit test that case, did I? (Kaboom!)

The lesson here is that you need to plan your UI and ViewModels for navigating both forward and backward.

Adding Tombstoning After the Fact

Figure 2shows the structure of my original application. In order to make my application pass certification, I needed to handle that Start/Back button case. I could either implement tombstoning in the Windows Phone project (Benday.Npr.Phone) or I could force it into my ViewModel (Benday.Npr.Presentation). Both involved some uncomfortable architectural compromises. If I added the logic to the Benday.Npr.Phone project, my UI would know too much about how my ViewModel works. If I added the logic to the ViewModel project, I’d need to add a reference from Benday.Npr.Presentation to Microsoft.Phone.dll to get access to the tombstone value dictionary (PhoneApplicationService.Current.State) in the Microsoft.Phone.Shell namespace. That would pollute my ViewModel project with unnecessary implementation details and would be a violation of the Separation of Concerns (SoC) principle. 

The Structure of the Application
Figure 2 The Structure of the Application

My eventual choice was to put the logic into the Phone project but also create some classes that know how to serialize my ViewModel into an XML string that I could put into the tombstone value dictionary. This approach allowed me to avoid the reference from the Presentation project to Microsoft.Phone.Shell while still giving me clean code that honored the Single Responsibility Principle. I named these classes *ViewModelSerializer. Figure 3 shows some of the code required to turn an instance of StoryListViewModel into XML.

Figure 3 Code in StoryListViewModelSerializer.cs to Turn IStoryListViewModel into XML

private void Serialize(IStoryListViewModel fromValue)
{
  var document = XmlUtility.StringToXDocument("<stories />");
  WriteToDocument(document, fromValue);
  // Write the XML to the tombstone dictionary.
  SetStateValue(SERIALIZATION_KEY_STORY_LIST, document.ToString());
}
private void WriteToDocument(System.Xml.Linq.XDocument document,
  IStoryListViewModel storyList)
{
  var root = document.Root;
  root.SetElementValue("Id", storyList.Id);
  root.SetElementValue("Title", storyList.Title);
  root.SetElementValue("UrlToHtml", storyList.UrlToHtml);
  var storySerializer = new StoryViewModelSerializer();
  foreach (var fromValue in storyList.Stories)
  {
    root.Add(storySerializer.SerializeToElement(fromValue));
  }
}

Once I had these serializers written, I needed to add logic to App.xaml.cs to trigger this serialization based on the screen currently being displayed (see Figure 4).

Figure 4 Triggering the ViewModel Serializers in App.xaml.cs

private void Application_Deactivated(object sender, 
  DeactivatedEventArgs e)
{
  ViewModelSerializerBase.ClearState();
  if (IsDisplayingStory() == true)
  {
    new StoryListViewModelSerializer().Serialize();
    new StoryViewModelSerializer().Serialize();
    ViewModelSerializerBase.SetResumeActionToStory();
  }
  else if (IsDisplayingProgram() == true)
  {
    new StoryListViewModelSerializer().Serialize();
    new ProgramViewModelSerializer().Serialize();
    ViewModelSerializerBase.SetResumeActionToProgram();
  }
  else if (IsDisplayingHourlyNews() == true)
  {
    new StoryListViewModelSerializer().Serialize();
    ViewModelSerializerBase.SetResumeActionToHourlyNews();
  }               
}

I eventually got it working and got the application certified but, unfortunately, the code was slow, ugly, brittle and buggy. What I should’ve done was design my ViewModel so it had less state that needed to be saved, and then build it so it would persist itself as it ran, rather than having to do one giant tombstoning event at the end. How would I do that?

The Control Freak School of Asynchronous Programming

So, here’s a question for you: Do you have “control freak” tendencies? Do you have trouble letting go? Do you choose to ignore obvious truths and, through sheer force of will, solve problems in ways that ignore the reality that’s clear as day and staring you directly in the eyes? Yup … that’s how I handled asynchronous calls in the first version of NPR Listener. Specifically, that’s how I approached the asynchronous networking in the first version of the application.

In Silverlight, all network calls must be asynchronous. Your code initiates a network call and immediately returns. The result (or an exception) is delivered sometime later via an asynchronous callback. This means that networking logic always consists of two pieces—the outgoing call and the return call. This structure has consequences and it’s a dirty little secret in Silverlight that any method that relies on the results of a network call can’t return a value and must return void. This has a side effect: Any method that calls another method that relies on the results of a network call must also return void. As you might imagine, this can be absolutely brutal for layered architectures because the traditional implementations of n-tier design patterns, such as Service Layer, Adapter and Repository, rely heavily on return values from method calls.

My solution is a class called ReturnResult (shown in Figure5), which serves as the glue between the method that requests the network call and the method that handles the results of the call and provides a way for your code to return useful values. Figure 6 shows some Repos­itory Pattern logic that makes a call to a Windows Communication Foundation (WCF) service and then returns a pop­u­lated instance of IPerson. Using that code, you can call LoadById(ReturnResult, int) and eventually receive the populated instance of IPerson when client_LoadBy­IdCompleted(object, LoadByIdCompleted­EventArgs) calls one of the Notify methods. It basically allows you to create code that’s similar to what you’d have if you could use return values. (For more information about ReturnResult, please see bit.ly/Q6dqIv.) 

ReturnResult
Figure 5 ReturnResult

Figure 6 Using ReturnResult to Start a Network Call and Return a Value from the Completed Event

public void LoadById(ReturnResult<IPerson> callback, int id)
{
  // Create an instance of a WCF service proxy.
  var client = new PersonService.PersonServiceClient();
  // Subscribe to the "completed" event for the service method.
  client.LoadByIdCompleted +=
    new EventHandler<PersonService.LoadByIdCompletedEventArgs>(
      client_LoadByIdCompleted);
  // Call the service method.
  client.LoadByIdAsync(id, callback);
}
void client_LoadByIdCompleted(object sender,
  PersonService.LoadByIdCompletedEventArgs e)
{
  var callback = e.UserState as ReturnResult<IPerson>;
  if (e.Error != null)
  {
    // Pass the WCF exception to the original caller.
    callback.Notify(e.Error);
  }
  else
  {
    PersonService.PersonDto personReturnedByService = e.Result;
    var returnValue = new Person();
    var adapter = new PersonModelToServiceDtoAdapter();
    adapter.Adapt(personReturnedByService, returnValue);
    // Pass the populated model to the original caller.
    callback.Notify(returnValue);   
  }           
}

When I finished writing the first version of NPR Listener, I quickly figured out that the application was slow (or at least appeared to be slow) because I didn’t do any caching. What I really needed in the app was a way to call an NPR Web service, get a list of stories for a given program and then cache that data so I didn’t have to go back to the service every time I needed to draw that screen. Adding that functionality, however, was fairly difficult because I was trying to pretend that the async calls didn’t exist. Basically, by being a control freak and trying to deny the essentially asynchronous structure of my application, I was limiting my options. I was fighting the platform and therefore contorting my application architecture.

In a synchronous application, things start to happen in the UI and the flow of control passes through the layers of the app, returning data as the stack unwinds. Everything happens inside a single call stack where the work is initiated, data is processed and a return value is returned back up the stack. In an asynchronous application, the process is more like four calls that are all loosely connected: the UI requests that something happens; some processing may or may not happen; if the processing happens and the UI has subscribed to the event, the processing notifies the UI that an action completed; and the UI updates the display with the data from the asynchronous action.

I can already picture myself lecturing some young whippersnapper about how hard we had it in the days before async and await. “In my day, we had to manage our own asynchronous networking logic and callbacks. It was brutal, and we liked it! Now get off my lawn!” Well, in truth, we didn’t like it that way. It was brutal.

Here’s another lesson: Fighting the underlying architecture of the platform will always cause you problems.

Rewriting the Application Using Isolated Storage

I first wrote the application for Windows Phone 7 and did only a minor update for Windows Phone 7.1. In an application whose entire mission is to stream audio, it had always been a disappointment that users couldn’t listen to audio while browsing other applications. When Windows Phone 7.5 came out, I wanted to take advantage of the new background-streaming features. I also wanted to speed up the application and eliminate a lot of unnecessary Web service calls by adding some kind of local data caching. As I started thinking about implementing these features, though, the limitations and the brittleness of my tombstoning, ViewModel and async implementation became more and more apparent. It was time to fix my previous mistakes and completely rewrite the application.

Having learned my lessons in the previous version of the application, I decided I was going to start by designing for “tombstone-ability” and also completely embrace the asynchronous nature of the application. Because I wanted to add local data caching, I started looking into using isolated storage. Isolated storage is a location on your device where your app can read and write data. Working with it is similar to working with the file system in any ordinary .NET application.

Isolated Storage for Caching and Simplified Network Operations

A huge upside of isolated storage is that these calls, unlike network calls, don’t have to be asynchronous. That means I can use a more-conventional architecture that relies on return values. Once I figured this out, I started thinking about how to separate the operations that have to be asynchronous from the ones that can be synchronous. Network calls have to be async. Isolated storage calls can be synchronous. So what if I always write the results of the network calls to isolated storage before I do any parsing? This lets me synchronously load data and gives me a cheap and easy way to do local data caching. Isolated storage helps me solve two problems at once.

I started by reworking how I do my network calls, embracing the fact that they’re a series of loosely associated steps instead of just one big synchronous step. For example, when I want to get a list of stories for a given NPR program, here’s what I do (see Figure 7):

  1. The ViewModel subscribes to a StoryListRefreshed event on the StoryRepository.
  2. The ViewModel calls the StoryRepository to request a refresh of the story list for the current program. This call completes immediately and returns void.
  3. The StoryRepository issues an asynchronous network call to an NPR REST Web service to get the list of stories for the program.
  4. At some point, the callback method is triggered and the StoryRepository now has access to the data from the service. The data comes back from the service as XML and, rather than turning this into populated objects that get returned to the ViewModel, the StoryRepository immediately writes the XML to isolated storage.
  5. The StoryRepository triggers a StoryListRefreshed event.
  6. The ViewModel receives the StoryListRefreshed event and calls GetStories to get the latest list of stories. GetStories reads the cached story list XML from isolated storage, converts it into objects that the ViewModel needs and returns the populated objects. This method can return populated objects because it’s a synchronous call that reads from isolated storage.

Sequence Diagram for Refreshing and Loading the Story List
Figure 7 Sequence Diagram for Refreshing and Loading the Story List

The important point here is that the RefreshStories method doesn’t return any data. It just requests the refresh of the cached story data. The GetStories method takes the currently cached XML data and converts it into IStory objects. Because GetStories doesn’t have to call any services, it’s extremely fast, so the Story List screen populates quickly and the application seems much faster than the first version. If there’s no cached data, GetStories simply returns an empty list of IStory objects. Here’s the IStoryRepository interface:

public interface IStoryRepository
{
  event EventHandler<StoryListRefreshedEventArgs> StoryListRefreshed;
  IStoryList GetStories(string programId);
  void RefreshStories(string programId);
    ...
}

An additional point about hiding this logic behind an interface is that it makes for clean code in the ViewModels and decouples the development effort of the ViewModels from the storage and service logic. This separation makes the code much easier to unit test and much easier to maintain.

Isolated Storage for Continuous Tombstoning

My tombstoning implementation in the first version of the application took the ViewModels and converted them into XML that was stored in the phone’s tombstone value dictionary, PhoneApplicationService.Current.State. I liked the XML idea but didn’t like that persistence of the ViewModel was the responsibility of the phone app’s UI tier rather than of the ViewModel tier itself. I also didn’t like that the UI tier waited until the tombstone Deactivate event to persist my entire set of ViewModels. When the app is running, only a handful of values actually need to be persisted, and they change very gradually as the user moves from screen to screen. Why not write the values to isolated storage as the user navigates through the app? That way the app is always ready to be deactivated and tombstoning isn’t a big deal. 

Moreover, instead of persisting the entire state of the application, why not save only the currently selected value on each page? The data is cached locally so it should be on the device already, and I can easily reload the data from the cache without changing the logic of the application. This decreases the number of values that have to be persisted from the hundreds in version 1 to maybe four or five in version 2. That’s a lot less data to worry about, and everything is much simpler.

The logic for all the persistence code for reading and writing to or from isolated storage is encapsulated in a series of Repository objects. For information related to Story objects, there will be a corresponding StoryRepository class. Figure 8 shows the code for taking a story Id, turning it into an XML document and saving it to isolated storage.

Figure 8 StoryRepository Logic to Save Current Story Id

public void SaveCurrentStoryId(string currentId)
{
  var doc = XmlUtility.StringToXDocument("<info />");
  if (currentId == null)
  {
    currentId = String.Empty;
  }
  else
  {
    currentId = currentId.Trim();
  }
  XmlUtility.SetChildElement(doc.Root, "CurrentStoryId", currentId);
  DataAccessUtility.SaveFile(DataAccessConstants.FilenameStoryInformation, doc);
}

Wrapping the persistence logic inside a Repository object keeps the storage and retrieval logic separated from any ViewModel logic and hides the implementation details from the ViewModel classes. Figure 9 shows the code in the StoryListViewModel class for saving the current story Id when the story selection changes. 

Figure 9 StoryListViewModel Saves the Current Story Id When the Value Changes

void m_Stories_OnItemSelected(object sender, EventArgs e)
{
  HandleStorySelected();
}
private void HandleStorySelected()
{
  if (Stories.SelectedItem == null)
  {
    CurrentStoryId = null;
    StoryRepositoryInstance.SaveCurrentStoryId(null);
  }
  else
  {
    CurrentStoryId = Stories.SelectedItem.Id;
    StoryRepositoryInstance.SaveCurrentStoryId(CurrentStoryId);
  }
}

And here’s the StoryListViewModel Load method, which reverses process when the StoryListViewModel needs to repopulate itself from disk:

public void Load()
{
  // Get the current story Id.
  CurrentStoryId = StoryRepositoryInstance.GetCurrentStoryId();
  ...
  var stories = StoryRepositoryInstance.GetStories(CurrentProgramId);
  Populate(stories);
}

Plan Ahead

In this article, I’ve walked you through some of the architectural decisions and mistakes that I made in my first Windows Phone application, NPR Listener. Remember to plan for tombstoning and to embrace—rather than fight—async in your Windows Phone applications. If you’d like to look at the code for both the before and after versions of NPR Listener, you can download it from msdn.com/magazine/msdnmag0912.

Benjamin Day is a consultant and trainer specializing in software development best practices using Microsoft development tools with an emphasis on Visual Studio Team Foundation Server, Scrum and Azure. He’s a Microsoft Visual Studio ALM MVP, a certified Scrum trainer via Scrum.org, and a speaker at conferences such as TechEd, DevTeach and Visual Studio Live! When not developing software, Day has been known to go running and kayaking in order to balance out his love of cheese, cured meats and champagne.

Thanks to the following technical experts for reviewing this article: Jerri Chiu and David Starr