May 2013

Volume 28 Number 05

Microsoft .NET Framework - Migrating Legacy .NET Libraries to Target Modern Platforms

By Josh Lane | May 2013

One of thegreatest strengths of the Microsoft .NET Framework is the wide variety of third-party open source and commercial libraries that target the platform. It’s a testament to the maturity of the .NET development ecosystem that you not only have great APIs to choose from within the .NET Framework itself, but also thousands of non-Microsoft libraries for serving HTTP requests, drawing grids in a desktop application, storing structured data on the file system, and everything in between. Indeed, a quick perusal of popular .NET code repositories shows more than 32,000 projects on CodePlex, more than 5,000 code samples on code.msdn.microsoft.com and more than 10,000 unique packages on the NuGet Gallery!

The emergence of new software platforms such as Windows Phone 8 and Windows 8 has the potential to breathe new life into these tried-and-true codebases. .NET libraries that have served you well for years on the desktop and server can prove equally (and sometimes more) useful in these new environments—provided you’re willing to expend the migration effort needed to target these new platforms. Traditionally such tasks can be difficult and tedious, but while care and explicit planning are still needed to ensure success, Visual Studio 2012 has several features that minimize potential difficulties and maximize opportunities for reuse across platforms.

In this article I’ll explore the challenges found during a real-world forward migration of the open source Sterling NoSQL object-­oriented database (OODB) project. I’ll walk you through a brief overview of the library, share the migration obstacles encountered and the solutions to overcome them, and then wrap up by considering some patterns and best practices advice you can exploit in your own library migration efforts.

What Is Sterling?

Sterling is a lightweight NoSQL data storage library that provides fast, indexed retrieval of .NET class instances. Updates, deletes, backup and restore, truncation, and the like are also supported, though like other NoSQL technologies it doesn’t provide a general-­purpose, SQL language-based query facility. Instead, the notion of “query” consists of a set of distinct, ordered operations:

• First, retrieve a collection of predefined keys or indexes mapped to lazy-loaded class instances.
• Next, index into the key collection to do initial fast filtering of the entire possible result set.
• Finally, use standard LINQ to Objects queries against the now-filtered key-value pairs to further refine the results.

Clearly the usage model for Sterling (and for similar NoSQL databases) differs from the one offered by traditional relational databases like SQL Server. The lack of a formal, distinct query language seems particularly strange to newcomers. In fact, this is presented by NoSQL proponents as a strength, given the potential complexity and overhead associated with mapping inputs and outputs between the world of query and the world of code. In a NoSQL solution such as Sterling, there is no mapping because query and code are one and the same.

A full treatment of the ins and outs of Sterling is beyond the scope of this article (see Jeremy Likness’ article, “Sterling for Isolated Storage on Windows Phone 7,” at msdn.microsoft.com/magazine/hh205658 for further details), but I’ll highlight some key advantages and trade-offs to keep in mind:

• It has a small footprint (around 150K on disk) and is well-suited for in-process hosting.
• It works out-of-the-box with the standard range of serializable .NET types.
• The set of concepts needed for basic functionality is small; you can be up and running with Sterling in as little as five lines of C# code.
• Traditional database features such as granular security, cascading updates and deletes, configurable locking semantics, Atomicity, Consistency, Isolation, Durability (ACID) guarantees, and so on are not supported in Sterling. If you need these features, you should consider a full relational engine like SQL Server.

The creator of Sterling (my Wintellect colleague, Jeremy Likness) intended from the outset that it target multiple platforms; he created binaries for the .NET Framework 4, Silverlight 4 and 5, and Windows Phone 7. So when considering the work needed to update Sterling to target the .NET Framework 4.5, Windows Phone 8 and Windows Store apps, I knew the architecture would lend itself to such an effort, but I didn’t know exactly what the project would entail.

The advice I outline in this article is a direct result of my experience updating Sterling to target the .NET Framework 4.5, Windows Phone 8 and Windows Store apps. While some of the details of my journey are specific to the Sterling project, many others are relevant to a wide range of projects and porting efforts in the Microsoft ecosystem.

Challenges and Solutions

Reflecting on the obstacles I faced as I ported Sterling to the new target platforms, a few broad categories emerged under which I can both lump the problems I encountered and provide more general guidance for anyone undertaking this kind of project.

Accommodate Divergent Design Philosophies The first set of potential problems is somewhat philosophical in nature, though it has a real impact on the overall migration effort you’ll expend. Ask yourself: “To what extent does the architecture and design of the library I want to migrate align with the common patterns and usage models of the new target platforms?”

This isn’t an easy question to resolve, and neat, one-size-fits-all answers are elusive. Your clever, custom Windows Forms layout manager might be difficult or impossible to port to Windows Presentation Foundation (WPF). The APIs are certainly distinct, but it’s the very different core design philosophies and notions of control management and positioning of those two worlds that are likely to trip you up in the long run. As another example, custom UI input controls that work well for the classic keyboard-and-mouse input style might yield a poor UX for touch-based environments like Windows Phone or Windows 8. The mere desire to migrate a codebase forward isn’t enough; there must be an underlying design compatibility between old and new platforms, plus a willingness to reconcile whatever minor differences that do exist. In the case of Sterling, I had a few such dilemmas to work through.

The most prominent design issue was the mismatch between the synchronous Sterling data update API and the expected asynchronous nature of such behavior in libraries that target Windows Phone 8 and Windows Store apps. Sterling was designed several years ago in a world where asynchronous APIs were rare, and the tools and techniques for creating them were crude at best.

Here’s a typical Save method signature in pre-migration Sterling:

object Save<T>(T instance) where T : class, new()

What’s important to note here is that this method executes synchronously; that is, no matter how long it takes to save the instance argument, the caller is blocked, waiting for the method to complete. This can result in the usual array of thread-blocking issues: unresponsive UIs, dramatically reduced server scalability and so forth.

User expectations with regard to responsive software design have increased dramatically in the last several years; none of us want to put up with a UI frozen for several seconds while waiting for a save operation to complete. In response, API design guidelines for new platforms like Windows Phone 8 and Windows 8 mandate that public library methods such as Save be asynchronous, non-blocking operations. Thankfully, features like the .NET Task-based Asynchronous Pattern (TAP) programming model and C# async and await keywords make this easier now. Here’s the updated signature for Save:

Task<object> SaveAsync<T>(T instance) where T : class, new()

Now Save returns immediately, and the caller has an awaitable object (the Task) to use for eventual harvesting of the result (in this case, the unique key of the newly saved instance). The caller isn’t blocked from doing other work while the save operation completes in the background.

To be clear, all I’ve shown here are the method signatures; the conversion from synchronous to async implementation under the hood required additional refactoring and a switch to the asynchronous file APIs for each target platform. For instance, the synchronous implementation of Save used a BinaryWriter to write to the file system:

using ( BinaryWriter instanceFile = _fileHelper.GetWriter( instancePath ) )
{
  instanceFile.Write( bytes );
}

But because BinaryWriter doesn’t support async semantics, I’ve refactored this to use asynchronous APIs appropriate to each target platform. For example, Figure 1 shows how SaveAsync looks for the Sterling Windows Azure Table Storage driver.

Figure 1 How SaveAsync Looks for the Sterling Windows Azure Table Storage Driver

using ( var stream = new MemoryStream() )
{
  using ( var writer = new BinaryWriter( stream ) )
  {
    action( writer );
  }
  stream.Position = 0;
  var entity = new DynamicTableEntity( partitionKey, rowKey )
  {
    Properties = new Dictionary<string, EntityProperty>
    {
      { "blob", new EntityProperty( stream.GetBuffer() ) }
    }
  };
  var operation = TableOperation.InsertOrReplace( entity );
  await Task<TableResult>.Factory.FromAsync(
    table.BeginExecute, table.EndExecute, operation, null );
}

I still use BinaryWriter to write discrete values to an in-memory stream, but then use the Windows Azure DynamicTableEntity and CloudTable.BeginExecute and .EndExecute to asynchronously store the stream’s byte array content in the Windows Azure Table Storage service. There were several other, similar changes necessary to achieve the async data-update behavior of Sterling. The key point: surface-level API refactoring may be only the first of several steps necessary to achieve a migration redesign goal like this. Plan your work tasks and effort estimates accordingly, and be realistic about whether such a change is even a reasonable goal in the first place.

In fact, my experience with Sterling surfaced just such an unrealistic goal. A core design characteristic of Sterling is that all storage operations work against strongly typed data, using standard .NET data contract serialization APIs and extensions. This works well for Windows Phone and .NET 4.5 clients, as well as C#-based Windows Store apps. However, there’s no notion of strong typing in the world of Windows Store HTML5 and JavaScript clients. After some research and discussion with Likness, I determined there was no easy way to make Sterling available to these clients, so I chose to omit them as supported options. Such potential mismatches must of course be considered case-by-case, but know that they can arise and be realistic about your options.

Share Code Across Target Platforms The next big challenge I faced was one we’ve all encountered at one time or another: how to share common code across multiple projects?

Identifying and sharing common code across projects is a proven strategy for minimizing time-to-market and downstream maintenance headaches. We’ve done this for years in .NET; a typical pattern is to define a Common assembly and reference that from multiple consumer projects. Another favorite technique is the “Add As Link” functionality in Visual Studio, which grants the ability to share a single master source file among multiple projects, demonstrated in Figure 2.

Figure 2 The Visual Studio 2012 Add As Link Feature

Even today, these options work well if the consumer projects all target the same underlying platform. However, when you want to expose common functionality across multiple platforms (as in my case with Sterling), creating a single Common assembly for such code becomes a significant development burden. Creation and maintenance of multiple build targets becomes a necessity, which increases the complexity of the project configuration and build process. Use of preprocessor directives (#if, #endif and so on) to conditionally include platform-specific behavior for certain build configurations is a virtual necessity, which makes the code more difficult to read, navigate and reason about. Energy wasted on such configuration burdens distracts from the primary goal of solving real problems through code.

Happily, Microsoft anticipated the need for easier cross-platform development and, beginning in the .NET Framework 4, added a new feature called Portable Class Libraries (PCLs). PCLs allow you to selectively target multiple versions of the .NET Framework, Silverlight and Windows Phone, as well as Windows Store and Xbox 360, all from a single Visual Studio .NET project. When you choose a PCL project template, Visual Studio automatically ensures that your code uses only libraries that exist on each chosen target platform. This eliminates the need for clumsy preprocessor directives and multiple build targets. On the other hand, it does place some restrictions on which APIs you can call from your library; I’ll explain more on how to work around such restrictions in a moment. See “Cross-Platform Development with the .NET Framework” (msdn.microsoft.com/library/gg597391) for further details on PCL features and use.

PCLs were a natural fit for achieving my cross-platform goals with Sterling. I was able to refactor more than 90 percent of the Sterling codebase into a single common PCL usable without modification from the .NET Framework 4.5, Windows Phone 8 and Windows 8. This is a huge advantage for the long-term viability of the project.

A brief note about unit test projects: As of today, there’s no PCL equivalent for unit test code. The primary obstacle to creating one is the lack of a single unified unit test framework that works across multiple platforms. Given that reality, for Sterling unit tests I defined separate test projects for .NET 4.5, Windows Phone 8 and Windows 8; the .NET 4.5 project contains the sole copy of the test code, while the other projects share the test code using the Add As Link technique mentioned earlier. Each platform project does reference the test framework assemblies unique to that platform; fortunately, the namespace and type names are identical in each, so the same code compiles unmodified in all the test projects. See the updated Sterling codebase on GitHub (bit.ly/YdUNRN) for examples of how this works.

Leverage Platform-Specific APIs While PCLs are a huge help in creating unified cross-platform codebases, they do pose a bit of a dilemma: How do you use platform-specific APIs that aren’t callable from PCL code? A perfect example is the asynchronous code refactoring I mentioned; while .NET 4.5 and Windows Store apps in particular have a wealth of powerful async APIs to choose from, none of these is callable from within a PCL. Can you have your cake and eat it too?

It turns out you can—with a bit of work. The idea is to define inside your PCL one or more interfaces that model the platform-specific behaviors you’re unable to call directly, and then implement your PCL-based code in terms of those interface abstractions. Then, in separate platform-specific libraries, you provide implementations for each interface. Finally, at run time you create instances of PCL types to accomplish some task, plugging in the specific interface implementation appropriate for the current target platform. The abstraction allows the PCL code to remain decoupled from platform details.

If this all sounds vaguely familiar, it should: the pattern I’ve described here is known as Inversion of Control (IoC), a tried-and-true software design technique for achieving modular decoupling and isolation. You can read more about IoC at bit.ly/13VBTpQ.

While porting Sterling, I resolved several API incompatibility issues using this approach. Most of the problem APIs came from the System.Reflection namespace. The irony is that while each target platform exposed all of the reflection functionality I needed for Sterling, each had its own minor quirks and nuances that made it impossible to support them uniformly in PCLs. Hence the need for this IoC-based technique. You’ll find the resulting C# interface abstraction I defined for Sterling to work around these problems at bit.ly/13FtFgO.

A Bit of General Advice

Now that I’ve outlined my migration strategy for Sterling, I’ll take a small step back and consider how the lessons from the experience might apply in the general case.

First—and I can’t stress this enough—use the PCL feature. PCLs are a huge win for cross-platform development, and they offer enough configuration flexibility to suit most any need. If you’re migrating an existing library forward (or even writing a new one) and it targets more than one platform, you should be using a PCL.

Next, anticipate some refactoring effort in order to accommodate updated design goals. In other words, don’t expect your code migration to be a simple mechanical process, replacing one API call with another. It’s entirely possible that the changes you’ll need to make go deeper than surface level and might require changing one or more core assumptions made when the original codebase was written. There’s a practical limit to the total churn you can impose upon the existing code without major downstream impact; you’ll have to decide for yourself where that line is, and when and if to cross it. One person’s migration is another’s source code fork into an entirely new project.

Finally, don’t abandon your existing toolbox of patterns and design techniques. I’ve demonstrated how I used the IoC principle and dependency injection with Sterling to take advantage of platform-specific APIs. Other, similar approaches will undoubtedly serve you well. Classic software design patterns such as strategy (bit.ly/Hhms), adapter (bit.ly/xRM3i), template method (bit.ly/OrfyT) and façade (bit.ly/gYAK9) can be very useful when refactoring existing code for new purposes.

Brave New Worlds

The end result of my work is a fully functional Sterling NoSQL implementation on the three target platforms of the .NET Framework 4.5, Windows 8 and Windows Phone 8. It’s gratifying to see Sterling run on the latest Windows-based devices like my Surface tablet and my Nokia Lumia 920 phone.

The Sterling project is hosted on the Wintellect GitHub site (bit.ly/X5jmUh) and contains the full migrated source code as well as unit tests and sample projects for each platform. It also includes an implementation of the Sterling driver model that uses Windows Azure Table Storage. I invite you to clone the GitHub repository and explore the patterns and design choices I’ve outlined in this article; I hope they serve as a useful starting point for your own similar efforts.

And remember, don’t throw out that old code … migrate it!

Josh Lane is a senior consultant for Wintellect LLC in Atlanta. He’s spent 15 years architecting, designing and building software on Microsoft platforms, and has successfully delivered a wide range of technology solutions, from call center Web sites to custom JavaScript compilers and many others in between. He enjoys the challenge of deriving meaningful business value through software. Reach him at jlane@wintellect.com.

THANKS to the following technical expert for reviewing this article: Jeremy Likness (Wintellect)