May 2011

Volume 26 Number 05

The Working Programmer - Multiparadigmatic .NET, Part 7: Parametric Metaprogramming

By Ted Neward | May 2011

image: Ted NewardWhen I was a university student, in a calculus-filled microeconomics lecture, the professor shared some words of wis-dom that resonate with me to this day:

“If, while slogging through the tedious details of our chosen subject matter, you find yourself unable to see the reason behind why we’re slogging through all these tedious details, your responsibility is to interrupt me and say, ‘Professor Anderson, what’s the point?’ And we’ll take a few moments to go back and explain how we got here.”

Readers who’ve been sitting through all the articles in this series may well have hit that wall, so let’s take a few moments to go back and review how we got here.

Recap

At its heart, as described by James Coplien in his book “Multi-Paradigm Design for C++” (Addison-Wesley, 1998), which inspired much of the writing in this series of articles, all programming is an exercise in capturing commonality—writing code that represents the “all the time” case—and then using the variability constructs within the language to allow it to behave or be structured differently under certain circumstances.

Object-oriented programming, for example, captures commonality into classes, then permits variability through inheritance, the creation of subclasses that alter that commonality. Typically that’s done by changing the behavior of certain parts of the class using methods or messages, depending on the language in question. The commonality and variability needs of the project don’t always fit into the object-oriented paradigm so neatly, however, or any other particular single paradigm, for that matter—object-oriented programming grew out of procedural programming as an attempt to offer variability that procedural programming couldn’t capture easily.

Fortunately for readers of this magazine, the languages offered by Microsoft in Visual Studio 2010 are multiparadigmatic languages, meaning that they draw several different programming paradigms together into a single language. Coplien first identified C++ as such a multiparadigmatic language, in that it brought three major paradigms together: procedural, object and metaprogrammatic (sometimes also more accurately referred to as metaobject). C++ was also widely criticized as a complicated language, too difficult for the average developer to master, mostly because it was hard to see when to use the various features of the language to solve particular problems.

Modern languages frequently develop into highly multiparadigmatic languages. F#, C# and Visual Basic, as of Visual Studio 2010, support five such paradigms directly: procedural, object-oriented, metaobject, dynamic and functional. All three languages—four, if you include C++/CLI in that mix—therefore run the risk of falling into the same fate as C++.

Without a clear understanding of each of the paradigms mixed into those languages, developers can easily run afoul of the favorite feature trap, where developers rely too much on one feature or paradigm to the exclusion of the others and end up creating overly complex code that eventually gets tossed and rewritten. Do this too many times and the language starts to bear the brunt of developer frustration, leading eventually to calls for a new language, or just retirement in general.

Procedural and Object-Oriented Paradigms

Thus far, we’ve seen the commonality/variability analysis applied to procedural or structural programming, wherein we capture commonality in data structures and operate on those structures by feeding them into different procedure calls, creating variability by creating new procedure calls to operate on those same data structures. We’ve also seen commonality/variability around objects, in which we capture commonality into classes and then create variability by subclassing those classes and changing bits and pieces of them through overriding methods or properties.

Bear in mind, too, that another problem emerges in that inheritance (for the most part) only allows for positive variability—we can’t remove something from a base class, such as member method or field. In the CLR, we can hide derived-accessible member implementation by shadowing it (using the virtual keyword instead of the override keyword in C#, for example). However, that means replacing its behavior with something else, not removing it outright. Fields remain present regardless.

This observation leads to a disturbing revelation for some: Objects can’t do everything we need—at least not pure objects. For example, objects can’t capture variability along structural lines using inheritance: a collection class that captures stack-like behavior, but for a variety of different data types (integers, doubles, strings and so on) can’t capture that structural difference. Granted, within the CLR we can use the unified type system. We can store reference instances of System.Object and downcast as necessary, but that’s not the same as being able to create a type that stores only one type.

This realization brought us to the subject of metaobject programming, as we’re looking for ways to capture things outside of traditional object axes.

The first such meta approach was generative, wherein source code was generated based on some kind of template. This approach allows for some variability on a variety of different axes, but is limited (for the most part) to a source-time execution model. It also begins to break down as the number of variabilities rises because the source template has to somehow vary the code generated (typically with decision-making statements hiding inside the template language) at code-generation time, and this can create complexities in the templates.

The second such meta approach was that of reflective or attributive programming. At run time, the code uses the full-fidelity metadata facilities of the platform (Reflection) to inspect code and behave differently based on what it sees there.

This permitted flexibility of implementation or behavior around runtime decision-making, but introduced its own limitations: no type relationships are present within a reflective/attributive design, meaning there’s no way to programmatically ensure that only database-persistable types can be passed into a method, for example, as opposed to XML-persistable types. The lack of an inheritance or other relationship means that a certain amount of type-safety (and therefore an important ability to prevent errors) is thus lost.

Which brings us to the third metaobject facility within the .NET environment: parametric polymorphism. This means the ability to define types that have types as parameters. Or, put more simply, what the Microsoft .NET Framework refers to as generics.

Generics

In its simplest form, generics permit the compile-time creation of types that have parts of their structure supplied at compile time from client code. In other words, the developer of a stack behavioral collection doesn’t have to know at the time his library is compiled what kinds of types his clients might want to store in different instances—they supply that information when they create instances of the collection.

For example, in a previous article, we saw that the definition of a Cartesian point type requires an ahead-of-time decision (on the part of the Point developer) on the representation of the axis values (X and Y). Should they be integral values? Should they be allowed to be negative?

A Cartesian point used in mathematics could very well need to be a floating point and negative. A Cartesian point used to represent a pixel on a computer graphics screen needs to be positive, integral and likely within a certain numerical range, because computer displays of 4 billion by 4 billion are not yet commonplace.

Thus, on the surface of it, a well-design Cartesian point library will need several different Point types: one using unsigned bytes as X and Y fields, one using doubles as X and Y fields and so on. The behavior, for the most part, will be identical across all of these types, clearly highlighting a violation of the desire to capture commonality (known colloquially as the DRY Principle: “Don’t Repeat Yourself”).

Using parametric polymorphism, we can capture that commonality quite neatly:

class Point2D<T> {
  public Point2D(T x, T y) { this.X = x; this.Y = y; }

  public T X { get; private set; }
  public T Y { get; private set; }
  // Other methods left to the reader's imagination
}

Now the developer can specify precisely the range and type properties of the Cartesian point he wishes to use. When working in a mathematical domain, he creates instances of Point2D<double> values, and when working to display those values to the screen, he creates instances of Point2D<sbyte> or Point2D<ushort>. Each is its own distinct type, so attempts to compare or assign Point2D<sbyte> to Point2D<double> will fail miserably at compile time, exactly as a strongly typed language would prefer.

As written, however, the Point2D type still has some drawbacks. We’ve captured the commonality of Cartesian points, certainly, but we’ve essentially allowed for any kind of values to be used for the X and Y values. While this could conceivably be useful in certain scenarios (“In this graph, we’re charting the ratings each person gave a particular movie”), as a general rule, trying to create a Point2D<DateTime> is potentially confusing, and trying to create a Point2D<System.Windows.Forms.Form> almost certainly is. We need to introduce some kind of negative variability here (or, if you prefer, throttle back the degree of positive variability), restricting the kinds of types that can be axis values in a Point2D.

Many .NET languages capture this negative variability via parameterization constraints—also sometimes referred to as type constraints—by explicitly describing conditions the type parameter must have:

class Point2D<T> where T : struct {
  public Point2D(T x, T y) { this.X = x; this.Y = y; }

  public T X { get; private set; }
  public T Y { get; private set; }
  // Other methods left to the reader's imagination
}

This means the compiler won’t accept anything for T that isn’t a value type.

To be honest, this isn’t exactly a negative variability, per se, but it serves as one compared to the problem of trying to remove certain functionality, which approximates a fair amount of what a true negative variability would.

Varying Behavior

Parametric polymorphism is typically used to provide variability on the structural axis, but as the developers of the C++ Boost libraries demonstrated, it’s not the only axis along which it can operate. With judicious use of type constraints, we can also use generics to provide a policy mechanism in which clients can specify a behavioral mechanism for objects being constructed.

Consider, for a moment, the traditional problem of diagnostic logging: To help diagnose problems with code running on a server (or even on client machines), we want to track the execution of code through the codebase. This typically means writing messages to file. But sometimes we want the messages to appear on the console, at least for certain scenarios, and sometimes we want the messages thrown away. Handling diagnostic logging messages has been a tricky problem over the years, and a variety of solutions have been proposed. The lessons of Boost offer a new approach as well.

We start by defining an interface:

interface ILoggerPolicy {
  void Log(string msg);
}

It’s a straightforward interface, with one or more methods defining the behavior we want to vary, which we do via a series of subtypes of that interface:

class ConsoleLogger : ILoggerPolicy {
  public void Log(string msg) { Console.WriteLine(msg); }
}

class NullLogger : ILoggerPolicy {
  public void Log(string msg) { }
}

Here we have two possible implementations, one of which writes the log message to the console while the other throws it away.

Using this requires clients to opt in by declaring the logger as a typed parameter, and creating an instance of it to do the actual logging:

class Person<A> where A : ILoggerPolicy, new() {
  public Person(string fn, string ln, int a) {
    this.FirstName = fn; this.LastName = ln; this.Age = a;
    logger.Log("Constructing Person instance");
  }

  public string FirstName { get; private set; }
  public string LastName { get; private set; }
  public int Age { get; private set; }

  private A logger = new A();
}

Describing which logger type to use, then, is a simple matter of passing a constructor-time parameter, like so:

Person<ConsoleLogger> ted = 
  new Person<ConsoleLogger>("Ted", "Neward", 40);
var anotherTed  = 
  new Person<NullLogger>("Ted", "Neward", 40);

This mechanism allows developers to create their own custom logging implementations, and plug them in to be used by Person<> instances without the Person<> developer having to know any details of the logging implementation used. But numerous other approaches also do this, such as having a Logger field or property that passes in a Logger instance from outside (or obtaining one through a Dependency-Injection approach). The generics-based approach has one advantage that the field-based approach doesn’t, however, and that’s compile-time distinction: a Person<ConsoleLogger> is a distinct and separate type from a Person<NullLogger>.

Money, Money, Money

One problem that plagues developers is that quantities are useless without the units being quantified. One thousand pennies is clearly not the same thing as 1,000 horses or 1,000 employees, or 1,000 pizzas. Yet, just as clearly, 1,000 pennies and 10 dollars are, in fact, the same value.

This becomes even more important in mathematical calculations where the need to capture the units (degrees/radians, feet/meters, Fahrenheit/Celsius) is even more critical, particularly if you’re writing guidance control software for a really big rocket. Consider the Ariane 5, whose maiden flight had to be self-destructed due to an error in conversion. Or the NASA probe to Mars, one of which slammed into the Martian landscape at full speed due to a conversion error.

Recently, new languages like F# have decided to acclimate units of measure as a direct language feature, but even C# and Visual Basic can do similar kinds of things, thanks to generics.

Channeling our inner Martin Fowler, let’s start with a simple Money class, which knows the amount (quantity) and currency (type) of a particular monetary amount:

class Money {
  public float Quantity { get; set; }
  public string Currency { get; set; }
}

On the surface, this seems workable, but before too long we’re going to want to start doing value-like things with this, such as add Money instances together (a pretty common thing to do with money, when you think about it):

class Money {
  public float Quantity { get; set; }
  public string Currency { get; set; }

  public static Money operator +(Money lhs, Money rhs) {
    return new Money() { 
      Quantity = lhs.Quantity + rhs.Quantity, Currency = lhs.Currency };
  }
}

Of course, the problem is going to arise when we try to add U.S. dollars (USD) and European euro (EUR) together, such as when we go out to lunch (after all, everybody knows Europeans brew the best beer, but Americans make the best pizza):

var pizza = new Money() { 
  Quantity = 4.99f, Currency = "USD" };
var beer = new Money() { 
  Quantity = 3.5f, Currency = "EUR" };
var lunch = pizza + beer;

Anybody who takes a quick glance at the financial dashboards is going to realize that somebody’s getting ripped off—the euros are being converted to dollars at a 1-1 ratio. In order to prevent accidental fraud, we probably want to make sure the compiler knows not to convert USD to EUR without going through an approved conversion process that looks up the current conversion rate (see Figure 1).

Figure 1 Conversion Is in Order

class USD { }
class EUR { }
class Money<C> {
  public float Quantity { get; set; }
  public C Currency { get; set; }

  public static Money<C> operator +(
    Money<C> lhs, Money<C> rhs) {
    return new Money<C>() { 
      Quantity = lhs.Quantity + rhs.Quantity, 
      Currency = lhs.Currency };
  }
}
... 
var pizza = new Money<USD>() { 
  Quantity = 4.99f, Currency = new USD() };
var beer = new Money<EUR>() { 
  Quantity = 3.5f, Currency = new EUR() };
var lunch = pizza + beer;    // ERROR

Notice how USD and EUR are basically just placeholders, designed to give the compiler something to compare against. If the two C type parameters aren’t the same, it’s a problem.

Of course, we’ve also lost the ability to combine the two, and there will be times when we want to do exactly that. Doing so requires a bit more parametric syntax (see Figure 2).

Figure 2 Combining Types Intentionally

class USD { }
class EUR { }
class Money<C> {
  public float Quantity { get; set; }
  public C Currency { get; set; }

  public static Money<C> operator +(
    Money<C> lhs, Money<C> rhs) {
    return new Money<C>() { 
      Quantity = lhs.Quantity + rhs.Quantity, 
      Currency = lhs.Currency };
  }

  public Money<C2> Convert<C2>() where C2 : new() {
    return new Money<C2>() { Quantity = 
      this.Quantity, Currency = new C2() };
  }
}

This is a specialized generic method within a generic class, and the <> syntax after the method name adds more type parameters to the scope of the method—in this case, the type of the second currency to convert over to. So, buying a pizza and beer now becomes something like:

var pizza = new Money<USD>() { 
  Quantity = 4.99f, Currency = new USD() };
var beer = new Money<EUR>() { 
  Quantity = 3.5f, Currency = new EUR() };
var lunch = pizza + beer.Convert<USD>();

If desirable, we could even use the conversion operator (in C#) to do the conversion automatically, but that could potentially be more confusing than helpful to readers of the code, depending on your aesthetic preferences.

Wrapping Up

What’s missing from the Money<> example is obvious: clearly there needs to be some way to convert dollars to euros and euros to dollars. But part of the goal in designs like this is to avoid a closed system—that is, as new currencies are needed (rubles, rupees, pounds, lira or whatever else floats your monetary boat), it would be nice if we, the original designers of the Money<> type, don’t have to be called to add them. Ideally, in an open system, other developers can plug them in as they need and everything “just works.”

Don’t tune out just yet, though, and certainly don’t start shipping the code as is. We still have a few adjustments to make to the Money<> type to make it more powerful, safe and extensible. Along the way, we’ll have a look at dynamic and functional programming.

But for now, happy coding!


Ted Neward is a principal with Neward & Associates, an independent firm specializing in enterprise .NET Framework and Java platform systems. He has written more than 100 articles, is a C# MVP and INETA speaker, and has authored or coauthored a dozen books, including “Professional F# 2.0” (Wrox, 2010). He consults and mentors regularly—reach him at ted@tedneward.com if you’re interested in having him come work with your team, or read his blog at blogs.tedneward.com.

Thanks to the following technical expert for reviewing this article: Krzysztof Cwalina