• Article

November 2009

Volume 24 Number 11

N-Tier Apps and the Entity Framework - Building N-Tier Apps with EF4

By Daniel Simmons | November 2009

This article is the third in a series about n-tier programming with the Entity Framework (see msdn.microsoft.com/magazine/dd882522.aspx and msdn.microsoft.com/magazine/ee321569.aspx), specifically about building custom Web services with the Entity Framework (EF) and Windows Communication Foundation (WCF). (In some situations, a REST-based service or some other approach is appropriate, but in these articles, I’ve focused on custom Web services.) The first article described a number of important design considerations and antipatterns. In the second article, I wrote about four patterns that can be used successfully in an n-tier application. That article also included code samples that illustrate how the first release of the Entity Framework (EF 3.5 SP1) can be used to implement what I call the Simple Entities pattern. In this article, I’ll look at some features coming in the second release of the Entity Framework (EF4) and how you use them to implement the Self-Tracking Entities and Data Transfer Objects (DTOs) n-tier patterns.

While Simple Entities is usually not the preferred pattern for n-tier applications, it is the most viable option in the first release of the EF.  EF4, however, significantly changes the options for n-tier programming with the framework.  Some of the key new features include the following:

  1. New framework methods that support disconnected operations, such as ChangeObjectState and ChangeRelationshipState, which change an entity or relationship to a new state (added or modified, for example); ApplyOriginalValues, which lets you set the original values for an entity; and the new ObjectMaterialized event, which fires whenever an entity is created by the framework.
  2. Support for Plain Old CLR Objects (POCO) and foreign key values on entities. These features let you create entity classes that can be shared between the mid-tier service implementation and other tiers, which may not have the same version of the Entity Framework (.NET 2.0 or Silverlight, for example). POCO objects with foreign keys also have a straightforward serialization format that simplifies interoperability with platforms like Java. The use of foreign keys also enables a much simpler concurrency model for relationships.
  3. T4 templates to customize code generation. These templates provide a way to generate classes implementing the Self-Tracking Entities or DTOs patterns.

The Entity Framework team has used these features to implement the Self-Tracking Entities pattern in a template, making that pattern a lot more accessible, and while DTOs still require the most work during initial implementation, this process is also easier with EF4. (The Self-Tracking Entities template and a few other EF features are available as part of a Web download feature community technology preview (CTP) rather than in the Visual Studio 2010/.NET 4 box. The samples in this article assume that Visual Studio 2010/.NET 4 and the feature CTP are installed.) With these new capabilities, one way to evaluate the four patterns I’ve described (Simple Entities, Change Set, Self-Tracking Entities and DTOs) is in terms of a trade-off between architectural goodness (separation of concerns/loose coupling, strength of contract, efficient wire format and interoperability) and ease of implementation and time to market. If you plot the four patterns on a graph that represents this trade-off, the result might look something like Figure 1.


Figure 1 Comparing N-Tier Patterns with EF4

The right pattern for a particular situation depends on a lot of factors. In general, DTOs provide many architectural advantages at a high initial implementation cost. Change Set exhibits few good architectural characteristics but is easy to implement (when it’s available for a particular technology—for example, the DataSet in traditional ADO.NET).

I recommend a pragmatic/agile balance between these concerns by starting with Self-Tracking Entities and moving to DTOs if the situation warrants it. Often, you can get up and running quickly with Self-Tracking Entities and still achieve many important architectural goals. This approach represents a much better trade-off than Change Set or Simple Entities, either of which I would recommend only if you have no other viable options. DTOs, on the other hand, are definitely the best choice as your application becomes larger and more complex or if you have requirements that can’t be met by Self-Tracking Entities, like different rates of change between the client and the server. These two patterns are the most important tools to have in your toolbox, so let’s take a look at each of them.

Self-Tracking Entities

To use this pattern with the Entity Framework, start by creating an Entity Data Model that represents your conceptual entities and map it to a database. You can reverse engineer a model from a database you have and customize it, or you can create a model from scratch and then generate a database to match (another new feature in EF4). Once this model and mapping are in place, replace the default code generation template with the Self-Tracking Entities template by right-clicking the entity designer surface and choosing Add Code Generation Item.

Next, choose the Self-Tracking Entities template from the list of installed templates. This step turns off default code generation and adds two templates to your project: one template generates the ObjectContext, and the other template generates entity classes. Separating code generation into two templates makes it possible to split the code into separate assemblies, one for your entity classes and one for your context.

The main advantage of this approach is that you can have your entity classes in an assembly that has no dependencies on the Entity Framework. This way, the entity assembly (or at least the code that it generates) and any business logic you have implemented there can be shared by the mid-tier and the client if you want. The context is kept in an assembly that has dependencies on both the entities and the EF. If the client of your service is running .NET 4, you can just reference the entity assembly from the client project. If your client is running an earlier version of .NET or is running Silverlight, you probably want to add links from the client project to the generated files and recompile the entity source in that project (targeting the appropriate CLR).

Regardless of how you structure your project, the two templates work together to implement the Self-Tracking Entities pattern. The generated entity classes are simple POCO classes whose only feature beyond basic storage of entity properties is to keep track of changes to the entities—the overall state of an entity, changes to critical properties such as concurrency tokens, and changes in relationships between entities. This extra tracking information is part of the DataContract definition for the entities (so when you send an entity to or from a WCF service, the tracking information is carried along).

On the client of the service, changes to the entities are tracked automatically even though the entities are not attached to any context. Each generated entity has code like the following for each property. If you change a property value on an entity with the Unchanged state, for instance, the state is changed to Modified:

[DataMember]
public string ContactName
{
    get { return _contactName; }
    set
    {
            if (!Equals(_contactName, value))
            {
                _contactName = value;
                OnPropertyChanged("ContactName");
            }
    }
}
private string _contactName;

Similarly, if new entities are added to a graph or entities are deleted from a graph, that information is tracked. Since the state of each entity is tracked on the entity itself, the tracking mechanism behaves as you would expect even when you relate entities retrieved from more than one service call. If you establish a new relationship, just that change is tracked—the entities involved stay in the same state, as though they had all been retrieved from a single service call.

The context template adds a new method, ApplyChanges, to the generated context. ApplyChanges attaches a graph of entities to the context and sets the information in the ObjectStateManager to match the information tracked on the entities. With the information that the entities track about themselves and ApplyChanges, the generated code handles both change tracking and concurrency concerns, two of the most difficult parts of correctly implementing an n-tier solution.

As a concrete example, Figure 2 shows a simple ServiceContract that you could use with Self-Tracking Entities to create an n-tier order submission system based on the Northwind sample database.

Figure 2 A Simple Service Contract for the Self-Tracking Entities Pattern

[ServiceContract]
public interface INorthwindSTEService
{
    [OperationContract]
    IEnumerable<Product> GetProducts();

    [OperationContract]
    Customer GetCustomer(string id);

    [OperationContract]
    bool SubmitOrder(Order order);

    [OperationContract]
    bool UpdateProduct(Product product);
}

The GetProducts service method is used to retrieve reference data on the client about the product catalog. This information is usually cached locally and isn’t often updated on the client. GetCustomer retrieves a customer and a list of that customer’s orders.  The implementation of that method is quite simple, as shown here:

public Customer GetCustomer(string id)
{
    using (var ctx = new NorthwindEntities())
    {
        return ctx.Customers.Include("Orders")
        .Where(c => c.CustomerID == id)
        .SingleOrDefault();
    }
}

This is essentially the same code that you would write for an implementation of this kind of method with the Simple Entities pattern. The difference is that the entities being returned are self-tracking, which means that the client code for using these methods is also quite simple, but it can accomplish much more.

To illustrate, let’s assume that in the order submission process you want not only to create an order with appropriate order detail lines but also to update parts of the customer entity with the latest contact information. Further, you want to delete any orders that have a null OrderDate (maybe the system marks rejected orders that way). With the Simple Entities pattern, the combination of adding, modifying and deleting entities in a single graph would require multiple service calls for each type of operation or a very complicated custom contract and service implementation if you tried to implement something like Self-Tracking Entities in the first release of the EF. With EF4, the client code might look like Figure 3.

Figure 3 Client Code for the Self-Tracking Entities Pattern

var svc = new ChannelFactory<INorthwindSTEService>(
    "INorthwindSTEService")
    .CreateChannel();

var products = new List<Product>(svc.GetProducts());
var customer = svc.GetCustomer("ALFKI");

customer.ContactName = "Bill Gates";

foreach (var order in customer.Orders
    .Where(o => o.OrderDate == null).ToList())
{
    customer.Orders.Remove(order);
}

var newOrder = new Order();
newOrder.Order_Details.Add(new Order_Detail()
    {
        ProductID = products.Where(p => p.ProductName == "Chai")
                    .Single().ProductID,
        Quantity = 1
    });
customer.Orders.Add(newOrder);

var success = svc.SubmitOrder(newOrder);

This code creates the service, calls the first two methods on it to get the product list and a customer entity, and then makes changes to the customer entity graph using the same sort of code you would write if you were building a two-tier Entity Framework application that talks directly to the database or were implementing a service on the mid-tier. (If you aren’t familiar with this style of creating a WCF service client, it automatically creates a client proxy for you without creating proxies for the entities, since we are reusing the entity classes from the Self-Tracking entities template. You could also use the client generated by the Add Service Reference command in Visual Studio if you want.) But here, there is no ObjectContext involved. You are just manipulating the entities themselves. Finally, the client calls the SubmitOrder service method to push the changes up to the mid-tier. 

Of course, in a real application the client’s changes to the graph would probably have come from a UI of some sort, and you would add exception handling around the service calls (especially important when you have to communicate over the network), but the code in Figure 3 illustrates the principles. Another important item to notice is that when you create the order detail entity for the new order, you set just the ProductID property rather than the Product entity itself. This is the new foreign key relationship feature in action. It reduces the amount of information that travels over the wire because you serialize only the ProductID back to the mid-tier, not a copy of the product entity.

It’s in the implementation of the SubmitOrder service method that Self-Tracking Entities really shines:

public bool SubmitOrder(Order newOrder)
{
    using (var ctx = new NorthwindEntities())
    {
        ctx.Orders.ApplyChanges(newOrder);
        ValidateNewOrderSubmission(ctx, newOrder);
        return ctx.SaveChanges() > 0;
    }
}

The call to ApplyChanges performs all the magic. It reads the change information from the entities and applies it to the context in a way that makes the result the same as if those changes had been performed on entities attached to the context the whole time.

Validating Changes

Something else you should notice in the SubmitOrder implementation is the call to ValidateNewOrderSubmission. This method, which I added to the service implementation, examines the ObjectStateManager to make sure that only the kinds of changes we expect in a call to SubmitOrder are present.

This step is really important because by itself, ApplyChanges pushes whatever changes it finds in an entire graph of related objects into the context. Our expectation that the client will only add new orders, update the customer and so on doesn’t mean that a buggy (or even malicious) client would not do something else. What if it changed the price on a product to make an order cheaper or more expensive than it should be? The details of how the validation is performed are less important than the critical rule that you should always validate changes before saving them to the database. This rule applies regardless of the n-tier pattern you use.

 A second critical design principle is that you should develop separate, specific service methods for each operation. Without these separate operations, you do not have a strong contract representing what is and isn’t allowed between your two tiers, and properly validating your changes can become impossible. If you had a single SaveEntities service method instead of a SubmitOrder and a separate UpdateProduct method (only accessible by users authorized to modify the product catalog), you could easily implement the apply and save part of that method, but you would be unable to validate properly because you would have no way to know when product updates are allowed and when they are not.

Data Transfer Objects

The Self-Tracking Entities pattern makes the n-tier process easy, and if you create specific service methods and validate each one, it can be fairly sound architecturally. Even so, there are limits to what you can do with the pattern. When you run into those limits, DTOs are the way out of the dilemma.

In DTOs, instead of sharing a single entity implementation between the mid-tier and the client, you create a custom object that’s used only for transferring data over the service and develop separate entity implementations for the mid-tier and the client. This change provides two main benefits: it isolates your service contract from implementation issues on the mid-tier and the client, allowing that contract to remain stable even if the implementation on the tiers changes, and it allows you to control what data flows over the wire. Therefore, you can avoid sending unnecessary data (or data the client is not allowed to access) or reshape the data to make it more convenient for the service. Generally, the service contract is designed with the client scenarios in mind so that the data can be reshaped between the mid-tier entities and the DTOs (maybe by combining multiple entities into one DTO and skipping properties not needed on the client), while the DTOs can be used directly on the client.

These benefits, however, come at the price of having to create and maintain one or two more layers of objects and mapping. To extend the order submission example, you could create a class just for the purpose of submitting new orders. This class would combine properties of the customer entity with properties from the order that are set in the new order scenario, but the class would leave out properties from both entities that are computed on the mid-tier or set at some other stage in the process. This makes the DTO as small and efficient as possible. The implementation might look like this:

public class NewOrderDTO
{
    public string CustomerID { get; set; }
    public string ContactName { get; set; }
    public byte[] CustomerVersion { get; set; }
    public List<NewOrderLine> Lines { get; set; }
}

public class NewOrderLine
{
    public int ProductID { get; set; }
    public short Quantity { get; set; }
}

Okay, this is really two classes—one for the order and one for the order detail lines—but the data size is kept as small as possible. The only seemingly extraneous information in the code is the CustomerVersion field, which contains the row version information used for concurrency checks on the customer entity. You need this information for the customer entity because the entity already exists in the database. For the order and detail lines, those are new entities being submitted to the database, so their version information and the OrderID aren’t needed—they are generated by the database when the changes are persisted.

The service method that accepts this DTO uses the same lower-level Entity Framework APIs that the Self-Tracking Entities template uses to accomplish its tasks, but now you need to call those APIs directly rather than let the generated code call them for you. The implementation comes in two parts. First, you create a graph of customer, order and order detail entities based on the information in the DTO (see Figure 4).

Figure 4 Creating a Graph of Entities

var customer = new Customer
    {
        CustomerID = newOrderDTO.CustomerID,
        ContactName = newOrderDTO.ContactName,
        Version = newOrderDTO.CustomerVersion,
    };

var order = new Order
    {
        Customer = customer,
    };

foreach (var line in newOrderDTO.Lines)
{
    order.Order_Details.Add(new Order_Detail
        {
            ProductID = line.ProductID,
            Quantity = line.Quantity,
        });
}

Then you attach the graph to the context and set the appropriate state information:

ctx.Customers.Attach(customer);
var customerEntry = ctx.ObjectStateManager.GetObjectStateEntry(customer);
customerEntry.SetModified();
customerEntry.SetModifiedProperty("ContactName");

ctx.ObjectStateManager.ChangeObjectState(order, EntityState.Added);
foreach (var order_detail in order.Order_Details)
{
    ctx.ObjectStateManager.ChangeObjectState(order_detail, 
       EntityState.Added);
}

return ctx.SaveChanges() > 0;

The first line attaches the entire graph to the context, but when this occurs, each entity is in the Unchanged state, so first you tell the ObjectStateManager to put the customer entity in the Modified state, but with only one property, ContactName, marked as modified. This is important because you don’t actually have all the customer information—just the information that was in the DTO. If you marked all properties as modified, the Entity Framework would try to persist a bunch of nulls and zeroes to other fields in the customer entity.

Next you change the state of the order and each of its order details to Added, and then you call SaveChanges.

Hey, where’s the validation code? In this case, because you have a very specific DTO for your scenario, and you are interpreting that object as you map the information from it into your entities, you perform the validation as you go along. There’s no way this code could inadvertently change the price of a product because you never touch the product entity. This is another benefit of the DTO pattern, but only in a roundabout way. You still have to do the validation work; the pattern just forces one level of validation. In many cases, your code needs to include additional validation of the values or other business rules.

One other consideration is properly handling concurrency exceptions. As I mentioned earlier, the version information for the customer entity is included in the DTO, so you are set up to properly detect concurrency issues if someone else modifies the same customer. A more complete sample would either map this exception to a WCF fault so that the client could resolve the conflict, or it would catch the exception and apply some sort of automatic policy for handling the conflict.

If you wanted to extend the sample further by adding another operation, like the ability to modify an order, you would create another DTO specifically for that scenario, with just the right information for it.  This object would look something like our NewOrderDTO, but it would have the OrderID and Version properties for the order and order details entities as well as each property you want to allow the service call to update. The service method implementation would also be similar to the SubmitOrderDTO method shown earlier—walking through the DTO data, creating corresponding entity objects and then setting their state in the state manager before saving the changes to the database.

If you were to implement the order update method both with Self-Tracking Entities and Data Transfer Objects, you would find that the Self-Tracking Entities implementation reuses the entities and shares almost all the same service implementation code between it and the new order submission method—the only difference would be the validation code, and even some of that might be shared. The DTO implementation, however, requires a separate Data Transfer Object class for each of the two service methods, and the method implementations follow similar patterns but have very little if any code that can be shared.

Tips from the Trenches

Here are some tips for what  to watch out for and understand.

  • **Make certain to reuse the Self-Tracking Entity template’s generated entity code on your client. ** If you use proxy code generated by Add Service Reference in Visual Studio or some other tool, things look right for the most part, but you will discover that the entities don’t actually keep track of their changes on the client.
  • **Create a new ObjectContext instance in a Using statement for each service method so that it is disposed of before the method returns. ** This step is critical for scalability of your service. It makes sure that database connections are not kept open across service calls and that temporary state used by a particular operation is garbage collected when that operation is over. The Entity Framework automatically caches metadata and other information it needs in the app domain, and ADO.NET pools database connections, so re-creating the context each time is a quick operation.
  • Use the new foreign key relationships feature whenever possible.  It makes changing relationships between entities much easier. With independent associations—the only type of relationship available in the first release of the Entity Framework—concurrency checks are performed on relationships independently of the concurrency checks performed on entities, and there is no way to opt out of these relationship concurrency checks. The result is that your services must carry the original values of relationships and set them on the context before changing relationships. With foreign key relationships, though, the relationship is simply a property of the entity, and if the entity passes its concurrency check, no further check is needed. You can change a relationship just by changing the foreign key value.
  • **Be careful of EntityKey collisions when attaching a graph to an ObjectContext. ** If, for instance, you are using DTOs and parts of your graph represent newly added entities for which the entity key values have not been set because they will be generated in the database, you should call the 
AddObject method to add the whole graph of entities first and then change entities not in the Added state to their intended state rather than calling the Attach method and then changing Added entities to that state. Otherwise, when you first call Attach, the Entity Framework thinks that every entity should be put into the Unchanged state, which assumes that the entity key values are final. If more than one entity of a particular type has the same key value (0, for example), the Entity Framework will throw an exception. By starting with an entity in the Added state, you avoid this problem because the framework does not expect Added entities to have unique key values.
  • Turn off automatic lazy loading (another new EF4 feature) when returning entities from service methods. If you don’t, the serializer will trigger lazy loading and try to retrieve additional entities from the database, which will cause more data than you intended to be returned (if your entities are thoroughly connected, you might serialize the whole database), or, more likely, you will receive an error because the context will be disposed of before the serializer tries to retrieve the data. Self-Tracking Entities does not have lazy loading turned on by default, but if you are creating a DTOs solution, this is something to watch out for.

And in the End

The .NET 4 release of the Entity Framework makes the creation of architecturally sound n-tier applications much easier. For most applications, I recommend starting with the Self-Tracking Entities template, which simplifies the process and enables the most reuse. If you have different rates of change between service and client, or if you need absolute control over your wire format, you should move up to a Data Transfer Objects implementation. Regardless of which pattern you choose, always keep in mind the key principles that the antipatterns and patterns represent—and never forget to validate your data before saving.

Daniel Simmons is an architect on the Entity Framework team at Microsoft.