• Article

October 2014

Volume 29 Number 10

Data Points : A Pattern for Sharing Data Across Domain-Driven Design Bounded Contexts

Julie Lerman

Julie LermanFor my entire programming life, reusable code and reusable data have been a driving objective. So, when I began learning about Domain-Driven Design (DDD), I struggled with its guidance to enforce such separation across its bounded contexts that the result might be duplicated code and even duplicated data. I had a small fit when some of the best minds in the DDD world attempted an intervention to help me see the potential pitfalls of my old ways. Finally, Eric Evans explained that one had to choose where to pay the price of complexity. Because DDD is about reducing complexity in the software, the outcome is that you pay a price with respect to maintaining duplicate models and possibly duplicate data.

In this column, I’ve written about DDD concepts and how they align with the data-driven experience, first in my January 2013 column, “Shrink EF Model with DDD Bounded Contexts” (bit.ly/1isIoGE), and then in a three-part series called “Coding for Domain-Driven Design: Tips for Data-Focused Devs,” which begins at bit.ly/XyCNrU. In the first of this series you can find a section titled “Shared Data Can Be a Curse in Complex Systems.” Take a look at it for some additional insight as to why the approach I’ll demonstrate here is useful.

I’ve been asked numerous times how, exactly, you can share data between bounded contexts if you follow one of the more extreme DDD patterns, which is to have each bounded context tied to its own database. Steve Smith and I talked about this in our Domain-Driven Design Fundamentals course on Pluralsight.com (bitly.com/PS-DDD), but we didn’t actually implement it, as doing so is a bit more advanced than the focus of that particular course.

There are a number of different ways to leverage common data across bounded contexts. In this column I’m going to focus on one scenario in particular: mirroring data from one system to another where the first system is designed for editing that data and the second just needs read-only access to a bit of that data.

I’ll first lay out the basic pattern, then add some additional detail. The implementation involves a number of working parts, including Inversion of Control (IoC) containers and message queues. If you’re familiar with these tools, the implementation should be easier to comprehend. I won’t go into great detail about the IoC and queue implementation, but you’ll be able to see and debug it in the download sample that goes with this article.

The Sample Scenario: Sharing a Customer List

I chose a very simple scenario to demonstrate this pattern. One system is dedicated to customer service. Here, users can maintain customer data, along with plenty of other data.  This system interacts with a data storage mechanism, but that isn’t important to the sample. The second system is designed for taking orders. In that system, users need access to customers, but really only to identify the customer making the order. That means this bounded context needs just a read-only list of customer names and identifiers. Therefore, the database connected to the second system needs an up-to-date list of customer names and IDs based on the customers that are maintained in the first system. The approach I chose to accomplish this is to mirror in the second system those two pieces of data for each customer that’s maintained in the first system.

Mirroring Data: High Level

At a very high level, the solution I’ll apply is that every time a customer is inserted into System A, the ID and name of that customer should be added to System B’s data store. If I change an existing customer’s name in System A, then System B needs to have the correct name as well, so a name change should cause an update in the System B data store. This domain doesn’t delete data, but a future enhancement could be to remove inactive customers from the System B data store. I won’t bother with that in this implementation.

So, from the previous paragraph, I care about only two events in System A to which I need to respond:

  • A customer was inserted
  • An existing customer’s name was updated

In a connected system, System B could expose methods for System A to call, such as InsertCustomer or UpdateCustomerName. Or System A could raise events, such as CustomerCreated and CustomerNameUpdated, for other systems, including System B, to catch and respond to.

In response to each event, System B needs to do something in its database.

Because these systems are disconnected, a better approach is to employ a publish-subscribe pattern. System A will publish one or more events to some type of operator. And one or more systems then subscribe to that same operator, waiting for particular events and performing their own actions in response to those events.

Publish-subscribe aligns with the principles of DDD that require these two systems be unaware of each other and, therefore, not talk directly to one another. So, instead, I’ll use a concept called an anti-corruption layer. Each system will communicate through the operator that will shuffle messages between the two.

This operator is a message queue. System A will send messages to the queue. System B will retrieve messages from the queue. In my example, I’ll have only one subscriber—System B—but it’s possible to have many subscribers.

What’s in the Event Message?

When the event being published is the CustomerCreated event, System A will send a message that says, “A customer has been inserted. Here’s the customer’s identity and name.” That’s the full message except that it’s represented in data, not in nice English sentences. The interesting part about publishing an event to a message queue is that the publisher doesn’t care what systems are retrieving the message or what they’ll do in response.

System B will respond to that message by inserting or updating the customer in its database. In reality, System B doesn’t even perform this task; I’ll let a service do the job. Moreover, I’ll let the database determine how an update is to be performed. In this case, the System B database will execute a customer “update” using a stored procedure whose logic deletes the original customer record and inserts a new one. Because I’ll be using GUIDs as identity keys, the identity of the customer will be maintained properly. I don’t want to worry about database-generated keys in my DDD software. Pre-created GUIDs vs. database-incremented keys is a sticky topic. You’ll need to define your logic to align with your company’s database practices.

In the end, System B, the ordering system, will have a complete list of customers it can use. Further along in the workflow, if the ordering system needs more information about a particular customer, for example credit card information or current shipping address, I can leverage other mechanisms, such as calling a service, to retrieve that data as needed. I won’t be addressing that workflow here, however.

Communicating with the Message Queue

A system that allows you to communicate messages in an asynchronous way is called an event bus. An event bus contains the infrastructure to store messages and provide them to whoever needs to retrieve them. It also provides an API for interacting with it.  I’ll focus on one particular implementation that I found to be an easy way to get started with this style of communication: a message queue. There are a number of message queues from which to choose. In the DDD Fundamentals course on Pluralsight.com, Smith and I chose to use the SQL Server Service Broker as our message queue. Because we both work with SQL Server, it was simple for us to set up and we just needed to write SQL to push messages to the queue and retrieve them. 

In writing this article, I decided it was time for me to learn to use one of the more popular open source message queues, RabbitMQ. This meant installing the RabbitMQ server (and Erlang!) onto my computer, as well as pulling in the RabbitMQ .NET Client so I could easily code against it in my application. You can learn more about RabbitMQ at rabbitmq.com. I also found the RabbitMQ for .NET Developers courses on Pluralsight.com to be very helpful.

System A, therefore, has a mechanism for sending messages to the RabbitMQ server. But System B, the Order system, has nothing to do with any of this interaction. System B simply expects the customer list to be in the database and doesn’t care how it gets there. A separate small Windows service will handle checking the RabbitMQ message queue for messages and updating the Order system’s database accordingly. Figure 1 shows a visual workflow of the entire process.

A Message Queue Allows Unacquainted Systems to Share Messages, in This Case to Update the System B Database
Figure 1 A Message Queue Allows Unacquainted Systems to Share Messages, in This Case to Update the System B Database

Sending Messages to the Queue

I’ll start with the Customer class in System A, shown in Figure 2. For the sake of a simple example, it comprises only a few properties—ID, Name, the source of the customer and some logging dates. Following DDD patterns, the object has built-in constraints to prevent random editing. You create a new customer using the Create factory method. If you need to fix the name, you use the FixName method.

Figure 2 The Customer Class in the Customer Maintenance-Bounded Context

public static Customer Create(string name, string source) {
   return new Customer(name, source);
  }
  private Customer(string name, string source){
    Id = Guid.NewGuid();
    Name = name;
    InitialDate = DateTime.UtcNow;
    ModifiedDate = DateTime.UtcNow;
    Source = source;
    PublishEvent (true);
  }
  public Guid Id { get; private set; }
  public string Name { get; private set; }
  public DateTime InitialDate { get; private set; }
  public DateTime ModifiedDate { get; private set; }
  public String Source { get; private set; }
  public void FixName(string newName){
    Name = newName;
    ModifiedDate = DateTime.UtcNow;
    PublishEvent (false);
  }
  private void PublishEvent(bool isNew){
    var dto = CustomerDto.Create(Id, Name);
    DomainEvents.Raise(new CustomerUpdatedEvent(dto, isNew));
 }}

Notice that the constructor and the FixName method both call the PublishEvent method, which, in turn, creates a simple CustomerDto (which has only an Id and Name property) and then uses the DomainEvents class from Udi Dahan’s 2009 MSDN Magazine article, “Employing the Domain Model Pattern” (msdn.microsoft.com/magazine/ee236415) to raise a new Customer­UpdatedEvent (see Figure 3). In my example, I’m publishing the event in response to simple actions. In a real implementation, you might prefer to publish these events after the data has been successfully persisted into the System A database.

Figure 3 A Class That Encapsulates an Event When a Customer Is Updated

public class CustomerUpdatedEvent : IApplicationEvent{
  public CustomerUpdatedEvent(CustomerDto customer, 
   bool isNew) : this(){
    Customer = customer;
    IsNew = isNew;
  }
  public CustomerUpdatedEvent()
  {
    DateTimeEventOccurred = DateTime.Now;
  }
  public CustomerDto Customer { get; private set; }
  public bool IsNew { get; private set; }
  public DateTime DateTimeEventOccurred { get; set; }
  public string EventType{
    get { return "CustomerUpdatedEvent"; }
 }}

A CustomerUpdatedEvent wrap’s all I need to know about this event: the CustomerDto along with a flag that indicates whether the customer is new. There’s also metadata that will be needed by a generic event handler.

The CustomerUpdatedEvent can then be handled by one or more handlers I define in my application. I’ve defined only one handler, a service called CustomerUpdatedService:

public class CustomerUpdatedService : IHandle<CustomerUpdatedEvent>
{
  private readonly IMessagePublisher _messagePublisher;
  public CustomerUpdatedService(IMessagePublisher messagePublisher){
    _messagePublisher = messagePublisher;
  }
  public void Handle(CustomerUpdatedEvent customerUpdatedEvent){
    _messagePublisher.Publish(customerUpdatedEvent);
}}

The service will handle all CustomerUpdatedEvent instances my code raises by using the specified message publisher to publish the event. I haven’t specified the publisher here; I’ve only referenced an abstraction, IMessagePublisher. I’m employing an IoC pattern that lets me loosely couple my logic. I’m a fickle gal. Today I may want one message publisher. Tomorrow, I might prefer another. In the background, I used StructureMap (structuremap.net), a popular tool among .NET developers for managing IoC in .NET applications. StructureMap lets me indicate where to find the classes that handle events raised by DomainEvents.Raise. StructureMap’s author, Jeremy Miller, wrote an excellent series in MSDN Magazine called “Patterns in Practice” that’s relevant to the patterns applied in this sample (bit.ly/1ltTgTw). With StructureMap, I configured my application to know that when it sees IMessagePublisher, it should use the concrete class, RabbitMQMessagePublisher, whose logic is shown here:

public class RabbitMqMessagePublisher : IMessagePublisher{
  public void Publish(Shared.Interfaces.IApplicationEvent applicationEvent) {
    var factory = new ConnectionFactory();
    IConnection conn = factory.CreateConnection();
    using (IModel channel = conn.CreateModel()) {
      [code to define the RabbitMQ channel]
      string json = JsonConvert.SerializeObject(applicationEvent, Formatting.None);
      byte[] messageBodyBytes = System.Text.Encoding.UTF8.GetBytes(json);
      channel.BasicPublish("CustomerUpdate", "", props, messageBodyBytes);
 }}}

Note that I’ve removed a number of lines of code specific to configuring RabbitMQ. You can see the full listing in the download (msdn.microsoft.com/magazine/msdnmag1014).

The meat of this method is that it publishes a JSON representation of the event object into the queue. Here’s what that string looks like when I’ve added a new customer named Julie Lerman:

{
"Customer":
  {"CustomerId":"a9c8b56f-6112-42da-9411-511b1a05d814",
    "ClientName":"Julie Lerman"},
"IsNew":true,
"DateTimeEventOccurred":"2014-07-22T13:46:09.6661355-04:00",
"EventType":"CustomerUpdatedEvent"
}

When this message has been published, the Customer Maintenance system’s involvement is complete.

In my sample application, I use a set of tests to cause messages to be published to the queue, as shown in Figure 4. Rather than build tests that will check the queue when they’re finished, I simply browse to the RabbitMQ Manager on my computer and use its tooling. Notice in the test constructor I initialize a class called IoC. This is where I’ve configured StructureMap to wire up the IMessagePublisher and my event handlers.

Figure 4 Publishing to RabbitMq in Tests

[TestClass]
public class PublishToRabbitMqTests
{
  public PublishToRabbitMqTests()
  {IoC.Initialize();
  }
  [TestMethod]
  public void CanInsertNewCustomer()
  {
    var customer = Customer.Create("Julie Lerman", 
      "Friend Referral");
    Assert.Inconclusive("Check RabbitMQ Manager for a message re this event");
  }
  [TestMethod]
  public void CanUpdateCustomer() {
    var customer = Customer.Create("Julie Lerman", 
      "Friend Referral");
    customer.FixName("Sampson");
    Assert.Inconclusive("Check RabbitMQ Manager for 2 messages re these events");
}}

Retrieving the Message and Updating the Order System’s Database

The message sits on the RabbitMQ server waiting to be retrieved. And that task is performed by a Windows service that runs continuously, periodically polling the queue for new messages. When it sees a message, the service retrieves and handles it. The message can also be handled by other subscribers as they come along. For the sake of this sample, I created a simple console application, rather than a service. This allows me to easily run and debug the “service” from Visual Studio while learning. For the next iteration, I might check out Microsoft Azure WebJobs (bit.ly/1l3PTYH), rather than tangling with a Windows service or using my console application hack.

The service employs similar patterns of raising events with Dahan’s DomainEvents class, responding to events in a handler class and initializing an IoC class that uses StructureMap to locate the event handlers.

The service listens to RabbitMQ for messages using the RabbitMQ .NET Client Subscription class. You can see the logic for this in the following Poll method, where the _subscription object keeps listening for messages. Every time a message is retrieved, it deserializes the JSON I stored in the queue back into a CustomerUpdatedEvent and then raises the event:

private void Poll() {
  while (Enabled) {
    var deliveryArgs = _subscription.Next();
    var message = Encoding.Default.GetString(deliveryArgs.Body);
    var customerUpdatedEvent =
      JsonConvert.DeserializeObject<CustomerUpdatedEvent>(message);
    DomainEvents.Raise(customerUpdatedEvent);
}}

The service contains a single class, Customer:

public class Customer{
  public Guid CustomerId { get; set; }
  public string ClientName { get; set; }
}

When the CustomerUpdatedEvent is deserialized, its Customer property—originally populated by a CustomerDto in the Customer Management system—deserializes to this service’s Customer object.

What happens to the raised event is the most interesting part of the service. Here’s the class, CustomerUpdatedHandler, that handles the event:

public class CustomerUpdatedHandler : IHandle<CustomerUpdatedEvent>{
  public void Handle(CustomerUpdatedEvent customerUpdatedEvent){
    var customer = customerUpdatedEvent.Customer;
    using (var repo = new SimpleRepo()){
      if (customerUpdatedEvent.IsNew){
        repo.InsertCustomer(customer);
      }
      else{
        repo.UpdateCustomer(customer);
 }}}}

This service uses Entity Framework (EF) to interact with the database. Figure 5 shows the relevant interaction is encapsulated in two methods—InsertCustomer and UpdateCustomer—in a simple repository. If the event’s IsNew property is true, the service calls the repository’s InsertCustomer method. Otherwise, it calls the UpdateCustomer method.

Figure 5 The InsertCustomer and UpdateCustomer Methods

public void InsertCustomer(Customer customer){
  using (var context = new CustomersContext()){
    context.Customers.Add(customer);
    context.SaveChanges();
}}
public void UpdateCustomer(Customer customer){
  using (var context = new CustomersContext()){
    var pId = new SqlParameter("@Id", customer.CustomerId);
    var pName = new SqlParameter("@Name", customer.ClientName);
    context.Database.ExecuteSqlCommand      
      ("exec ReplaceCustomer {0}, {1}", 
        customer.CustomerId, customer.ClientName);
}}

Those methods perform the relevant logic using an EF DbContext. For an insert, it adds the customer and then calls SaveChanges. EF will perform the database insert command. For an update, it will send the CustomerID and CustomerName to a stored procedure, which uses whatever logic I—or my trusted DBA—has defined to perform the update.

Therefore, the service performs the necessary work in the database to make sure the Customers list in the Ordering system always has an up-to-date roster of customers as maintained in the Customer Maintenance system.

Yes, This Is a Lot of Layers and Puzzle Pieces!

Because I used a simple sample to demonstrate this workflow, you might be thinking the solution is an incredible amount of overkill. But remember, the point is that this is how to orchestrate the solution when you’re using DDD practices to solve complex software problems. When focusing on the domain of customer maintenance, I don’t care about other systems. By using abstractions with the IoC, handlers and message queues, I can to satisfy needs of external systems without muddying up the domain itself. The Customer class simply raises an event. For this demo, this is the easiest place to ensure the workflow makes sense to readers, but it might already be too muddy for your domain. You can certainly raise the event from another place in your application, perhaps from a repository just as it is about to push changes into its own data store.

The sample solution download for this column does employ RabbitMQ and that requires installing its lightweight, open source server on your computer. I’ve included references in the download’s ReadMe file. I’ll also post a short video on my blog at thedatafarm.com, so you can see me stepping through the code, inspecting the RabbitMQ Manager and the database to see the results.

Julie Lerman is a Microsoft MVP, .NET mentor and consultant who lives in the hills of Vermont. You can find her presenting on data access and other.NET topics at user groups and conferences around the world. She blogs at thedatafarm.com/blog and is the author of “Programming Entity Framework” (2010), as well as a Code First edition (2011) and a DbContext edition (2012), all from O’Reilly Media. Follow her on Twitter at twitter.com/julielerman and see her Pluralsight courses at juliel.me/PS-Videos.

Thanks to the following Microsoft technical expert for reviewing this article: Cesar de la Torre