August 2016

Volume 31 Number 8

[Cutting Edge]

Beyond CRUD: Commands, Events and Bus

By Dino Esposito | August 2016

In recent installments of this column, I discussed what it takes to build a Historical create, read, update, delete (H-CRUD). An H-CRUD is a simple extension to classic CRUD where you use two conceptually distinct data stores to persist the current state of objects and all the events that happened during the lifetime of individual objects. If you simply limit your vision to the data store that contains the current state, then all is pretty much the same as with classic CRUD. You have your customer records, your invoices, orders and whatever else forms the data model for the business domain.

The key thing that’s going on here is that this summary data store isn’t the primary data store you create, but is derived as a projection from the data store of events. In other words, the essence of building a historical CRUD is to save events as they happen and then infer the current state of the system for whatever UI you need to create.

 Designing your solution around business events is a relatively new approach that’s gaining momentum, though there’s a long way ahead for it to become the mainstream paradigm. Centering your design on events is beneficial because you never miss anything that happens in the system; you can reread and replay events at any time and build new projections on top of the same core data for, say, business intelligence purposes. Even more interesting, with events as an architect, you have the greatest chance to design the system around the business-specific ubiquitous language. Well beyond being a pillar of Domain-Driven Design (DDD), more pragmatically the ubiquitous language is a great help to understand the surrounding business domain and plan the most effective architectural diagram of cooperating parts and internal dynamics of tasks and workflows.

The implementation of events you might have seen in my May (msdn.com/magazine/mt703431) and June 2016 (msdn.com/magazine/mt707524) columns was very simple and to some extent even simplistic. The main purpose, though, was showing that any CRUD could be turned into an H-CRUD with minimal effort and still gain some benefits from the introduction of business events. The H-CRUD approach has some obvious overlapping with popular acronyms and keywords of today such as CQRS and Event Sourcing. In this column, I’ll take the idea of H-CRUD much further to make it merge with the core idea of Event Sourcing. You’ll see how H-CRUD can turn into an implementation made of commands, buses and events that at first might look like an overly complex way to do basic reads and writes to a database.

One Event, Many Aggregates

In my opinion, one of the reasons software is sometimes hard to write on time and on budget is the lack of attention to the business language spoken by the domain expert. Most of the time, acknowledging requirements means mapping understood requirements to some sort of relational data model. The business logic is then architected to tunnel data between persistence and presentation, making any necessary adjustments along the way. While imperfect, this pattern worked for a long time and the number of cases where monumental levels of complexity made it impractical were numerically irrelevant and, anyway, brought to the formulation of DDD, is still the most effective way to tackle any software projects today.

Events are beneficial here because they force a different form of analysis of the domain, much more task-oriented and without the urgency of working out the perfect relational model in which to save data. When you look at events, though, cardinality is key. In H-CRUD examples I discussed in past columns, I made an assumption that could be quite dangerous if let go without further considerations and explanation. In my examples, I used a one-to-one event-to-­aggregate association. In fact, I used the unique identifier of the aggregate being persisted as the foreign key to link events. To go with the example of the article, whenever a room was booked, the system logs a booking-created event that refers to a given booking ID. To retrieve all events for an aggregate (that is, the booking) a query on the events data store for the specified booking ID is sufficient to get all information. It definitely works, but it’s a rather simple scenario. The danger is that when aspects of a simple scenario become a common practice, you typically move from a simple solution to a simplistic solution. And this isn’t exactly a good thing.

Aggregates and Objects

The real cardinality of the event/aggregate association is written in the ubiquitous language of the business domain. At any rate, a one-to-many association is much more likely to happen than a simpler one-to-one association. Concretely, a one-to-many association between events and aggregates means that an event may sometimes be pertinent to multiple aggregates and that more than one aggregate may be interested in processing that event and may have its state altered because of that event.

As an example, imagine a scenario in which an invoice is registered in the system as a cost of an ongoing job order. This means that in your domain model, you probably have two aggregates—invoice and job order. The event invoice registered captures the interest of the invoice aggregate because a new invoice is entered into the system, but it might also capture the attention of the JobOrder aggregate if the invoice refers to some activity pertinent to the order. Clearly, whether the invoice relates to a job order or not can be determined only after a full understanding of the business domain. There might be domain models (and applications) in which an invoice may stand on its own and domain models (and applications) in which an invoice might be registered in the accounting of a job order and subsequently alter the current balance.

However, getting the point that events may relate to many aggregates completely changes the architecture of the solution and even the landscape of viable technologies.

Dispatching Events Breaks Up Complexity

At the foundation of CRUD and H-CRUD lies the substantial constraint that events are bound to a single aggregate. When multiple aggregates are touched by a business event, you write business logic code to ensure that the state is altered and tracked as appropriate. When the number of aggregates and events exceeds a critical threshold, the complexity of the business logic code might become hard and impractical to handle and evolve.

In this context, the CQRS pattern represents a first step in the right direction as it basically suggests you reason separately on actions that “just read” or “just alter” the current state of the system. Event Sourcing is another popular pattern that suggests you log whatever happens in the system as an event. The entire state of the system is tracked and the actual state of aggregates in the system is built as a projection of the events. Put another way, you map the content of events to other properties that altogether form the state of objects usable in the software. Event Sourcing is built around a framework that knows how to save and retrieve events. An Event Sourcing mechanism is append-only, supports replay of streams of events and knows how to save related data that might have radically different layouts.

Event store frameworks such as EventStore (bit.ly/1UPxEUP) and NEventStore (bit.ly/1UdHcfz) abstract away the real persistence framework and offer a super-API to deal in code directly with events. In essence, you see streams of events that are somewhat related and the point of attraction for those events is an aggregate. This works just fine. However, when an event has impact on multiple aggregates, you should find a way to give each aggregate the ability to track down all of its events of interest. In addition, you should manage to build a software infrastructure that, beyond the mere point of events persistence, allows all standing aggregates to be informed of events of interest.

To achieve the goals of proper dispatching of events to aggregates and proper event persistence, H-CRUD is not enough. Both the pattern behind the business logic and the technology used for persisting event-related data must be revisited.

Defining the Aggregate

The concept of an aggregate comes from DDD and in brief it refers to a cluster of domain objects grouped together to match transactional consistency. Transactional consistency simply means that whatever is comprised within an aggregate is guaranteed to be consistent and up-to-date at the end of a business action. The following code snippet presents an interface that summarizes the main aspects of just any aggregate class. There might be more, but I dare say this is the absolute minimum:

public interface IAggregate
{
  Guid ID { get; }
  bool HasPendingChanges { get; }
  IList<DomainEvent> OccurredEvents { get; set; }
  IEnumerable<DomainEvent> GetUncommittedEvents();
}

At any time, the aggregate contains the list of occurred events and can distinguish between those committed and those uncommitted that result in pending changes. A base class to implement the IAggregate interface will have a non-public member to set the ID and implement the list of committed and uncommitted events. Furthermore, a base Aggregate class will also have some RaiseEvent method used to add an event to the internal list of uncommitted events. The interesting thing is how events are internally used to alter the state of an aggregate. Suppose you have a Customer aggregate and want to update the public name of the customer. In a CRUD scenario, it will simply be an assignment like this:

customer.DisplayName = "new value";

With events, it will be a more sophisticated route:

public void Handle(ChangeCustomerNameCommand command)
{
 var customer = _customerRepository.GetById(command.CompanyId);
 customer.ChangeName(command.DisplayName);
 customerRepository.Save(customer);
}

Let’s skip for a moment the Handle method and who runs it and focus on the implementation. At first, it might seem that ChangeName is a mere wrapper for the CRUD-style code examined earlier. Well, not exactly:

public void ChangeName(string newDisplayName)
{
  var evt = new CustomerNameChangedEvent(this.Id, newDisplayName);
  RaiseEvent(e);
}

The RaiseEvent method defined on the Aggregate base class will just append the event in the internal list of uncommitted events. Uncommitted events are finally processed when the aggregate is persisted.

Persisting the State via Events

With events deeply involved, the structure of repository classes can be made generic. The Save method of a repository designed to operate with aggregate classes described so far will simply loop through the list of the aggregate’s uncommitted events and call a new method the aggregate must offer—the ApplyEvent method:

public void ApplyEvent(CustomerNameChangedEvent evt)
{
  this.DisplayName = evt.DisplayName;
}

The aggregate class will have one overload of the ApplyEvent method for each event of interest. The CRUD-style code you considered way back will just find its place here.

There’s still one missing link: How do you orchestrate front-end use cases, end-user actions with multiple aggregates, business workflows and persistence? You need a bus component.

Introducing a Bus Component

A bus component can be defined as a shared pathway between running instances of known business processes. End users act through the presentation layer and set instructions for the system to deal with. The application layer receives those inputs and turns them into concrete business actions. In a CRUD scenario, the application layer will call directly the business process (that is, workflow) responsible for the requested action.

When aggregates and business rules are too numerous, a bus greatly simplifies the overall design. The bus can be a component you write yourself, the Azure Service Bus, MSMQ or a product such as Rebus (bit.ly/2cmTd1s), NServiceBus (bit.ly/2cruxDI) or MassTransit (bit.ly/2cmT2Dl). The application layer pushes a command or an event to the bus for listeners to react appropriately. Listeners are components commonly called “sagas” that are ultimately instances of known business processes. A saga knows how to react to a bunch of commands and events. A saga has access to the persistence layer and can push commands and events back to the bus. The saga is the class where the aforementioned Handle method belongs. You typically have a saga class per workflow or use case and a saga is fully identified by the events and commands it can handle. The overall resulting architecture is depicted in Figure 1.

Figure 1 Using a Bus to Dispatch Events and Commands

Finally, note that events must also be persisted and queried back from their source. This raises another nontrivial point: Is a classic relational database ideal to store events? Different events can be added at any time in the development and even post production. Each event, in addition, has its own schema. In this context, a non-relational data store fits in even though using a relational database still remains an option—at least an option to consider and rule out with strong evidence.

Wrapping Up

I dare say that most of the perceived complexity of software is due to the fact that we keep on thinking the CRUD way for systems that although based on the fundamental four operations in the acronym (create, read, update, delete) are no longer as simple as reading and writing to a single table or aggregate. This article was meant to be the teaser for more in-depth analysis of patterns and tools, which will continue next month when I present a framework that attempts to make this sort of development faster and sustainable.

Dino Esposito is the author of “Microsoft .NET: Architecting Applications for the Enterprise” (Microsoft Press, 2014) and “Modern Web Applications with ASP.NET” (Microsoft Press, 2016). A technical evangelist for the .NET and Android platforms at JetBrains, and frequent speaker at industry events worldwide, Esposito shares his vision of software at software2cents.wordpress.com and on Twitter: @despos.

Thanks to the following Microsoft technical expert for reviewing this article: Jon Arne Saeteras

Discuss this article in the MSDN Magazine forum