September 2018

Volume 33 Number 9

[Microservices]

Architect Blockchain Applications as Microservices

By Stefano Tempesta | September 2018

Microservices and blockchain smart contracts have a lot in common. They’re both expected to run in isolation (on-chain) and communicate with the outside (off-chain) via a message-based channel. They should both be small in size, developed to run auto­nomously and independently, and perform better when they’re deployed on a decentralized network.

This article presents design principles, artifacts and code samples for building blockchain applications using a microservice architecture style and deploying them on the Microsoft Azure Blockchain platform.

Microservices perfectly embody the spirit of the Unix philosophy: Do one thing and do it well (tcrn.ch/2vnq5Pb). A microservice is an independent, deployable component of bounded scope that supports interoperability through message-based communication. Given this premise, microservice architecture is a style of engineering that helps build highly automated, evolvable software systems made up of single-capability microservices.

What do blockchain applications have in common with microservices, and what design principles can be applied from microservice architectures to the decentralized world? The table in Figure 1compares microservices and smart contracts against specific design attributes.

Figure 1 Microservices and Blockchain Design Principles

Design Principle Microservice Smart Contract
Single responsibility Typically provides a CRUD interface on a single entity. Defines roles, state and the relevant logic for a validation workflow on a single object, using the CRAB approach (more later in this article).
Context bounded No dependency on other services, owns its data model for persistent storage. Does not have dependency on other smart contracts and leverages the on-chain data model (that is, the blockchain itself) as the preferred data model.
Messaging enabled Can leverage an API gateway for inter-service communication, and a service bus for intra-service communication. Can leverage “oracles” or “cryptlets” for off-chain data access, or tools like Azure Blockchain Workbench that expose a REST API.
Autonomously developed Multiple programming languages and frameworks. Multiple blockchain platforms available, although no cross-platform communication currently exists.
Independently deployable With proper design (event sourcing, CQRS) can reduce or remove dependency completely. Similar design patterns apply (described in this article).
Distributed and decentralized Distributed architecture as opposed to a centralized “monolith.” Built-in distributed and decentralized digital ledger by design.

 

Designing blockchain applications as microservices can bring the following benefits to your solution:

  • Allow many software engineering initiatives to run in parallel.
  • Lessen dependencies between software development and test teams.
  • Support multiple technologies, languages and frameworks.
  • Promote ease of innovation through disposable code.

Microservices typically speak to the outside world via an application programming interface (API) that shares a common language, say JSON or SOAP, with the client—providing a lingua franca of messaging-enabled systems across different technologies (.NET, Java, Node.js and so on) and platforms (Windows, Linux). This is true also for the blockchain API exposed by Azure Blockchain Workbench, as you’ll see later in this article.

From Microservices to Decentralized Applications

If you’re familiar with the DevOps sentiment of treating your servers like cattle and not pets (bit.ly/2vrdM4p), you might apply the same approach to your source code. Easily disposable code can reduce technical debt, promote modernization of engineering processes and decrease operational costs by optimizing infrastructure (for example, in containers or entirely serverless configurations).

Designing blockchain applications with microservice architecture principles can also yield business benefits. Improved efficiency in the software system reduces infrastructure costs and the risk of capacity-related service outages. These aspects are of particular value to private blockchains, where cost effectiveness and service continuity are key requirements for businesses.

Microservice architecture principals can support use of replaceable components, reducing technical debt that can lead to aging, unreliable environments. Solidity, the programming language for smart contracts in Ethereum, has a mechanism for specifying the exact runtime version on which each contract is executed. Over time, a smart contract’s version number can be used to identify obsolete blockchain-stored code that may be a candidate for replacement. Just keep in mind that in a blockchain, smart contracts that have already been processed (that is, are part of a “mined” block) cannot be deleted—a new version of a contract must be published for future transactions.

Another benefit is better runtime scalability, which allows a software system to grow or shrink with demand. Smart contracts implemented as microservices enable permissioned blockchains in a private business or consortium environment to distribute workloads for transaction and mining nodes in a more flexible way.

At its most basic level, microservice architecture is about breaking up an application or system into smaller parts and gaining benefit from the distributed setup. In the same vein, smart contracts that run on a blockchain benefit from the distributed nature of the peer-to-peer network. With a microservice architecture-oriented design, smart contracts can deliver improved efficiency, scalability and manageability—all attributes that are essential for proper implementation of blockchain solutions in the enterprise.

 

Decentralized Domain-Driven Design

Write an application for a decentralized blockchain digital ledger and you’re working with infrastructure and software requirements typical of a distributed system, such as storage isolation, asynchronous messaging and distributed transactions. Blockchain applications also require authentication of users and devices, and authorization to execute specific actions within a smart contract. Expanding on the popular Domain-Driven Design (DDD) approach, I refer to the practice of considering entities, processes and properties of a blockchain system as “Decentralized Domain-Driven Design,” or DDDD.

Let’s start defining the domain, or context, of the execution of smart contracts within a digital ledger. Smart contracts represent the business logic of the blockchain application as workflows, with each stage of a workflow identified by a message sender (a user or device that executes a function of the contract) and a state (the parameters of the contract, represented as a message sent to a function, and its internal status). Other parties (again, users or devices) may be affected by the execution of a function of a smart contract.

In this context, we refer to all parties involved as application roles. The very first application role that creates the contract is called the initiator. On change of internal state of a contract, an event may be raised to signal this change to other parts of the smart contract, or to external applications. This is a typical pattern, for example, for populating off-chain data, by using a service bus for processing events raised by a smart contract and publishing messages to the relevant listeners. Figure 2 identifies the entities involved in a workflow within a smart contract.

Workflow Entities in a Smart Contract
Figure 2 Workflow Entities in a Smart Contract

Ethereum uses Solidity as its programming language for writing self-enforcing business logic for smart contracts. Smart contracts in Solidity are similar to classes in object-oriented languages. Each contract contains roles, state and functions to implement actors, stages and actions of a business process.

The code snippet in Figure 3 shows the different types of variable declaration in Solidity for roles, state and properties that may be used in a smart contract for a betting application. Roles (the gambler and the bookmaker) are defined as address, which is the unique identifier of a user or contract in Ethereum. State is an enumeration of labels that identifies the current state of a bet placed via the smart contract. Functions, as you’ll see later, define a change of state. The bet amount is expressed as an unsigned number (currently, Solidity doesn’t support decimal values).

Figure 3 Declaration of Roles, State and Properties in the Betting Smart Contract

pragma solidity ^0.4.20;
contract Betting
{
  // Roles
  address public Gambler;
  address public Bookmaker;
  // State
  enum BetType { Placed, Won, Lost }
  BetType public State;
  // Properties
  uint public BetAmount;

Please note the indication of the version of Solidity that I’m targeting for this smart contract. It’s good practice to indicate this pragma instruction to avoid incompatibilities with future versions of the Solidity programming language and compiler. This also helps identify old code in a smart contract, which may need to be replaced with a new version to support updated code. The process of removing an existing smart contract from a blockchain is called “self-destruct.”

A smart contract should have a single responsibility and contain as little business logic as possible—optimally only the validation logic needed to deem a contract valid or not. In my betting application, a smart contract may expose functions for placing a bet by the gambler, and acknowledging win or loss by the bookmaker. Currency may be exchanged between the two roles, as part of the bet workflow. The usual pattern for monetary transactions that require validation by a contract sees the amount transfer happening in two phases, as indicated in Figure 4. The gambler (initiator of the contract) places a bet for a certain amount, which is then stored in the smart contract. If the bet is won, the won amount, indicated by the bookmaker, is transferred to the gambler. Other­wise, the bookmaker cashes the bet amount.

Betting Workflow
Figure 4 Betting Workflow

As shown in Figure 5, the implementation of this smart contract in Solidity requires that several functions be defined for each action of the workflow, as follows:

  • The constructor stores the message sender as the gambler; this is the role that initiates the smart contract.
  • The Bet function accepts an amount as input, performs some validation (this function can be called only by the gambler and the amount should be more than zero), and then transfers the bet amount to the contract. To allow for on-chain currency transfers, it’s necessary to flag a function as payable.
  • The Won function, after validating that the invoker isn’t the gambler, transfers the won amount to the gambler and closes the bet as “Won.”
  • The Lost function, which again can only be invoked by the bookmaker, transfers the amount initially bet, and now lost by the gambler, to the bookmaker and closes the bet as “Lost.”
  • By closing a bet, the gambler is removed (its address is set to 0x0) and the amount set to zero, ready for another bet.

Figure 5 Functions in the Betting Smart Contract

constructor() public {
  Gambler = msg.sender;
}
function Bet(uint amount) public payable {
  require(msg.sender == Gambler, "Only a gambler can place a bet.");
  require(amount > 0, "Amount should be greater than zero.");
  BetAmount = amount;
  address(this).transfer(amount);
  State = BetType.Placed;
}
function Won(uint amount) public payable {
  require(msg.sender != Gambler, "Only the bookmaker can mark a bet as won.");
  require(amount > 0, "Amount should be greater than zero.");
  Gambler.transfer(amount);
  Close(BetType.Won);
}
function Lost() public payable {
  require(msg.sender != Gambler, "Only the bookmaker can mark a bet as won.");
  Bookmaker = msg.sender;
  Bookmaker.transfer(BetAmount);
  Close(BetType.Lost);
}
function Close(BetType state) internal {
  Gambler = 0x0;
  BetAmount = 0;
  State = state;
}

Though simple in implementation, this scenario identifies a typical pattern for managing monetary spending in a blockchain application. Other scenarios may need to incorporate attestable files, such as documents, spreadsheets, certificates and pictures. For multiple reasons, mainly concerning storage limitation, it’s inappropriate to put files on a blockchain. A common approach is to perform a cryptographic hash (for example, SHA-256) against a file and share that hash on a distributed ledger. The external system would instead persist the file into a storage mechanism, such as Azure Storage or IPFS (ipfs.io).

Performing the hash again with the same hashing algorithm at any future time will return the same result, unless the persisted file was modified—even if just one pixel is modified in an image. This process grants proof of existence that an information object like an email, file, document, phone call or video existed at a certain point in time. It also grants proof of authenticity—you know a digital asset hasn’t changed because the digital ledger stores immut­able and independent, verifiable records of all transactions. For more on the value of blockchain in enterprise content management, read the post I published at bit.ly/2OC2Ycp.

Event Sourcing and CQRS

As discussed in the previous section, I recommend building smart contracts with responsibility for one capability only. While capability-oriented design is a crucial technique for isolation of smart contracts, it’s not sufficient to ensure independent deployability. Smart contracts may operate on a common data model within the domain of the system, despite being isolated in execution. For example, in an application there may be a smart contract for managing bets and another for managing sporting events on which to bet. The smart contract for betting may reference a sporting event, creating a dependency between the two smart contracts (betting cannot happen if an event doesn’t exist). Is there a way to model data that can help avoid data sharing among smart contracts?

Whatever format and storage (SQL, NoSQL, JSON) we use, we generally model databases according to objects and Create, Read, Update, Delete (CRUD) operations. Instead of storing structures that model the state of the domain, we can store events that lead to the current state of our world. This modeling approach is called event sourcing (bit.ly/2O68nrt).

Event sourcing is all about storing facts. A fact is a representative value of an event occurrence. Just as in life, we can’t go back in time and change the past—we can only do something in the present to compensate for earlier actions. Data is immutable; so we always issue a new command or event to compensate for, rather than update, the state of an entity. This approach operates under the acronym of CRAB—Create, Retrieve, Append and Burn (bit.ly/2MbpUOb), which is exactly what a blockchain allows to execute: no data updates or deletions, it only appends to the chain. Deleting something from a blockchain conflicts with its immutability, but you can stop asset transfer by “burning” the recipient address.

An immediate concern of this approach is performance. If any state value is a function of events, you may assume that every access to the value would require recalculation of the current state from the source events. Obviously, that would be extremely slow. In event sourcing, you can avoid such expensive operations by using a so-called rolling snapshot: a projection of the entity state at a given point in time. For instance, banks pre-calculate your bank account balance on the last day of every month, so you don’t need to sum all debits and credits operations since the day you opened your bank account to obtain your current balance.

Figure 6 shows the structural data model for the betting application. This is sometimes called the snowflake model, because each entity (a database table) is different from any other.

Structural Data Model
Figure 6 Structural Data Model

The structural data model saves only the current state of the system, while the event-sourcing approach saves individual facts. State, in event sourcing, is a function of all the pertinent facts that occurred. Not only does this give full auditability, but it also allows us to build state projections toward any time in the past.

To push this further in terms of isolation of responsibilities, Command Query Responsibility Segregation (CQRS) complements event sourcing as a design pattern for data storage. CQRS encourages effective single responsibility and deployability of microservices, and by extension, smart contracts. It states that you can—and should—separate data-update from data-query capabilities into separate models.

When using CQRS, the need to access data across multiple contexts can be eliminated. A smart contract can own and encapsulate any update to the model’s state and raise events on change of this state. By subscribing to notifications of these events, a separate smart contract can build a completely independent and query-­optimized model that doesn’t need to be shared with any other contract or external service. You can learn more about CQRS from Martin Fowler’s post at bit.ly/2Awoz33.

Figure 7 describes the data model that I designed for the betting application using event sourcing. This simple model employs a similar structure regardless of the event handled. It’s not necessary to know the current state of the bet to read the sequence of events. The event’s data structure depends on the event itself. Although a sequence of states exists as defined in the workflow, it’s irrelevant from a data-model perspective. Think bigger: In a supply-chain scenario, multiple workflows and events exist, with different entities and attributes. Your structural data model may grow complex, whereas the event-based model, bar a few different attributes for each event, remains constant.

Event-Sourcing Data Model
Figure 7 Event-Sourcing Data Model

Distributed Transactions

A shared data model isn’t the only use case that can introduce tight coupling between smart contracts. Another important threat is workflows. A lot of real-life processes cannot be represented with a single, atomic operation. With such workflows, the result only makes sense if all the steps can be executed. If any step in the sequence fails, the resulting state of the relevant system becomes invalid. In the RDBMS world, such processes are called “transactions.” Database transactions are typically local, contained within the confines of a single database, and rely on locks on tables before updates. If a step fails, you can roll back the steps already attempted before a final commit.

For distributed workflows and stateless microservices, a traditional transaction implementation with data locks and Atomicity, Consistency, Isolation, Durability (ACID) compliance (en.wikipedia.org/wiki/ACID) is impractical. Sagas (bit.ly/2AzdKNR) are long-lived distributed transactions that allow running workflows in loosely coupled environments, without making any assumption of the reliability of each component of the complex system.

In sagas, every step in the workflow executes its portion of the work, registers a call back to a compensating transaction in a message called “routing slip” and passes the updated message down the activity chain. If any step downstream fails, that step looks at the routing slip and invokes the most recent step’s compensating transaction, passing back the routing slip. The previous step does the same thing, calling its predecessor’s compensating transaction and so on until all already executed transactions are compensated. This pattern leads to eventual consistency of data in a distributed transaction (bit.ly/2v8T360). Due to its highly fault-tolerant, distributed nature, sagas are very well-suited to a microservice architecture, as well as to blockchain smart contracts.

You can implement a sort of routing slip via fallback functions in Solidity. A fallback function is an unnamed function defined with no input argument and no return value. It’s executed on a call to the contract if none of the other functions match the given function identifier or whenever the contract receives Ether (in the case of Ethereum). Additionally, in order to receive Ether, the fallback function must be marked payable. If no such function exists, the contract cannot receive Ether through regular (address-to-address) transactions.

It’s worth mentioning that a contract without a payable fallback function can receive Ether as a recipient of a coinbase transaction, such as a miner block reward. A contract cannot react to such Ether transfers and, thus, also cannot reject them. This is a design choice of Ethereum, and Solidity cannot work around it. A contract can have exactly one unnamed function, as shown here:

// Fallback function
function() public payable {
  emit AmountTransfered(msg.sender);
}
event AmountTransfered(address sender);

In Ethereum, fallback functions are necessary for a smart contract to allow account-to-account direct transfers. This is because the transferring account may need to make transfers to both Externally Owned Accounts (EOAs) and to other smart contracts. As EOAs can only accept direct transfers, the only way for an account to transfer value to another account is for the executing contract to implement a fallback function. This means that any contract that wants to accept such transfers must be prepared for direct transfers by having a fallback function. Without that function, the transfer would fail, and it would be impossible for the contract to accept Ether from the other contract.

A best practice is to not have any logic in the fallback function. It’s possible to put code in the body of this function, but it’s best to avoid anything beyond very short, simple logging. The reason is important and unique to smart contracts: You don’t want this function to fail because it runs out of gas. As a rule of thumb, you’ll have just enough gas to raise an event, but not enough to write data to storage.

Asynchronous Messaging

Asynchronous messaging plays a key role in keeping things loosely coupled in a microservice architecture. For example, we can use a message broker to deliver event notifications in an asynchronous manner, preventing point-to-point connections that create a dependency on each endpoint availability and message format. By the same token, smart contracts can benefit from messaging-enabled integration for inbound (from outside to inside the blockchain) and outbound (from the blockchain toward external applications) communication.

In addition to providing a REST API, Azure Blockchain Workbench provides messaging-based integration based on ledger-centric events. Events are published to an Azure Event Grid, and consumers can ingest data or take action based on these events. For those clients that require reliable messaging, Azure Blockchain Workbench delivers messages to an Azure Service Bus endpoint, as well. You can raise events, for example, to notify users and systems of transactions or changes of state in a smart contract. Event notifications can be consumed directly in code or used with tools such as Logic Apps (bit.ly/2n4EgoP) to trigger flow of data to downstream systems.

Smart contracts often represent a business workflow that integrates with external systems and devices. As a result, transactions must be able to initiate on a distributed ledger that includes data from an external system or device. You may also want to have external systems react to events originating from smart contracts on a distributed ledger. The REST API and messaging integration provide the ability to both send transactions from external systems to smart contracts included in an Azure Blockchain Workbench application, and send event notifications to external systems based on changes that take place within an application. Let’s explore now the patterns identified for each of these types of integrations in your end-to-end solutions:

  • One-way event delivery from a smart contract to an event consumer.
  • One-way event delivery of a message from an external system to a smart contract.

Smart Contract to an Event Consumer

In the first scenario, an event occurs within a smart contract; for example, a state change or the execution of a specific type of transaction. This event is broadcast via an Event Grid to downstream consumers, and those consumers then take appropriate actions. An example of this scenario is that when a transaction occurs, a consumer would be alerted and could take action, such as recording the information in a database. This is the same pattern that Azure Blockchain Workbench follows to populate its off-chain SQL database. Another would be if a smart contract transitions to a particular state; for example, when a contract goes into an “Out of Compliance” state. When this state change happens, it could trigger an alert to be sent to an administrator. This occurs using the process depicted in Figure 8, where:

  1. The smart contract transitions to a new state and sends an event to the ledger.
  2. The ledger receives and delivers the event to Azure Blockchain Workbench.
  3. Azure Blockchain Workbench is subscribed to events from the ledger and receives the event.
  4. Azure Blockchain Workbench publishes the event to subscribers on the Event Grid.
  5. External systems are subscribed to the Event Grid, consume the message and take the appropriate action.

Propagation of an Event Raised in a Smart Contract to an External System
Figure 8 Propagation of an Event Raised in a Smart Contract to an External System

External System to a Smart Contract

There’s also a scenario that flows from the opposite direction. In this case, an event is generated by a sensor or an external system and the data from that event should be sent to a smart contract. A common example is the delivery of data from financial markets, such as prices of commodities, stocks or bonds, to a smart contract. This occurs using the process depicted in Figure 9, where:

  1. An event occurs in an external system that triggers the creation of a message for Azure Blockchain Workbench.
  2. The external system has code written to create this message in a known format and sends this directly to the Service Bus.
  3. Azure Blockchain Workbench is subscribed to events from the Service Bus and retrieves the message.
  4. Azure Blockchain Workbench initiates a call to the ledger, sending data from the external system to a specific contract.
  5. Upon receipt of the message, the contract transitions to a possible new state.

Propagation of an Event Raised by an External System to a Smart Contract
Figure 9 Propagation of an Event Raised by an External System to a Smart Contract

The implementation of an end-to-end solution for either integration scenario is beyond the scope of this article. Good examples of integration in both directions can be found at bit.ly/2M8yflL.

In summary, similar integration patterns can be found in micro­service architectures and blockchain smart contracts. The distributed nature of both architectures encourages asynchronous messaging and the use of a message broker, in the form of an event grid or service bus, for conveying information among services and smart contracts. Design patterns like event sourcing, ultimately, match the immutable nature of a digital ledger and the event-driven approach to off-chain communication.

Exploring the Azure Blockchain Workbench API

Azure Blockchain Workbench (aka.ms/abcworkbench) is a service that in minutes creates a blockchain digital ledger (currently, Ethereum). It also provides resources hosted in Azure for rapid development of smart contracts, while minimizing concerns about the underlying infrastructure required to run Ethereum or similar blockchain platform.

If you’re new to Azure Blockchain Workbench, check out my introduction to it in the June issue of MSDN Magazine (msdn.com/magazine/mt846726). Azure Blockchain Workbench provides developers with a REST API as a gateway to integrate desktop, Web and mobile applications to blockchain applications. For example, a developer can use the API to enable IoT devices to send data to a smart contract. Or the API can be consumed by Power BI to create visualization of blockchain data. The API exposes a large set of capabilities of Azure Blockchain Workbench, organized in operation groups as listed in Figure A, and can be used to automate the following:

  • Creation and management of workflows within a blockchain consortium.
  • Association of users and organizations with a consortium, blockchain application, or application workflow.
  • Execution of transactions on a blockchain.
  • Retrieval of transactional and contract data from a blockchain.

 

Operation Group Description
Applications Management of Blockchain Workbench blockchain applications.
Capabilities List of capabilities a user can perform within Blockchain Workbench.
Checkers Developer tools used to validate Workbench configuration and blockchain smart contracts.
Connections Connection to blockchain networks.
Contracts Smart contract instances, including ability to take actions on smart contracts.
Graph Proxy Represents a proxy method to the Azure Active Directory Graph API for users.
Health Health status of Blockchain Workbench API.
Ledgers Supported blockchain networks.
Users Management of users within Blockchain Workbench.

Figure A The Azure Blockchain Workbench REST API Operation Groups

HTTP requests to the REST API are protected with Azure Active Directory (Azure AD). To make an authenticated request to a REST API, clients have to be authenticated with valid credentials before you can call the API. Authentication is coordinated between the various actors by Azure AD, and provides your client with an access token as proof of the authentication. The token is then sent in the HTTP authorization header of each REST API request. See these examples on GitHub for how to authenticate a client application to the Azure Blockchain Workbench REST API at bit.ly/2vxzlAC.

That GitHub repository contains additional examples of how to invoke the different services for listing applications, workflows and contracts on a blockchain. Although the current implementation of Azure Blockchain Workbench supports only Ethereum, future releases will extend support to other digital ledgers. In the end, the API will provide a common access to any supported blockchain platform using a single-application programming interface, regardless of the technology and programming language in use.


Stefano Tempesta is a Microsoft Regional Director and double MVP on AI and Business Applications, as well as chapter leader for the Microsoft Dynamics 365 community in Switzerland. Tempesta is an instructor of courses about Dynamics 365, blockchain and machine learning, and a regular speaker at international IT conferences, including Microsoft Ignite and Tech Summit. He founded Blogchain Space (blogchain.space), a blog about blockchain technologies, writes for MSDN Magazine and MS Dynamics World, and publishes machine learning experiments on the Azure AI Gallery (gallery.azure.ai).

Thanks to the following technical expert: Jonathan Waldman


Discuss this article in the MSDN Magazine forum