Business Patterns for Software Engineering Use, Part 2
Summary: In the second of two articles, the authors describe how to develop business patterns based on business functions, data, and business components, and show how these can be used to engineer software systems. (16 printed pages)
This is the second article in a series of two. The purpose of these articles is to explore whether business pattern can be defined in a structured way, and if so, whether these business patterns would be of value for those who build software systems that support the business. Our point of view is that this value does exist. The first article was published in JOURNAL2 and it explored how to define business patterns that would be useful for software engineers. Article 1 sets the scene for this article and should be read before this one. In the first article, we used an Enterprise Architecture Framework called SAM to analyze the opportunity and we concluded that such business patterns will describe:
- The business functions being supported.
- The data that is required to support the described functions.
- The business components that are the IT representations of the data and functions the business needs.
- Optionally, the infrastructure needed to support the functions, data and components. This is necessary in highly distributed enterprises or those made up of divisions or units with diverse technical or operational environments.
In addition, we defined the business patterns that describe the key relationships between these dimensions.
In this second article, we describe how to develop business patterns based on business functions, data, and business components. We also show how these can be used to engineer software systems.
Summary of Article 2
In this article we use a realistic but simplified example to show how to use standard techniques to develop descriptions of the business functions, data and business components required for a business pattern. We do not describe the infrastructure needed, and hence will not explore the relationships to infrastructure either. Our goal is to say "it can be done, and you already know how" rather than to provide a detailed guide to every step. First we show a way to define the business functions, data and components we need, and then second we use the PRM to show a roadmap for the journey. The example we use is based on the Healthcare industry but the techniques are valid for any industry. The techniques shown are those that were actually used in a real project. It is important to stress that we are not saying that you must use these techniques. The techniques in themselves are not important—it's the results that matter. These should enable you to engineer either more refined patterns or real software systems from the deliverables.
Difference Between Patterns and Systems
If you haven't had much experience with patterns, you might look at the examples in this article and see them as a system development rather than a pattern. This is because, for software engineering, a pattern is a stepping-stone on the way to developing part of a system. The pattern gets you part of the way to the final objective by starting you off from a tried-and-trusted place. It looks familiar, but it is not a solution in itself—it is an abstraction of the solution that you have to embellish to make it the solution that suits your needs. The tricky bit in creating patterns is to get the level of abstraction right. Very little abstraction constrains the usefulness of the pattern, as it is then very specific. But a lot of abstraction also limits the usefulness as it provides very little practical value.
Think of a common situation today where you might have designed a service called "message broker" to provide message routing and transformation between a variety of IT services. A highly abstracted pattern would be exactly what we just said—if you want to provide message routing and transformation between a variety of IT services then use a message broker. How useful was that? Well, it starts you thinking the right way, so it is some use but does not provide a lot of practical help.
On the other hand, after you've built a message broker for handling the IT services required by a bank to handle a consolidated customer view, could you take that solution and call it a pattern? Yes, but how many would it be useful for? It is too specific and is a potentially repeatable solution rather than a pattern. Somewhere in between is the sweet spot, where the key ideas at all the levels involved in building the system can be abstracted and reapplied to solve many industry problems. These are the most powerful patterns. In this article we think the Business Component specification example we show is in that sweet spot for the healthcare industry. We think this is an important aspect of business patterns that should be clearly recognized—some will not be of interest outside the industry (like this one) and some will (that represent common functions). We speculate that the common ones that cross industries actually identify the areas mostly ripe for outsourcing—but that's a whole different discussion.
To develop business patterns our first goal must be to discover, define and document relevant business functions. This has two main tasks:
- We want to discover the atomic level function set that describes the problem space. These are known as "primitive functions" in functional modeling. The same things are known as "elementary processes" in Business Process Re-engineering.
- We want to aggregate the atomic functions into larger and larger grained functional groups.
Note that by "atomic" we mean that the function cannot be meaningfully divided any further—once started, the function has to be completed or aborted. This can be more fine-grained, and therefore voluminous, than is strictly necessary for business pattern definition. Therefore what we actually seek for a business pattern is business functions defined at to level of decomposition at which meaningful relationships can be formed with the other spheres, particularly the data sphere. In the case of business functions we find this in practice to be at about the fourth level of decomposition from the root of the hierarchy. At this level it usually possible to formulate CRUD (Create, Read, Update and Delete) relationships with data entities held in the Data sphere. We can tackle the definition of business functions either bottom-up or top-down or in a mixture of the two.
In this approach the analyst works with a representative set of users from the business domain, to analyze their view of the business processes that they perform. This may be done using use case diagrams or any technique that can show the sequential and parallel tasks carried out in the process. The collection of processes is then catalogued and analyzed and it is usually noticed that many of the steps or low-level tasks carried out in these processes are repeated many times in different processes. The redundant tasks are identified and eliminated a non-redundant set of primitive functions. This is shown in Figure 1 using a simple flowcharting technique. It can be seen that task A is redundant and so will appear only once in the primitive set. Then having derived the non-redundant primitive set by this analysis, we can now iteratively aggregate the functions into a provisional hierarchy, such as that shown in Figure 2. From the point of view of a business pattern, there are issues with using a bottom-up approach:
Figure 1. Process discovery and synthesis
- This can be a very labor-intensive process for a large business area, involving many users. (This can be mitigated by compromising on a scenario + use-case approach rather than full process analysis).
- The nature of the process leads to an analysis of the "as-is" situation. This is acceptable as long as this is clear and the results are used accordingly.
- In order to be sure that you are documenting a pattern, you would have to repeat or at least verify the analysis in several similar enterprises. This could present many practical problems.
The benefits of the bottom-up approach are that the fundamental analysis has been now done and can readily be applied as a foundation for a solution derived from the pattern. The point of a pattern is to document successful practice, and given that the example has been well chosen, this will naturally occur.
The functional decomposition in Figure 2 could be derived another way. This could involve hypothesizing the upper levels of the hierarchy and filling in the levels below until a satisfactory degree of detail has been obtained.
Figure 2. Example of typical result of business function analysis
Clearly it is necessary to verify that the results are accurate and reflect reality. Thus, a Top-Down analysis starts by describing the business problem domain at its most abstract, and then iteratively decomposing until the primitive level is reached. This can be done by using a standard approach such as IDEF0 for functional modeling. Alternatively use the UML extensions for business process modeling.
In fact, the top-down approach need not address business processes at all. There are pros and cons to this. A benefit of using this approach in pattern definition is that the work can be done with industry consultants who have worked with many clients in the problem domain. In essence, one is performing "knowledge mining" with experts and this can be much faster. It reduces the number of examples you need to work with directly, and alters the analyst's role to that of reviewer rather than worker.
The Mixed Approach
In practice, it is often the case that we carry out the top-down and bottom-up approaches together, iteratively switching from process synthesis to hierarchy construction to decomposition to the next lower level. This has the benefit of verifying the hypothetical top-down decomposition with real world functions gleaned bottom-up. In practice this can be the quickest and most reliable method.
Functional Decomposition Example
The following example, and those that follow, are based on a real world situation in Patient Healthcare and show the deliverables for a core business pattern, using a process analysis approach.
Firstly, we carried out an analysis of the functional requirements of the problem domain. Working from provided scenarios such as that for breast cancer care, we have derived over 40 processes carried out by patients, professionals, system administrators and the "confidentiality guardian." This latter role is charged with the stewardship of confidentiality of information.
These processes were documented using UML use cases. These proved to be highly repetitious in that the same or similar sub-activities re-occurred within many use cases. Thus we extracted and consolidated these activities into a single list as follows:
- Patient Logon (Authentication)
- Patient Logoff (Audit)
- Apply Patient Search
- Manage Own Patient's Details
- Manage Own Patient's Preferences
- View Personalised Area
- View Own GP Details
- Manage Own General Health Data
- Manage Donor Details
- Manage Patient Details
- Manage Family Members
- View Immunisations/Vaccinations
- Manage Personal Preferences
- View Patient Health Records
- View Patient Journey
- View Patient Personalised Area
- Apply Clinical Override
- Review Clinical Overrides
- Generate Patient Events
- Manage Own Patient Events
- Construct Patient Journey
- View Own Patient Journey
- View Own Medical History
- Define General Consents
- Manage Own Consents
- Maintain Health Subjects
- Perform Health Code Translation
- Capture Other Health Code
- Model Other Health Code Structure
- Maintain Care Pathways
- Manage Book Appointment
- Change Appointment
- Access Event Detail
- Access System Index
- Maintain Clinical Processes
- Maintain Role Definitions
- Maintain Group/Team Structure
- Maintain Group/Team Membership
- Maintain Permission Delegations
- Maintain Professional Register
- Professional Logon (Authentication)
- Professional Logoff (Audit)
- View Own Prof Permissions
- Maintain Specific Permissions
- Define General Permissions
Figure 3. Healthcare example—functional decomposition
These processes may be represented in a hierarchy as shown in Figure 3. In so doing we have derived a higher level which summaries and contains these activities according to their subject and functional similarity.
The Healthcare example is underpinned by a comprehensive data model. We carried out an analysis of the data created and managed within the scope of our problem domain. This revealed 31 main entities such as Patient, Healthcare Professional, Patient Event, Care Pathway, and so on. These were defined. We identified candidate primary keys and principal attributes for these entities and mapped the relationships between the entities, including resolving any many-to-many relationships. The identification of these entities and their relationships was driven by the functional analysis being carried out in parallel.
As far as we are aware, all identified functional requirements are supported by the data model and vice versa. The 31 entities have been allocated to eight data subjects based on the cohesion of the entities in terms of the relative strengths of the mutual relationships. These groups are:
- Patient Consents
- Care Pathways
- Health Subjects
- Clinical Processes
- Roles, Teams and Organisations
- Professionals and Permissions
- Local Systems
These data subjects form the first pass definition of the required databases and their provisional content. The Patients' data subject is shown in Figure 4. This takes the form of a conventional Entity-Relationship model showing data entities named and identified by their primary keys. The entities within the colored area belong to the Patients data subject. The entities outside the colored area belong to other data subjects but have significant relationships with entities within the Patient data subject.
Figure 4. Example data model—patients' data subject
A "Crow's Foot" relationship notation has been used but an IDEF1x notation could have been used if preferred. All many-to-many relationships have been resolved. This is necessary because M:M relationships usually conceal further entities and relationships. This approach allows us to form a shallow hierarchy for data (Root > Subject Area > Entity > Attribute) and then to form relationships between the members of Data and the members of Business Function. This normally takes the form of a CRUD Matrix showing the actions of specific Business Functions upon specific Data Entities.
Having defined the required functionality in the form of a functional decomposition hierarchy and also defined the data required in an entity relationship data model, we can now derive a first cut component architecture by comparing the identified functions and data. We do this by forming a matrix, the rows of which are the identified functions and the columns the identified data entities. In the cells we place a value:
- "C" meaning this function CREATES an instance of this data entity
- "R" meaning this function READS an instance of this data entity
- "U" meaning this function UPDATES an instance of this data entity
- "D" meaning this function DELETES an instance of this data entity
The result is shown in Figure 5.
Figure 5. Function versus data CRUD matrix
We have ensured that every column (data entity) has at least one create operation and that each row (function) has some activity. Please note that the values are not absolutely precise, particularly with regard to read operations. We have not specified delete operations. We now analyze the matrix by using the affinity analysis and clustering technique. The objective is to deduce groups of functions and entities that share create and update operations. We are exploiting the loose coupling and tight cohesion notions used in the functional decomposition with the inter-entity relationships (some of which are vital and others merely transient) in the data model. We adjust the model to bring together functions and entities with strong affinity. Description of the detailed algorithm used (the "North West" method) is beyond the scope of this paper, however the result is the emergence of mutually exclusive groups of functions and entities formed round create and update actions. These groups are our candidate business components as shown in Figure 6.
Figure 6. Clustered matrix
Business component derivation
Firstly we should clarify what we mean by business component. A business function creates, reads, updates and deletes data. Grouping together all the functions that create and update the same data entities, using a technique such as commutative clustering, defines non-redundant building blocks—business components—that can be used to construct patterns, systems or applications that in turn support particular business processes. The business components sphere is an example of a derived sphere—one which is deduced from the relationships between two other spheres. This is a powerful technique that exploits hidden value in an enterprise architecture. By encapsulating functionality and data into components, software reuse and replaceability become practical. Further, components offer services that may be orchestrated in conjunction with the services offered by other components to create an application.
The business components resulting from our cluster analysis include service interfaces, business entity components, data access components and perhaps service agents. These artifacts align with the .Net Application Architecture5. The business component does not contain agile elements such as UI Components and UI process components, business workflows, or elements such as security, operational management and communication. We think this coarse grained definition is very useful. It provides the stable part of the overall solution architecture, which is then rounded out with the agile elements—like UI, UI Processes and Business Workflows. Thus we can provide an agile solution based on the foundation of a stable pattern.
The coarse-grained business components and their services are discovered through an affinity and clustering analysis performed on the relationships between business function and data. After these spheres are defined, a CRUD matrix is created to determine the component-data relationship.
Figure 7. Sample business component
In SAM, business component forms a hierarchy too. Thus the business component can decompose one level to specify the services, business entity and data access sub-components. At this level, business components will also specify the business services that we intend to expose. It is useful to explicitly call these out, so that decisions can be made around whether these will be internal services or exposed as web services.
Components and services
The list of components is as follows:
- Patient Component
- Professional Access History Component
- Patients' Events Component
- Patient Consents Component
- Health Subject Component
- Care Pathways Component
- Appointments Component
- GP & Hospital Systems Access Component
- Clinical Processes Component
- Groups & Teams Component
- Professionals Component
- Permissions Component
We think that this set of components represents the essence of a Business Pattern definition. One of these components - the Patient component is shown in Figure 7. This indicates the functionality, data managed and services and features offered by the component.
Figure 8. PRM with Architect and Design views
How the business patterns are useful for software engineering
Now we've reached the second part of this article where we talk about the roadmap for how to use the above descriptions of business patterns to engineer either other types of patterns, or actual software systems. We illustrate this using the five layers of the PRM as shown in Figure 8. You'll note that the PRM distinguishes between the architect's view (a higher level view, for example guiding a program of projects) and a designer's view (for example, design for a project or subset thereof). Figure 8 intends to indicate the different techniques and notations appropriate to these different views, just to give you a flavor for how the refinement is done. What is key to note is that the PRM simply identifies the elements needs to get right through to an implementation—it makes no assumptions or judgments about how best to achieve this! You can use whatever approach you favor to fill out the set of deliverables and in whatever order you want (after the business patterns are defined). You chose the best way for the organization that you are working in.
When considering business patterns we need to recognize this essential difference between architecture and design in the context of the side-bar about the right level of abstraction.
Where the business patterns fit
The sweetest spot for the business patterns is shown in Figure 9. Here they define those stable elements of a business, the business functions, data and business components that are in scope for the effort. Used this way they can be used to guide large programs of work, which leads to greater consistency across projects, with less overall effort.
Figure 9. Business patterns sweet spot in the PRM
Business patterns and Service Oriented Architecture
For those that want to provide either other IT patterns, or further guidance related to providing an IT solution for the business problem, the business patterns can be refined further with the elements of a Service Oriented Architecture (SOA is not required, but is an excellent fit; what is needed is an architectural elaboration to transform into an IT solution).
Figure 10. Adding an IT architecture to the business patterns
To do this we need to add the topics that we said provided the opportunity for business agility, as shown in Figure 10. The elaboration that we are describing here and in the next part could still be described in the form of patterns. The addition of these two sets of elements changes a business pattern into a business solution pattern. Alternatively, we could build a specific business solution architecture for the business pattern. We'll revisit this in a moment. Note that at this stage all the work is still technology product-independent.
Business patterns, SOA, and (Microsoft) product details
Finally we can elaborate yet further with product and practice recommendations for successfully implementing the solution on Microsoft technology, as in Figure 11.
Figure 11. Adding Microsoft technology elements
At this point we have delivered business patterns, a service-oriented application architecture, and a set of Microsoft patterns for implementation. At this stage we would have completed the delivery of a full set of linked patterns, or an IT solution, to solve a common business problem at the architecture level. But what if you'd like to actually build a system? You probably need more detail, which is what the design view will deliver.
The above deliverables provide a guiding architecture, which can be used to scope, costs and govern the IT projects that implement it. However, it is not detailed enough for solution implementation. If we want to provide a solution, we need further elaboration at the design level, which is guided by the architecture provided. It is probable that the solution will be implemented in several projects, and in this case the architecture is what keeps all the projects on the right track for cross project consistency.
Figure 12. Refinement for solution design
Figure 12 shows this next refinement. In the figure we show UML as it is a common way to drive a design phase. Again though we want to be clear that this is an illustration and there is nothing to say that is has to be done this way. All we are saying is that this is a common way to achieve the refinement of the architecture into a design for implementation.
While we are performing this refinement, we can again be creating patterns—IT design ones this time. In fact we are now getting (finally) to the territory that most of today's software pattern literature describes! Examples include Microsoft's Enterprise Solution Patterns Using Microsoft .NET6, and Martin Fowler's Patterns of Enterprise Application Architecture7.
Or, rather than patterns we can be creating an actual solution design. This is the end of the roadmap for this article. Clearly the last stage would be to implement the design, and that is where patterns such as those of the Gang of Four8 and the Patterns-Oriented Software Architecture set are very relevant, as well as the Microsoft and Martin Fowler patterns already mentioned.
What we have described in this article is a framework and approach for creating and then using business patterns through a guiding architecture that can be used to scope, cost and govern IT projects to implement it. To provide a solution, you need further elaboration at the design level, and this is guided by the architecture provided.
Examples of the Microsoft technology patterns can be found on http://www.microsoft.com/resources/practices/.
- Our definition of a pattern follows the classic definition created by Christopher Alexander in A Timeless Way of Building, Oxford University Press, 1979. When writing patterns we use a Coplien-style notation.
- For information on SAM, see Enterprise Architecture—Understanding the Bigger Picture, Bob Jarvis, a Best Practice Guideline published by the National Computing Centre, UK, May 2003 or http://www.systems-advisers.com.
- Problem Refinement Model—please see Article 1 in JOURNAL2.
- A member of the IDEF family of FIPS standards from NIST. See article number 183 at http://www.itl.nist.gov/fipspubs/by-num.htm.
- Application Architecture for .Net:Designing Applications and Services - Patterns and Practices Guide—Microsoft Corporation 2002
- See http://www.microsoft.com/resources/practices/ or Amazon.
- Design Patterns Elements of Reusable Object-Oriented Software by Erich Gamma, Richard Helm, Ralph Johnson, and John Vlissides. http://hillside.net/patterns/DPBook/DPBook.html
Disclaimer The opinions expressed in this paper are those of the authors. These are not necessarily endorsed by their companies and there is no implication that any of these ideas or concepts will be delivered as offerings or products by those companies.
About the authors
Philip Teale is a Partner Strategy Consultant working for Enterprise & Partner Group in Microsoft UK. Previously, he worked for the Microsoft Prescriptive Architecture Group in Redmond, and for Microsoft Consulting Services before that. He has 29 years of Enterprise IT experience of which four years have been with Microsoft and 16 with IBM, in both field and software development roles. His international experience includes nine years working in the USA, three years in Canada and 17 years in the UK. Phil's background is in architecting, designing and building large complex distributed commercial systems. His most recent contribution to industry thought-leadership was to drive Microsoft in the creation of patterns for enterprise systems development. He is a Fellow of the RSA. Philip can be reached at firstname.lastname@example.org.
Robert Jarvis is a Director of Systems Advisers Limited, a UK consultancy specializing in the development of Strategic Systems Architectures for major international enterprises. He is also an Associate Architectural Consultant with Microsoft Ltd. Bob has over 30 years experience as an International Systems Consultant and Architect advising business and governmental organizations in the UK, Continental Europe and the Americas. He specializes in Enterprise Architecture working particularly on the business/technology intersection. He is the author of Enterprise Architecture—Understanding the Bigger Picture, a best practice guideline published by the UK's National Computing Center in 2003. Robert can be reached at email@example.com.
This article was published in the Architecture Journal, a print and online publication produced by Microsoft. For more articles from this publication, please visit the Architecture Journal website.