The Rules Engine That Saved the Day (But Not the Application)
Summary: With our BRE implementation plan, we did not build the perfect beast; but, at least, we provided the heart. (6 printed pages)
"I can't stand it any longer! They keep changing the requirements. I've had it: I'm going to go work in fast food or something! At least, I won't have any problems with thin clients there!"
The voice belonged to Chris, the lead developer who was working on a five-month-old application-development project. The application dealt with loan origination, and it was subject to all kinds of rules, regulations, and associated activities. I had been invited onto the project earlier in the week, and I was just getting up to speed with all of the gory details.
Chris was speaking the truth: The business analysts had been changing the requirements frequently. As far as I could tell, this was the fault of both the business analysts, who kept uncovering new or changed requirements, and the marketing department, who kept adding perceived "killer features." In the four days that I had been on the project, we had already had two major shifts in functional requirements. I shared Chris's frustrations, but I had not yet had enough time on the job to express credibly that I felt his pain.
"I'm sorry, Chris. I know all the changes make things exceedingly difficult. I've voiced my concerns to the major stakeholders already, and will do so again.
"That's what the last guy said, too. It seems we don't have much leverage over management or marketing, do we?"
I had to admit that it was true. My earlier conversations had gotten me nowhere.
"Isn't there something we can do? Anything would be better than death by rework. I've been doing some reading and think maybe a workflow engine could save us a lot of trouble."
I gave it a little thought. I had worked with workflow engines previously, but I did not see a good fit in this situation. Another thought crossed my mind: There was another application building block that might come in handy.
"Well, I'm not sure about a workflow engine. The changes I've seen so far, and the changes you've told me about, seem to center on business logic that doesn't much change the way the user interacts with the application. There is another tool in our toolbox, though. We might be able to use a rules engine."
Chris perked up a little. "A rules engine, huh? Isn't that sort of like workflow?"
"In some respects, it is. They both allow the user to script rules without having to write code, but there are many more differences than there are similarities."
"Really? Tell me a little about them. It sounds really interesting."
"Well, I suppose I can. Let's start with workflow."
"Workflow can be described from many points of view. Let's start with the point of view of the process designer, the person who is responsible for scripting how the application is supposed to interface with users and software components. The process designer uses a GUI that allows them to write processes. A process is a series of activities and the control structures that connect them. The whole thing can look like a tool that is used to draw flowcharts to the process designer. The interesting thing is that the flowcharts actually determine how the application behaves!"
"That sounds cool. Tell me a little more about the control structures that connect activities."
"Sure. That's a good question. The application needs some way to transition from activity to activity. The simplest mechanism is a straight connection: After activity A is completed, activity B starts. Instead of a straight connection like that, you might go for a decision node. Say that we're dealing with an inventory-control system. You might say activity A is part of a customer order. You would follow that up with a decision node that dictates what to do next. If the part that the customer wants is in stock, you proceed to activity B; or else, you go to the procurement step with activity C."
"So, your process carries data and can query the values to see what to do next?"
"Yes, that's right. Still other control constructs might have to do with the timing of when activities are allowed to happen. You might have more than one activity that you want to occur in parallel, or you might have several things that you want to execute in parallel, but you want to continue on with the process only after all of them are completed."
"Hold on! What's that supposed to mean?"
"Say that your task is to bake cookies. Maybe your process would be first to buy the cookie dough, then to preheat the oven, then to put the cookies on the cookie sheet, and then to wait for the cookies to bake. You could get away with graphing the process, so that you preheat the oven and put the dough on the cookie sheet at the same time; you just can't go on to the baking activity until both are completed."
"Oh, I see. Sort of like managing threads in multithreaded programming. You can do a join on all the hanging threads before continuing on!"
"Exactly. Really, the design tool is a lot like a GUI for drawing flowcharts; but the boxes in the flowcharts represent interactions with users or software components. Another important topic from this point of view is the way in which work is presented to the end users. Many workflow products have their own work queues, which are often Web-based. If you don't mind losing a little control of this part of your application, you can sometimes use these right out of the box, and leave to the engine the management of which tasks go to which users, which ones are done, which ones need escalation, and so on."
"Wow! That sounds pretty powerful."
"It is. But, if you don't want the workflow engine to control that aspect of the application, you'll have to work out some sort of scheme in which your application wraps those interfaces, though."
"That's a lot to think about."
"You're right. There is a lot to consider. But there are other aspects of workflow that an architect needs to think about, too."
"The way in which the workflow product itself is constructed. Some are made for long-lived processes; others are made for short-lived processes. A loan application, for example, is a long-lived process. First, the application is filled out; then, the application is validated. The credit check has to occur, an underwriter should review the whole thing, papers have to be printed, and funds must be disbursed. Finally, the loan must be booked on the host accounting system. This can all take days, or weeks."
"Yes, I see. I can even picture how the process definition might look, as a sort of flowchart. But how does that differ from a short-lived workflow?"
"Well, a short-lived process is more often seen in the user-interface portion of an application. A good example might be the flow of screens users see in one interaction with a Web application. If users make one choice, they get screen A next; if users make a different choice, they might get screen B. Can you see how the series of screens with which users interact might also be a type of workflow? This type might never have a process that lives more than a minute or two. This type also can be used to compose GUI wizards and other multiscreen interfaces. This shorter-lived kind of workflow might be called page flow or screen flow.
"Cool. But both types of workflow take a process template from the process design and then guide the application through the state changes, right? Can a single type of workflow engine be used for both kinds?"
"Well, I guess it's possible, but it would probably be tough for a single engine to handle both types well. The long-running types are usually pretty hefty applications in themselves; they usually rely on databases to make sure state changes are persisted, and they can be somewhat intrusive to install into an application. Workflow engines of this type usually require your application to work heavily with their APIs. These kinds of workflow products can be fairly intrusive and can really affect your application's design."
"Yeah, it sounds like it might. How about the shorter-lived flavor, the one that's appropriate for scripting GUI components?"
"Well, depending on your philosophy concerning application recovery in the case of server failure, you can get away with a much lighter and more nimble engine there. There might not be a database involved, and implementation can be as simple as adding a few prepackaged components to your front-end framework. This type of workflow can really be as simple as a pretty basic state machine, but you might have a need for more than the very basics."
"Okay, I think I'm starting to get the workflow picture. It's all about letting the user define processes with a friendly GUI, and then having the engine track what actions should come next as a process instance executes. Is that about right?"
"Yes, you've got it."
"So, how about a rules engine, then? Why do you think a rules engine would be better for us than a workflow engine?"
"In this case, I'd favor a business-rules engine, or BRE, because we aren't often changing the sequence of activities in which the user takes part. We are changing the business logic often, though. That's exactly the place in which a rules engine can add value. A rules engine is used to externalize and ease access to your business logic. It, too, can come with a GUI that the power user drives to write business logic—sometimes, in a language that can be pretty close to plain English."
"So, the business analysts can make changes in some sort of friendly user interface, and it changes the application logic?"
"Yes, it's true. Not only that, but they can write the rules using business speak. So, users can describe the application logic in terms of the business domain, which is how they like to write specifications and requirements. With a rules engine, the translation layer from business speak to technical speak can be thinned out quite a bit."
"Wow! I bet the BAs love that!"
"I think they do love that part. The part they're not wild about is the responsibility that comes with the power, though. When the business analysts change the rules, they are responsible for validating that the changes do what they're supposed to do. That's a burden that's traditionally fallen to the programmer."
Chris' eyes lit up. "Yes, I see how that would let the BAs have more control, and also more responsibility. They can change requirements as often as they like, but they have to implement and test the changes! Wonderful! So, are all rules engines alike?"
"I'm afraid that's not the case. Some use English-like language syntax, some use arcane specialized languages. Some excel in managing rules, a lot like source-control management (or SCM); others don't pay much attention to it. Some are best for executing a bunch of rules, when some action like an incoming transaction causes a sort of chain reaction to begin; others sort of monitor a bunch of data, and fire specific rules when certain conditions arise. There are different kinds of rules engines, and they have different strengths and weaknesses."
"Yowsa! A BRE does sound like it might help us, though. Let's see if we can find one that'll work for us."
Chris and I started our BRE implementation plan by working through our own application first. We carefully examined the application, extracted the business logic that tended to be volatile, and ran tests with a simple configuration-driven model before we went whole hog and plugged in a rules engine. The business analysts instantly took a liking to the whole thing; the project managers were ecstatic over the quick turnaround times for change requests; and the developers loved being able to concentrate on writing new code, instead of constantly reworking the same patches of logic. Everybody won; except that, at the time, we just did not realize how our application would finally lose out in the end.
It turned out to be a classic case of win the battle, lose the war. Our application was shaping up nicely—about three months away from release date—when management announced that our line of business had been bought by a competitor. Nobody lost their job (we all got to stay, branded under a new name), but our application was slated for "integration" with the new company's existing flagship product. In a stunning twist of irony, the hard-coded application flow that we had held so firmly was cavalierly tossed away, along with our core business rules; but the rules-engine framework that held it was moved into the new killer app and retrofitted with their business rules. We did not build the perfect beast; but, at least, we provided the heart.
· When might you want to use a workflow product, instead of a rules engine?
· What are the pros and cons of using embedded versus application programming interface (API)–integrated workflow products?
· How might your development team have to adjust, if you use a rules engine?
· "JSRs: Java Specification Requests" (contains papers that concern rules-engine standardization efforts). Java Community Process Program Web site: http://jcp.org/en/jsr/detail?id=94 (Accessed January 10, 2007.)
· Von Halle, Barbara. Business Rules Applied: Building Better Systems Using the Business Rules Approach. New York: John Wiley & Sons, Inc., 2001.
· Workflow Management Coalition Web site (vendors, analysts, and users contribute to a common understanding of workflow): http://www.wfmc.org/ (Accessed January 10, 2007.)
Escalate—To increase the value that represents importance. When a task has gone past a reasonable deadline, it might be escalated by the incrementing of an attribute, or by the issuing of a page or e-mail.
SCM—Source-control management. The practice of organizing a changing source code base.
Work queue—A list of tasks for a user or an agent to accomplish.
About the author
Rick Wagner is a developer and architect who has nine years of experience in workflow and rules engines. He first evaluated workflow products when state-of-the-art products delivered task assignments by using e-mail, a practice that happily has fallen out of favor. In his off-work time, Rick likes to spend time with his family, run, and play the piano. He lives in rural Arkansas.
This article was published in Skyscrapr, an online resource provided by Microsoft. To learn more about architecture and the architectural perspective, please visit skyscrapr.net.