Why Business Activity Monitoring Must Use CEP

The dashboard model of business activity monitoring (see article) is the basic model for the current generation of BAM tools. The essential idea is that these tools, when configured properly on your IT layers, will alert you when events happen that have significance for your enterprise’s business goals and operations. These tools work by detecting and processing single events from the underlying enterprise IT layer. This is simple event processing. Some tools may let you specify event-condition-action rules that are triggered when significant events happen. This is a step towards automating the real time actions you want to take to manage your enterprise.

I predicted in that article that if you bought a BAM tool and got it installed and working on your IT layer, it wouldn’t be long before you would be asking for more than the current version of the tool could do. In other words, you’d find it helpful in managing your business. And using it would give you ideas about what else you’d like to do. Some BAM vendors have already predicted this too, and are working like crazy – building the next version of their tools. They think they know what you might want their tools to do next. So the coming year in the BAM market is going to be an interesting one!

I’m going to make a few guesses of my own. Three of the trends are business policy monitoring, business impact analysis and probable cause analysis, all of which have many variations and approaches. I will not contribute any more three letter acronyms, but you’re welcome to call them whatever you like.

Starting from the dashboard BAM capability, here are things you can expect in the next year. I’ll explain what I mean by each of them below. By the way, some tools may do some of them now. Here are five levels of increasing BAM capabilities:

BAM + a predefined set of policy constraints
BAM + roll your own policy constraints (BAM+)
BAM + probable cause analysis (p-cause)
BAM + business impact analysis (impact)
BAM + + p-cause + impact + programmable corrective action rules
Tools that monitor conformance to a predefined (out-of-the-box) set of policies are out there now. For example, you might want to monitor for a policy that “the wait times for access to the financials database and the customer database should never together exceed 30 seconds”. The tool will monitor this policy because it has a built-in capability (out-of-the-box) to monitor metrics such as the wait times on each resource, and it understands simple arithmetic. If some of your business workflow processes accessed both data bases you might want this policy monitored to keep them running on time. Of course, you could change the 30 second time limit to something else on the fly.

Let’s start with level 2 on my list.

Roll your own business policy constraints. You might have some new policies that you’d like to know are not being violated by the business processes of your enterprise. Lets take a simple example for illustration. Suppose you’re trying to improve your “demand-driven” restocking of inventory to keep purchasing costs down. Ideally you would purchase stock only when it is needed. Suppose you have a very simple design for your processes consisting of two concurrent workflows both triggered by a customer’s order. One is an order fulfillment process that draws on inventory to fill the order. The other is an inventory control process that, concurrently, sends out restock events to replace the items in the order. Some of those restock events may be unnecessary, resulting in increased costs. You want to enforce a basic policy, “don’t order stock until a customer’s order reduces current inventory to a critical level.”

Standard practice today uses a software package that tracks inventory to make decisions in the restocking process. Such packages are often driven as much by supply chain delays as customer demand. But in an event driven enterprise one design issue may be to maximize the speed and simplicity of the processes while allowing policies to be changed on the fly to meet demand. How do you do this?

Well, you have a choice. You can try adding a new test to the restocking process every time you change the policy so that it won’t produce a restock event that violates your new policy. Or you can keep your process lean and mean – no new additions. Instead you introduce a separate test to be applied to each restocking event it spits out, intercepting the ones that violate the policy. Notice, the tests and the process will execute concurrently. The new level 2 BAM tools will let you follow the latter strategy. These kinds of policies, particularly the definition of critical level, might vary week-to-week depending upon fluctuations in predicted demand, time to deliver stock, finance interest rates etc. So this strategy may be preferable – you don’t want to be altering your basic processes every week.

In an event driven world, where a customer’s order is an event that triggers inventory restock events, this kind of policy is called a constraint. Its role is to constrain inventory purchases so that they satisfy a “just-in-time” test. It’s a negative statement, do not allow a restock event unless the inventory level in that item is critical.

Of course, you have to describe the constraint precisely, by specifying what “critical level” means. The constraint must disallow a restock event if the customer’s order that triggered that event will not reduce inventory to a critical level when that order is filled. This requires taking account of outstanding restock events which have not yet been filled (i.e., corresponding delivery events have not yet happened). And you may wish to factor in expected delivery times for items, and predicted future customer orders. So, a just-in-time constraint uses the current state of inventory and the set of outstanding inventory restock events in making its decision whether or not to allow a new restock event to go forward. We are entering an area of complex event processing – constraints over sets of events.

A level 2 BAM tool will provide an interface to let you define constraints precisely. The interface is intended for use by business managers, not IT engineers. You define your own policy constraints — they don’t come out-of-the-box. There’s a fairly sophisticated declarative language and compiler hiding behind that interface to support this activity. This means you roll your own constraint, specifying the set of events it applies to and how to decide when inventory is at a critical level. The decision can use the inventory tracking software package too.

A few final remarks are in order. First, the simple example I’ve chosen is not simple at all! But, with a level 2 BAM tool you can experiment with constraints that embody various fast tests for “almost just in time”, some simpler than others. Maybe the financial savings won’t change much. You can use simple constraints or complicated constraints, its up to you. The BAM tool will let you change your constraints on the fly so you can experiment without modifying the fundamental processes. In fact level 2 BAM can be viewed as a way to keep complexity out of the processes. Tests to see if a process will violate a new business policy when it produces an event aren’t added to the process. The process simply produces the event and concurrently the event is tested for conformance to the constraint before letting it go to its destination.

Keep those processes simple so you know what they do! Frequent changes and updates to your business processes may result in unpredictable behavior. Lose control of your business processes and you might find you’re out sourcing the whole of the enterprise’s IT – that has happened!

There are myriads of other examples of policy constraints, and I’m sure you’ve all got your favorites, particularly in the area of security policies – especially policies constraining insider activity. An event driven enterprise needs to install level 2 constraint monitoring as a sanity check to test conformance to its critical policies, maybe not all policies, but the ones you really care about. And if there are violations, then we come to the next level of BAM tools – probable cause analysis. I’ll deal with the other levels of BAM and why they too need CEP, next time.

© David Luckham 2004