CEP and Big Data 2

by: Philip Howard, Research Director – Data Management, Bloor

There have been a couple of things floating around in the ether about CEP (complex event processing) recently. The first is the question, supposedly credited to Curt Monash, of whether it should be called something different.

I’ve been going back through my records. When I first wrote a product evaluation of what is now Progress Apama in 2002, I stated that “the company’s contention is that conventional approaches to real-time queries are only suitable for small scale environments or those in which limited numbers of data feeds are being monitored. In particular, its view is that these solutions cannot cope with environments where large numbers of data feeds need to be combined in a complex and dynamic fashion.” That was the only use of the word “complex” in a total of nearly 3,000 words. Events were mentioned several times, streams not at all.

As an aside, look at Wikipedia and other sources about the development of CEP and you’ll see lots of mentions of David Luckham, who coined the CEP term in his book “The Power of Events”, published in 2001. You will also see various other attributions to American scholarship but no mention at all to Cambridge (that’s UK not Harvard), which is where Apama came from. I guess all the writers are American. …..

…… What I didn’t mention was that there are a number of companies/products that have been specifically designed to work in conjunction with Hadoop, namely HStreaming, S4 (from Yahoo!) and Storm (from Twitter). I haven’t looked at any of these in detail so can’t comment on them but that is definitely an area that is heating up.  Read the Blog

DCL: John Bates’ PH.D. work on event processing at Cambridge in the 1990’s should definitely be credited in any account of the development of CEP.