Real-time analysis of streaming data is catching on, but industry experts say CEP is still an immature technology.

by Ivy Schmerken, WallStreet & Technology, May 15, 2008

Complex event processing (CEP) is supposedly gaining traction on Wall Street as a way to analyze real-time streams of information and find patterns or correlations in unrelated data. But there are still many myths about the technology and confusion over the terminology, according to last week’s Accelerating Wall Street conference hosted by Wall Street and Technology.

One of the main myths is that the technology is mature. According to panelists, CEP technology is immature. Another myth is that it’s a sell-side only technology. A third myth is that it’s out-of-the box and easy to use.

Article

by K. Mani Chandy and Michael Olson
California Institute of Technology
Event-driven applications that are constructed as compositions of Web applications offer significant benefits. Just as mashups compose Web services to create added value, so too can compositions of event-driven applications create added value.

This article first reviews concepts about event driven applications, sense & respond systems, and presents metrics for evaluating and comparing such systems; then a few examples of federated Web applications are presented; and finally two examples of Business Application Monitoring (BAM) dashboards dealing with politics and with startup companies are presented as illustrations.

Download the article - Federated Event Systems: The Event Web (768kB)

by Joe McKendrick, ZDNet.

What is event processing and how could it make a difference? Here is one example of where it could take us: Scientists are talking about the possibility of an in-body network that could detect heart attacks or diabetic collapses and alert emergency services.

So we have the need-to-have, lifesaving value of event processing, and the intrusive, nagging aspect. Now, apply this kind of scenario with a business. That is, if the ongoing health of the business is monitored and maintained with intelligent technology, serious issues could be headed off either by alerting decision makers before things happened, or even in an automated fashion. But is it possible to have too much processing of events that aren’t as critical, and merely drive staff nuts? Article

Financial services companies are stronger adopters of technology in call centers,by Maria Bruno-Britz, Bank Systems & Technology, April 28, 2008

Banks are tops when it comes to their call centers, at least according to a study from Genesys Labs (San Francisco), a provider of contact center solutions. Genesys polled call center managers and technical support personnel at 82 financial services companies from around the world, including banks, credit unions, investment management funds and realty lending firms, for its Contact Center Realities Study. The survey concluded that financial services is among the leading industries in terms of effective contact center use. …

Adam Honore, senior analyst with Boston-based Aite Group, notes that banks increasingly are deploying technologies to facilitate cross-selling in various channels. “One of the newest technologies in play for financial services firms is complex event processing [CEP] engines,” he relates. “These engines are being used to foster cross-selling between channels.” Report

Wall Street and Technology, April 25 2008.

Intrigued by Vhayu’s announcement of a complex event processing tool geared toward quant researchers, yesterday I looked at this software. It gives quants a means to look for patterns, test ideas and strategies and run what-if scenarios against massive volumes of real-time and historical tick data. It combines some of the features of mathematical algorithmic modeling tools (such as Numerix and Algorithmics) with a fast historical and real-time tick database (a la Vhayu, Kx) with a front-end complex event processing graphical user interface (in the vein of Progress Apama, Streambase or Coral8). Report.

April 25th 2008: An update to a previous article

A previous article described the event processing requirements needed to support “Global Air Traffic Management“. The main theme of a number of articles published on this site is that the underlying event processing requirements for many global monitoring systems such as air traffic control, epidemiology monitoring, climate change monitoring etc. are very similar. Perhaps they could all be supported by the same basic event processing infrastructure.

However, hidden in that article is the caveat,

“As with any system integration, there will be political issues, especially where funding is concerned. These issues go beyond the scope of this article.”

Now, in the case of air traffic control systems, these issues have bubbled up to the surface of the New York Times, and elsewhere!

Download the article: The Cost & Politics of the Air-Traffic Mess

Politics is also surfacing with technical concerns. Matthew Wald in the New York Times, April 25th 2008, “Fault-Finding in Dallas Between Controllers and Pilots Becomes a Problem for the FAA”, surveys the issues.

“Air traffic control supervisors in Dallas, caught in 2005 hiding instances in which planes had passed too closely, switched to blaming pilots for the incidents when they were actually the controllers’ fault, aviation officials said Thursday.”

The article goes on to summarize the main technical issue very simply,

“The F.A.A. relies on accurate data so it can identify trouble spots and fix them before they contribute to a crash, he said. One category of trouble spots is air traffic routes that are difficult to navigate and, like a badly designed highway intersection, pose a disproportionate danger.”

and the political issue as well,

“Another problem is that bonuses for controllers and supervisors depend in part on the rate of reported “operational errors,” like close encounters. “You don’t want to have an incentive for people hiding the ball,” he said.”

Clearly, an automated monitoring system would go a long way towards a solution to both problems.

Big Blue gets into BPM, while trying to mix social networking with SOA. - Apr 8, 2008 - by Andy Dornan, NetworkComputing.com

Think complex event processing sounds too complex? IBM can help. It has renamed the technology business event processing in one of several announcements it’s making at Impact, its annual SOA conference that starts today.

Though the conference is nominally about SOA, IBM’s two biggest product launches are both focused on BPM. As well as the new spin on CEP, it’s releasing the IBM Business Process Management Suite, a bundle that’s essentially BPM in a box: a complete stack that contains a modeling environment, fabric, and all the underlying SOA components needed to get BPM working. This focus on BPM isn’t a huge surprise. Many vendors (IBM evidently included) believe that BPM is the killer app for SOA, thanks to its relatively rapid ROI for some users.

Called WebSphere Business Events, the new CEP product has more than just a friendlier-sounding name. IBM says it’s also trying to make the technology more accessible to nondevelopers — which means any staff member within an organization, not just business analysts. The theory is that though the processing itself and the pattern recognition on which it depends may be complex, CEP ultimately follows rules that can be set by anyone. Read the complete News report on NetworkComputing.com.

See also:

IBM Business Events challenges TIBCO.

LONDON - England - The new Heathrow Terminal 5 which opened three days ago has gone down as a huge success with passengers and staff.

“Terminal 5 has incorporated a revolutionary technique in processing passenger luggage and speeding up the process of boarding a plane.” “It is quite easy, passengers simply drop their bags off and never see them ever again,” an operations manager for BAA told the Daily Squib.

Excellent customer service
On one of the departing planes, passengers on flight BA0469 to Benidorm were spirited through the Fast Bag Drop in record time and their flight left on schedule. One, Elizabeth Drusus, told the Daily Squib the captain said they would be leaving without any luggage. They had been told this was part of the new BA system where passengers were encouraged to travel light.”We are encouraging travellers to forget about essentials like luggage,” an airport official said.” Read the Daily Squib. (the world’s finest news source)

Reality checks:

• Getting to grips with UK’s biggest WLAN
• Cancelled flights and baggage chaos mar Terminal 5 opening
• Simon Calder: Terminal turmoil for British Airways
• BAA’s Terminal 5 baggage woes (no RFID)
• http://www.it-director.com/business/content.php?cid=10384

EPTS - the Event Processing Technical Society - is going to be officially launched in late May or early June (exact date – TBD). EPTS has existed as an informal society for the last two years, holding three Event Processing Symposiums to date; for details see:

• 1st Workshop on Event Processing - Presentations
• 2nd Workshop on Event Processing - Presentations
• 3rd EPTS Workshop on Event Processing - Presentations

EPTS is now being officially founded to promote all areas of event processing, including theoretical foundations, design and architecture of event processing applications, implementation techniques, use cases and applications in all areas of business, government and the military. Organizations, companies and individuals are invited to join at this time as founding members.

Download the EPTS Symposium call for participation

by Doug Henschen, in Intelligent Enterprise, March 12, 2008

“Follow the money.” This approach to investigation, applied by criminal prosecutors going back before Eliot Ness and made famous as a line in the movie “All the President’s Men,” is exactly how soon-to-be-ex New York Governor Eliot Spitzer was tied to a high-end prostitution ring. In this case it was fraud detection technology, of the kind routinely applied by banks in money laundering investigations, that led directly to Spitzer and to his resignation.

… The detection, of course, was done by technology routinely and automatically applied to spot transaction patterns indicative of money laundering — not by bored bank employees randomly dipping into high-profile accounts on some sort of fishing expedition. Doug Henschen’s analysis.