Vendors and Products in the Event Stream Processing Market

7 March 2021 W. Roy Schulte

*****This represents my personal opinion, not the position of my employer, Gartner Inc., or any other entity. The vendors and products listed here are examples and not a complete list of all participants in the market.*****

The proliferation of streaming event data is driving more use of event stream processing (ESP) software platforms. ESP platforms provide development-time and run-time facilities that make it easier for developers to build applications that process streaming data “in motion.” They operate on event data as it arrives before (optionally) landing it into a database, file store, or in-memory data grid.

ESP applications generate complex events by performing stateful calculations on moving time windows of event data. Some ESP applications implement stream analytics to update real-time dashboards, send alerts, forward intermediate results (complex events) to other applications, or trigger immediate actions in response to situations that have been detected (i.e., sense-and-respond behavior). Other ESP applications implement stream data integration pipelines to filter, enrich and transform input data and then load the data into data stores for subsequent processing (typically for analytics).

We can sort modern general-purpose ESP platforms into three major categories:

A. Free, community-supported open source
B. Commercial, vendor-supported “open core” products that combine proprietary value-added capabilities with an open-source core
C. Commercial, closed-source products

Vendors also offer a wide variety of purpose-specific, vertically- or horizontally-oriented products that have embedded ESP platforms. These sometimes embed a general-purpose, open-source ESP platform (see (A)) but otherwise provide their own custom-built ESP capabilities. Examples of these include IoT platforms, stream data integration products, and supply chain visibility, security information and event management (SIEM), asset performance management, fraud detection, real-time customer relationship and other applications.

Details by Category

A. Free, community-supported open source

User organizations choose this kind of ESP platform when they want to avoid paying license fees while leveraging the work of multiple other software developers. Pure open-source products appeal to risk-tolerant buyers who are willing to develop some ancillary utilities or perform manual operations themselves. Apache Flink is probably the best known of the current open-source offerings. However, Apache Spark Streaming, Apache Storm, Kafka KSQL and others are also in wide use.

One of the newest open-source products is IoTPy, a free, community-supported open-source Python library. The name reflects the fact that IoT applications that continuously monitor event streams from physical objects are the fastest growing kind of ESP application (for all ESP products not just this one). However, IoTPy is a general-purpose library that can also be used to build many other kinds of stream processing applications. It wraps conventional (non-streaming) Python code from Scikit-learn and other popular ML libraries to implement simple or sophisticated geo-distributed streaming applications that can run on very small edge devices, cloud platforms or anything in between.

Examples (origin in parenthesis):
1. Apache Flink (Alibaba Ververica)
2. Apache Samza (LinkedIn)
3. Apache Spark Streaming (Databricks)
4. Apache Storm, Apache Heron (Twitter)
5. Drools Fusion (Red Hat)
6. Esper, Neser (EsperTech)
7. IoTPy (PyPI.org, www.assemblesoftware.com)
8. Jet (Hazelcast)
9. Kafka ksqlDB (Confluent)

B. Commercial, vendor-supported “open core” products

These products appeal primarily to risk-averse organizations that want to leverage the benefits of an open-source core from multiple developers but that also want the assurance of support from a vendor and the benefits of added-value, differentiated development and management features.

Examples:
1. Alibaba Ververica Platform (Flink)
2. Amazon Kinesis Data Analytics for Apache Flink
3. Confluent Platform ksqlDB
4. Crosser Cloud, Node
5. EsperTech Esper Enterprise Edition
6. Google Cloud DataFlow (Beam), Cloud IoT Edge
7. Hazelcast Jet
8. VMware Tanzu Spring Cloud Data Flow
9. WSO2 Stream Processor, Siddhi

C. Commercial, closed-source products

Some of the most popular ESP platforms are proprietary, closed-source products. These products appeal to risk-averse buyers who require vendor support and typically are buying lots of other software from the same vendor. Most of these products come from major vendors with vast product lines.

Examples:
1. Amazon Kinesis Data Analytics for SQL
2. FASTDATA.io PlasmaENGINE
3. Hitachi Streaming Data Platform
4. IBM Streams
5. Microsoft Azure Stream Analytics, Stream Insight
6. Oracle Stream Analytics, Stream Explorer (partly open core based on Spark)
7. SAP Leonardo Edge Services
8. SAS Event Stream Processing
9. Software AG Cumulocity IoT Analytics (Apama)
10. TIBCO Streaming