streaming frameworks vs reactive programming frameworks
Streaming Vs. Reactive programing
Recently, in a Tikal Fuse Day event, we tested several frameworks that are built to provide Reactive Streams, like Akka, Vert.x, Java 9 “Flow” API and others. all those frameworks provide reactive messaging systems that produce events on demand, according to the speed of consumer (“back pressure”).
On the other hand, we heard of other streaming processing systems that produce high throughput of events to buffers (queues) then consume them. Some examples are Apache Storm, Apache Flink, Spark Streaming and more.
What is the different between streaming and reactive programming?
Reactive streams and streaming processing can be used for the same use cases but they are not exactly the same
Different use cases
Let’s consider the following use case: We develop a game to be installed on smart-phones. The users play one against the other in groups. We expect that this game will be viral.
Use Case 1
A web-server application reactively responds to the users playing our great game and needs to process high amount of events in real-time.
Use case 2
An ETL service collects the events from the web-server and sends them to a key/value database to show statistics and aggregations of the current scores of the game.
Implementing the use cases
For Use Case 1, we will choose a reactive programming framework. For example: Vert.x. Vert.x will reactively process each event and may pass it through several reactive processes in a Micro-services architecture.
For Use Case 2, we will choose a streaming framework. For example Spark Streaming. It will collect all the load of the events, process them, calculate statistics and aggregations, and store them to some key/value store like Redis to be consumed by a dashboard.
People tend to confuse between streaming frameworks and reactive programmings frameworks, because they both deal with high throughput of real-time events. The different is with what they do after consuming the events: Reactive systems are used to provide a quick action for each event. Streaming systems are used to collect the data from the events and store it in files or databases after aggregations and other calculations.