Hey everyone👋
Check out this amazing article on Stream Processing
Here are the highlights for you:
-
Stream processing is a technology used to analyze continuous data streams and quickly detect conditions. It can provide insights faster, often within milliseconds to seconds from the trigger, making it suitable for real-time analytics.
-
Stream processing is also called real-time analytics, streaming analytics, Complex Event Processing, real-time streaming analytics, and event processing.
-
There are several reasons to use stream processing, including handling never-ending data streams gracefully and naturally, fitting with time series data, spreading the processing over time, handling large fire hose-style data, and being more natural for IoT use cases.
-
However, stream processing is not a tool for all use cases, and it is not suitable for processing needs that require multiple passes through full data or have random access.
-
To implement stream processing, you can either build the app yourself or use a stream processing framework. An event stream processor lets you write logic for each actor, wire the actors up, and hook up the edges to the data source(s).
-
Some examples of stream processing frameworks are Storm, Flink, and Samza.
-
Since 2016, a new idea called Streaming SQL has emerged, which enables users to write SQL-like queries to query streaming data. Many stream processors support processing data via a streaming SQL language.
Read the full article here⤵️