Data Flow now Supports Structured Spark Streaming
- Services: Data Flow
- Release Date: January 26, 2022
Streaming applications require continuous execution for a long period of time that often extends beyond 24 hours, and might be as long as weeks or even months.
Data Flow:
- Implements a graceful shutdown of the Spark clusters for Spark structured streaming to allow for regular maintenance.
- Uses Oracle Cloud Infrastructure Streaming.
- Uses Spark structured streaming check-pointing to restart from the point of failure without producing incorrect computational results.
- Provides access to the Spark UI and Spark History Server.
For more information see the Spark Streaming chapter in the Data Flow documentation.