Spark Dynamic Allocation is now Available with Data Flow
- Services: Data Flow
- Release Date: June 30, 2022
To help you save resources and reduce time on management, Spark Dynamic Allocation is now enabled in Data Flow.. You can define a Data Flow cluster based on a range of executors, instead of just a fixed number of executors. Spark provides a mechanism to dynamically adjust the resources application occupies based on the workload. For more information, see the documentation on Spark Dynamic Allocation with Data Flow.