Billing

Data Flow tracks underlying compute, block storage, and other resources' times when Spark has requested or released an executor.

Data Flow starts usage recording when the actual resource is made available, and stops usage recording when the resource is released. About a minute's lag time occurs from when a resource for Spark executor is allocated, and when the Spark executor is registered with Spark driver. This time is taken to start the Spark executor instance and perform a handshake with the Spark driver. Similarly, a minute's lag time occurs from when the resource is removed from the driver and when the resource is released. This time is taken to shut down the resource.