Troubleshooting Data Flow
Use troubleshooting information to identify and address common issues that can occur while working with Data Flow.
Resources Exceed Quota Limit
Fix problems where the resources requested exceed to quota limit.
Your job won't run.
You get a message: The resources you have requested exceed the tenant's Data Flow quota limit.
Cause:
You have exceeded your quota of resources.
Remedy:
- In the Oracle Cloud Infrastructure Console, navigate to Governance.
- Select Limits, Quotas and Usage.
- Click Request a service limit increase, and request more quota for Data Flow.
Spark UI Fails to Load or Returns a 500 Internal Server Error
For in-progress Runs, the Spark UI might fail to load or give a 500 internal server error.
Whilst a Run is in progress, you can't load the Spark UI. It might return a 500 internal server error.
If the Driver is too busy running jobs or there are too many jobs to load, it might take longer for the UI to load and it might time out, resulting in an error.
Use a bigger driver shape
Reduce the retained jobs and stages
spark.ui.retainedjobs and
spark.ui.retainedstages are set to 1000. Set the values to
a lower number to try to avoid a timeout.