Creating an OCI Data Flow Task
An OCI Data Flow task in Data Integration is associated with an existing application that's created in OCI Data Flow.
Before you create an OCI Data Flow task, ensure that you have the required policies, setup, and information you need for working with OCI Data Flow tasks in Data Integration, as described in Required Policies and Setup.
Create an OCI Data Flow task in a project or folder. Data Integration includes one default project to get you started. To create another project or folder, see Projects and Folders.
In Data Integration, by default you can have simultaneous or parallel task runs of a task at a given time. To disallow concurrent task runs that are initiated manually, select the Disable simultaneous execution of the task checkbox when you create the task. When simultaneous task runs are disallowed, a run request for the task fails if there's already a task run in progress that's in a non-terminal state.
Flexible shape usage considerations:
-
When you use a flexible shape (such as VM.Standard.E4.Flex) for the driver node, executor node, or both, customize the number of OCPUs and the amount of memory that you need.
-
A flexible shape provides a specific range of OCPUs values that you can use for that shape.
-
The number of OCPUs that you use for a shape determines the range of memory values that you can allocate.
-
If you parameterize the driver or executor shape, OCPUs and memory must be configured for the shape. The values of OCPUs and memory are used only when the shape parameter value is a flexible shape. The OCPUs and memory values are ignored if a non-flexible shape is configured in the parameter.
Publish the OCI Data Flow task to an application in Data Integration before you run the task or schedule the task for running. For publishing information, see Publishing to a Data Integration Application. Use the oci data-integration task create-task-from-dataflow-task command and required parameters to create an OCI Data Flow task:
oci data-integration task create-task-from-dataflow-task [OPTIONS]
For a complete list of flags and variable options for CLI commands, see the Command Line Reference.
Run the CreateTask operation with the appropriate resource subtype to create an OCI Data Flow task.