Oracle Cloud Infrastructure Documentation

Create an Integration Task to Configure and Run a Data Flow

In Data Integration, an integration task lets you prepare a data flow for running.

You can add parameters to a data flow, so that it can be used with different values each time you run the task. Default parameter values can be updated within an integration task. You can then validate the data flow with the updated parameters, before you publish the task to an Application.

In this tutorial, you:

  1. Create an integration task to configure runtime context for a data flow.
  2. Create an Application to contain and run published tasks.
  3. Publish a task to an Application.
  4. Create and monitor a task run.

1. Creating an Integration Task

  1. In the tab bar, click Open tab (plus icon), and then select Projects.

    Open tab

  2. On the Projects page, click DI_Lab.
  3. On the DI_Lab project details page, click Tasks in the submenu on the left side.
  4. On the Tasks page, click Create task and select Integration.

    The Create integration task page opens in a new tab.

  5. On the Create integration task page, change the Name to Load Customers Lab.

    The value in the Identifier field is automatically generated based on the value you enter for Name. You can change the generated value, but after you save the task, you aren't allowed to update it.

    Entering a Description for the task is optional.

  6. In the Data Flow section, click Select.
  7. In the Select a data flow panel, select Load Customers and Revenue Data, and then click Select.
  8. Click Create and close.

2. Creating an Application

An Application is a container for published tasks, data flows, and their dependencies. You can run published tasks in an Application for testing, or roll them out into production.

  1. On the workspace Home page, in the Quick actions tile, click Create application.
  2. On the main Create application page, click Create blank application.
  3. Enter Lab Application for the Name.
  4. Verify or change the Compartment in which this Application is to be created.
  5. Click Create.

    The Application details page for Lab Application opens in a new tab.

3. Publishing a Task to an Application

  1. On the DI_Lab project details page, click Tasks in the submenu.
  2. In the Tasks list, click the Actions menu (Actions menu) for Load Customers Lab and then select Publish to application.
  3. In the Publish to application dialog, select Lab Application.
  4. Click Publish.

    A notification message displays. When you publish a task, a patch is created in the Application.

    You can still change the task or edit its data flow without impacting the published task. This means you can test a version of the data flow while working on other changes.

  5. Go to the Lab Application details page, and click Patches in the submenu on the left side to view details of the task patch.

    A patch contains updates to a published task in an Application. When you publish a task, a publish patch is created. When you unpublish a task, an unpublish patch is created. You can find a list of patches and their status on the Application details page.

  6. Click Refresh to get the latest status updates.

    When the status of the patch changes to Success, a published task entry is created on the Tasks list of the Application details page.

  7. Click Tasks on the Lab Application details page.

    The Tasks page shows the list of tasks you have published to the Application. You run published tasks from the Tasks page.

4. Creating and Monitoring a Task Run

  1. On the Lab Application details page, click Tasks in the submenu on the left side.
  2. In the Tasks list, click the Actions menu (Actions menu) for Load Customers Lab and then select Run.

    A success message appears. Running a task creates a task run on the Runs page of the Application. You're automatically brought to the Runs page, where you can view all task runs and their status. The initial status of a run is Not started.

  3. Click Refresh to get the latest task run status updates.

    Learn more about task runs

    When the run status turns to Success, you can see the values for Data read and Rows written.