What are Data Flows

Data flows in Data Integration define the flow of data between source and target data assets, and any operations on that data. Data engineers and ETL developers can then analyze or gather insights and use the data to make business decisions.

To design a data flow in Data Integration, you use components called operators to build the flow of data from source to target, and apply transformations on attributes to aggregate, cleanse, and shape the data. You can also use parameters to externalize and override values so that different configurations of sources, targets, and transformations can be reused within the same data flow.

You can use Data Integration built-in functions with operators in the data flow to build expressions that process data or extract data values from arguments passed to the functions. You can also create custom (user-defined) functions for use in expressions.

You create data flows, and access the data flows from a project or folder details page.

The Data flows section of a project or folder details page shows the list of data flows created in that project or folder. To quickly find a data flow, enter the complete name of a data flow in the Filter by name field, and press Enter. You can also use the Favorite filter menu to display only those data flows that are marked as favorite. Clicking Clear removes the filter.

On the list of data flows, the Actions menu () for a data flow has these options:

• View details: Displays the data flow for editing.
• Copy data flow key: Copies to the clipboard the key of the data flow, which you can use with Data Integration API.
• Duplicate: Lets you make a copy of the data flow.
• Move: Lets you move the data flow from its current location to another project or folder.
• Delete: Lets you delete the data flow.