Considerations When Using Export and Import

Ensure that you understand what is supported and any current limitations before using export and import operations in Data Integration.

Note

During export, Data Integration clears sensitive information in data assets such as passwords. After the operation is complete, you must manually update the sensitive information fields of applicable data assets in the imported object.

Consider the following before exporting and importing objects in Data Integration.

  • No objects are deleted in export operations.
  • You can export and import an entire workspace, a whole project, or different object types separately such as a folder, task, data flow, or pipeline.

  • Data Integration includes referenced objects in an export request. For example, when you export a data flow, by default Data Integration includes referenced objects such data assets. In an import request, you have the option to exclude existing data assets.

  • In an import request, Data Integration uses the identifier values of source and target objects to match objects, and performs the import using the conflict resolution option that you specify. The conflict resolution options are:

    • Retain the conflicting objects
    • Replace the conflicting target objects
    • Duplicate the conflicting source objects
  • When importing projects, folders, tasks, data flows, pipelines, applications, and workspaces, you have the option to exclude importing data assets that already exist in the target. When you exclude existing data assets, Data Integration imports to the target only those source data assets that are not present in the target. Data assets that are already present in the target are not imported.

    Before you initiate an import request, if you create data assets in the target by using the same data asset identifiers as used by the source data assets, Data Integration automatically wires the target data assets to relevant target objects when matching target and source data assets are found during the import operation.

  • When a project has many folders, we recommend that the first export and import operations be the following:

    • Export the whole project (source)
    • Import the whole project (target)

    When a new folder is added in the original project (source), export the new folder and import that exported folder zip to the target.

  • When objects in a project's folder have references to objects in another folder, Data Integration exports and imports any dependent objects and their dependencies as well.

    Suppose project Project1 has three folders, Folder1, Folder2, and Folder3, with the following dependencies:

    • Objects in Folder1 reference objects in Folder2

    • Objects in Folder2 reference objects in Folder3

    If you export Folder1, Data Integration exports Folder1, Folder2, and Folder3. Folders Folder2 and Folder3 are included in the export operation because some or all objects in Folder2 and Folder3 are referenced by objects in Folder1. Similarly, if you export Folder2, Data Integration exports Folder2 and Folder3; if you export Folder3, only Folder3 gets exported.

    In the scenario with folder interdependencies, the options are:

    • Export the whole project first. As and when folders are created or updated, export and import folders to the target using the replace option. We recommend this option as the best practice.
    • Export the folders in the following order: Folder3, Folder2 with the replace option, Folder1 with the replace option
Note

Before exporting and importing, ensure that you have access to Object Storage and the relevant permissions and IAM policies, as described in Required Setup and Policies.