Copying Data from the Export Appliance

Learn about the steps for copying data from the Oracle-supplied transfer appliance to your environment.

Copy phase indicator for data export

This topic describes the tasks associated with copying data from the export appliance to your data center's Data Host using the Control Host. The Data Administrator role typically performs these tasks. See Roles and Responsibilities.

Note

You can only run Oracle Cloud Infrastructure CLI commands from a Linux host. This differs from running CLI commands for other Oracle Cloud Infrastructure Services on a variety of host operating systems. Appliance-based commands require validation that is only available on Linux hosts.

Information Prerequisites

Before performing any export appliance copying tasks, you must obtain the following information:

  • Appliance IP address - typically is provided by the Infrastructure Engineer.

  • IAM login information, Data Transfer Utility configuration files, export job ID, and job label - typically is provided by the Project Sponsor.

Setting Up an HTTP Proxy Environment

You might need to set up an HTTP proxy environment on the Control Host to allow access to the public internet. This proxy environment allows the Oracle Cloud Infrastructure CLI to communicate with the Data Transfer Appliance Management Service and the export appliance over a local network connection. If your environment requires internet-aware applications to use network proxies, configure the Control Host to use your environment's network proxies by setting the standard Linux environment variables on your Control Host.

Assume that your organization has a corporate internet proxy at http://www-proxy.myorg.com and that the proxy is an HTTP address at port 80. You would set the following environment variable:

export HTTPS_PROXY=http://www-proxy.myorg.com:80

If you configured a proxy on the Control Host and the export appliance is directly connected to that host, the Control Host tries unsuccessfully to communicate with the export appliance using a proxy. Set a no_proxy environment variable for the appliance. For example, if the appliance is on a local network at 10.0.0.1, you would set the following environment variable:

export NO_PROXY=10.0.0.1

Setting Firewall Access

If you have a restrictive firewall in the environment where you are using the Oracle Cloud Infrastructure CLI, you may need to open your firewall configuration to the following IP address ranges: 140.91.0.0/16.

Configuring the Export Appliance

After you have physically set up the export appliance and connected it to your network, you can configure it using the CLI to allow the copying of the data it contains to your data center.

Note

You can only use the CLI command to configure the appliance.

To configure the export appliance using the CLI:

oci dts export configure-physical-appliance --job-id job-id --appliance-cert-fingerprint appliance_cert_fingerprint --appliance-ip appliance_ip --rw [true|false] --world [true|false] [OPTIONS]

fingerprint is the export appliance X.509/SSL certificate fingerprint.

appliance_ip is the IP address of the export appliance.

rw indicates whether the exported data has read/write permissions. Specify true or false.

world indicates whether the exported data is accessible by all. Specify true or false.

OPTIONS are:

  • --appliance-port: The port number used by the export appliance.

  • --appliance-profile: User-defined name or description of the transfer appliance. This is useful if you have multiple transfer appliances.

  • --access-token: The access token to authenticate with the export appliance.

  • --appliance-port: The port number used by the export appliance.

  • --subnet-mask-length: The subnet mask length for the IP address.

Setting Your Data Host as an NFS Client

Note

Only Linux machines can be used as Data Hosts.

Set up your Data Host as an NFS client:

  • For Debian or Ubuntu, install the nfs-common package. For example:
    sudo apt-get install nfs-common
  • For Oracle Linux or Red Hat Linux, install the nfs-utils package. For example:
    sudo yum install nfs-utils

Mounting the NFS Share

To mount the NFS share:

At the command prompt on the Data Host, create the mountpoint directory:

mkdir -p /mnt/mountpoint

For example:

mkdir -p /mnt/nfs-ds-1

Next, use the mount command to mount the NFS share.

mount -t nfs appliance_ip:/data/dataset_name mountpoint

For example:

mount -t nfs 10.0.0.1:/data/nfs-ds-1 /mnt/nfs-ds-1
Note

The appliance IP address in this example (10.0.0.1) may be different that the one you use for your appliance.

After the NFS share is mounted, you can write data to the share.

Copying Files to the Data Host

Copy your file from the appliance to your NFS-mounted Data Host using normal file system tools.

Viewing Data Export Metrics

You can view the metrics associated with an export job in the appliance's Details page in chart or table format. Select Metrics under Resources to display each of these measures:

  • Export Files Downloaded: Total number of files downloaded to the appliance for export.

  • Export Bytes Downloaded: Total number of bytes downloaded to the appliance for export.

  • Export Files Remaining: Total number of files left to be downloaded for export.

  • Export Bytes Remaining: Total number of bytes left to be downloaded for export.

  • Export Files in Error: Total number of files in error for export.

  • Export Download Verification Progress: Progress of verification of files that have already been downloaded for export.

Select the Start Time and End Time for these measures, either by manually entering the days and times in their respective fields, or by selecting the Calendar feature and picking the times that way. As an alternative to selecting a start and end time, you can also select from a list of standard times (last hour, last 6 hours, and so forth) from the Quick Selects list for the period measured. The time period you specify applies to all the measures.

Specify the Interval (for example, 5 minutes, 1 hour) that each measure is recorded from the list.

Specify the Statistic being recorded (for example, Sum, Mean) for each measure from the list.

Tip

Mean is the most useful statistic for data export.

Choose additional actions from the Options list, including viewing the query in the Metrics Explorer, capturing the URL for the measure, and switching between chart and table view.

Click Reset Charts to delete any existing information in the charts and begin recording new metrics.

See Monitoring for general information on monitoring your Oracle Cloud Infrastructure services.

Erasing Files from the Export Appliance

Use standard system tools to remove your data from the export appliance after you have copied your files.

Notifying Export Appliance Return

When the erasure of the client data from the export appliance is complete, notify the Infrastructure Engineer that the appliance is ready to be returned to Oracle.