Preparing for Appliance Data Transfers

Minimum Required CLI Version

The minimum CLI version required for Appliance-Based Data Import is 2.12.1.

Determining CLI Versions

Access the following URL to see the currently available version of the CLI:

https://github.com/oracle/oci-cli/blob/master/CHANGELOG.rst

Enter the following command at the prompt to see the version of the CLI currently installed on your machine:

oci --version

If you have a version on your machine older than the version currently available, install the latest version.

Linux Operating System Requirements

See Requirements for a list of the Linux operating systems that support the CLI.

Installing the CLI

Installation and configuration of the CLIs is described in detail in Command Line Interface (CLI).

Using the CLI

You can specify CLI options using the following commands:

• --option value or

• --option=value

The basic CLI syntax is:

oci dts resource action options

This syntax is applied to the following:

• oci dts is the shortened CLI command name.

• job is an example of a resource.

• create is an example of an action.

• Other strings are options.

The following command to create a transfer job shows a typical CLI command construct.

oci dts job create --compartment-id ocid1.compartment.oc1..exampleuniqueID --bucket MyBucket --display-name MyApplianceImportJob --device-type appliance

Accessing Command Line Interface Help

oci dts job --help
			
NAME
  dts_job -

DESCRIPTION
  Transfer disk or appliance job operations

AVAILABLE COMMANDS
  o change-compartment
  o close
  o create
  o delete
  o detach-devices-details
  ...

oci dts job create --help
			
NAME
  dts_job_create -

DESCRIPTION
  Creates a new transfer disk or appliance job.

USAGE
  oci dts job create [OPTIONS]

REQUIRED PARAMETERS
  --bucket [text]

Upload bucket name

--compartment-id, -c [text]

Compartment OCID

--device-type [text]

Creating the Data Transfer Directory

Create a Oracle Cloud Infrastructure directory (.oci) on the same Control Host machine where the Oracle Cloud Infrastructure CLI is installed. For example:

mkdir /root/.oci/

The two configuration files (config and config_upload_user) are placed in what ever location you choose.

Creating the Data Transfer Administrator Configuration File

The data transfer administrator configuration file contains the required credentials for working with Oracle Cloud Infrastructure. You can create this file using a setup CLI or manually using a text editor.

Using the Setup CLI

Run the oci setup config command line utility to walk through the first-time setup process. The command prompts you for the information required for the configuration file and the API public/private keys. The setup dialog generates an API key pair and creates the configuration file.

For more information about how to find the required information, see:

• Where to Get the Tenancy's OCID and User's OCID

• Regions and Availability Domains

Manual Setup

If you want to set up the API public/private keys yourself and write your own configuration file, see SDK and CLI Configuration File.

Create the data transfer administrator configuration file /root/.oci/config with the following structure:

[DEFAULT]
user=<The OCID for the data transfer administrator>
fingerprint=<The fingerprint of the above user's public key>
key_file=<The _absolute_ path to the above user's private key file on the host machine>
tenancy=<The OCID for the tenancy that owns the data transfer job and bucket>
region=<The region where the transfer job and bucket should exist. Valid values are: 
supported regions>.

where supported regions are the regions listed in Data Transfer Supported Regions.

For example:

[DEFAULT]
user=ocid1.user.oc1..unique_ID
fingerprint=4c:1a:6f:a1:5b:9e:58:45:f7:53:43:1f:51:0f:d8:45
key_file=/home/user/ocid1.user.oc1..exampleuniqueID.pem
tenancy=ocid1.tenancy.oc1..unique_ID
region=us-phoenix-1

For the data transfer administrator, you can create a single configuration file that contains different profile sections with the credentials for multiple users. Then use the --profile option to specify which profile to use in the command. Here is an example of a data transfer administrator configuration file with different profile sections:

[DEFAULT]
user=ocid1.user.oc1..exampleuniqueID
fingerprint=4c:1a:6f:a1:5b:9e:58:45:f7:53:43:1f:51:0f:d8:45
key_file=/home/user/ocid1.user.oc1..exampleuniqueID.pem
tenancy=ocid1.tenancy.oc1..exampleuniqueID
region=us-phoenix-1
[PROFILE1]
user=ocid1.user.oc1..exampleuniqueID
fingerprint=4c:1a:6f:a1:5b:9e:58:45:f7:53:43:1f:51:0f:d8:45
key_file=/home/user/ocid1.user.oc1..exampleuniqueID.pem
tenancy=ocid1.tenancy.oc1..exampleuniqueID
region=us-ashburn-1

By default, the DEFAULT profile is used for all CLI commands. For example:

oci dts job create --compartment-id ocid.compartment.oc1..exampleuniqueID --bucket MyBucket --display-name MyDisplay --device-type appliance

Instead, you can issue any CLI command with the --profile option to specify a different data transfer administrator profile. For example:

oci dts job create --compartment-id ocid.compartment.oc1..exampleuniqueID --bucket MyBucket --display-name MyDisplay --device-type appliance --profile MyProfile

Using the example configuration file above, the profile_name would be profile1.

If you created two separate configuration files, use the following command to specify the configuration file to use:

oci dts job create --compartment-id compartment_id --bucket bucket_name --display-name display_name --config 
                
                

Creating the Data Transfer Upload User Configuration File

The config_upload_user configuration file is for the data transfer upload user, the temporary IAM user that Oracle uses to upload your data on your behalf. Create this configuration file with the following structure:

[DEFAULT]
user=<The OCID for the data transfer upload user>
fingerprint=<The fingerprint of the above user's public key>
key_file=<The _absolute_ path to the above user's private key file on the host machine>
tenancy=<The OCID for the tenancy that owns the data transfer job and bucket>
region=<The region where the transfer job and bucket should exist. Valid values are: 
supported regions>.

where supported regions are the regions listed in Data Transfer Supported Regions.

Configuration File Entries

The following table lists the basic entries that are required for each configuration file and where to get the information for each entry.

You can verify the data transfer upload user credentials using the following command:

oci dts job verify-upload-user-credentials --bucket bucket_name

Setting Up Notifications for a New Import Appliance Request

To include job notifications when requesting an import appliance, include the --setup-notifications option as part of the oci dts appliance request command:

oci dts appliance request --job-id ... --setup-notifications

Setting up Notifications for an Existing Import Appliance Request

To set up notifications for an existing import appliance request, run the oci dts appliance setup-notifications command on the appliance:

oci dts appliance setup-notifications --appliance-label appliance_label

Failure Scenarios

This section describes possible failure scenarios detected during validation:

• Connectivity to Data Transfer service

  • Timed out request to Data Transfer endpoint

    Checking Data Transfer Service connectivity... datatransfer.r1.oracleiaas.com is not reachable.
    Please ensure there there are no firewall rules blocking connectivity.
    RequestException:
    {
        "client_version": "Oracle-PythonCLI/3.22.0",
        "logging_tips": "Please run the OCI CLI command using --debug flag to find more debug information.",
        "message": "The connection to endpoint timed out.",
        "request_endpoint": null,
        "target_service": "CLI",
        "timestamp": "2023-02-01T11:46:02.374857",
        "troubleshooting_tips": " See [https://docs.oracle.com/en-us/iaas/Content/API/Concepts/sdk_troubleshooting.htm] for more information about resolving this error. If you are unable to resolve this issue, run this CLI command with --debug option and contact Oracle support and provide them the full error message."
    }

• Connectivity to Object Storage

  • Timed out request to the Object Storage endpoint

    Checking Object Storage connectivity... objectstorage.r1.oracleiaas.com is not reachable.
    Please ensure there there are no firewall rules blocking connectivity.
    RequestException:
    {
        "client_version": "Oracle-PythonCLI/3.22.0",
        "logging_tips": "Please run the OCI CLI command using --debug flag to find more debug information.",
        "message": "The connection to endpoint timed out while trying to reach https://objectstorage.r1.oracleiaas.com",
        "request_endpoint": " GET https://objectstorage.r1.oracleiaas.com",
        "target_service": "CLI",
        "timestamp": "2023-02-01T12:52:33.979490",
        "troubleshooting_tips": "Try running curl https://objectstorage.r1.oracleiaas.com. If the curl doesn't work, check your network setting or contact your network administrator. See [https://docs.oracle.com/en-us/iaas/Content/API/Concepts/sdk_troubleshooting.htm] for more information about resolving this error. If you are unable to resolve this issue, run this CLI command with --debug option and contact Oracle support and provide them the full error message."
    }

• Required IAM Users, Groups, and Policies for admin user

  • No Object Storage authorization policy for admin user

    Checking Required IAM Users, Groups, and Policies... Fail
    Unauthorized access to Object Storage.
    Please ensure you have the required IAM Users, Groups, and Policies to access Object Storage.

  • No Data Transfer authorization policy for admin user

    Checking Required IAM Users, Groups, and Policies... Fail
    Unauthorized access to Data Transfer Service.
    Please ensure you have the required IAM Users, Groups, and Policies to access Data Transfer Service.

• Upload bucket belongs to compartment

  • Upload bucket does not exist

    Checking Upload Bucket exists in Compartment... Fail
    Bucket not found.
    The bucket 'e2e_test_hdx' does not exist in namespace 'bdtstest1' or you are unauthorized to access it.

  • Upload bucket not in compartment

    
    Checking Upload Bucket exists in Compartment... Fail
    The bucket 'e2e_test_hdd' does not belong to compartment 'ocid1.compartment.region1..aaaaaaaaph2r5w24faket2vsp22hiboxjzk4k3s47d2ut37jghjagfb3abca'.

• Data Transfer upload users can create, overwrite, inspect and delete objects from bucket in Object Storage

  • No authorization policy for the upload user

    Checking Upload User credentials...
    Failed to Create object BulkDataTransferTestObject in bucket e2e_test_hdd using upload user in tenancy bdtstest1 as upload user
    Fail
    ServiceError:
    {
        "client_version": "Oracle-PythonSDK/2.90.0+2664, Oracle-PythonCLI/3.22.0",
        "code": "BucketNotFound",
        "logging_tips": "Please run the OCI CLI command using --debug flag to find more debug information.",
        "message": "Either the bucket named 'e2e_test_hdd' does not exist in the namespace 'bdtstest1' or you are not authorized to access it",
        "opc-request-id": "sea-1:tpz7tEO-SLezjoTd506L8R5wm3HewQNHFfpg-EKsAINe6P8fFj4b_znPyfujFeqq",
        "operation_name": "put_object",
        "request_endpoint": "PUT https://objectstorage.r1.oracleiaas.com/n/bdtstest1/b/e2e_test_hdd/o/BulkDataTransferTestObject",
        "status": 404,
        "target_service": "object_storage",
        "timestamp": "2023-02-07T06:39:24.816918+00:00",
        "troubleshooting_tips": "See [https://docs.oracle.com/iaas/Content/API/References/apierrors.htm] for more information about resolving this error. If you are unable to resolve this issue, run this CLI command with --debug option and contact Oracle support and provide them the full error message."
    }

• Approved Data Transfer appliance entitlement

  • No Data Transfer service entitlement

    Checking Data Transfer Appliance Entitlement... Fail
    Unable to find Data Transfer Appliance Entitlement in Compartment 'ocid1.compartment.region1..aaaaaaaaa5mxm3fd6tanvthad5oplmdcmxjqdsoehidua7iz3sxauih3deja'.

  • Unapproved Data Transfer entitlement

    Checking Data Transfer Appliance Entitlement... Fail
    Data Transfer Appliance Entitlement needs to be in an APPROVED lifecycle state.

• Data Transfer import appliance-based transfer job exists

  • No Data Transfer appliance-based transfer job

    Checking Transfer Job and Appliance Request... Fail
    There is no Appliance Transfer Job 'ocid1.datatransferjob.oc1..exampleuniqueID' for Data Import into Bucket 'e2e_test_hdd'.

• Appliance request associated with a valid transfer job

  • No appliance request

    Checking Transfer Job and Appliance Request... Fail
    There is no Transfer Appliance Request for Appliance Transfer Job 'ocid1.datatransferjob.oc1..exampleuniqueID'.