Oracle Cloud Infrastructure Documentation

Terraform: Set Up Resource Discovery

The OCI resource discovery feature creates Terraform files for existing resources in your compartment. Use it to:

  • Create state files for existing resources in the Console, and then add those resources to a Terraform setup.
  • Duplicate your existing infrastructure in a new tenancy or region.
  • Detect state drift for updated resources.

In this tutorial, you set up Oracle Cloud Infrastructure Terraform provider's resource discovery feature in your local environment. To confirm your setup, you run resource discovery to fetch information from your tenancy and create a script for it.

Key tasks include how to:

  • Create RSA keys.
  • Install Terraform OCI provider binaries.
  • Set up Terraform OCI provider API authentication variables.
  • Authenticate your OCI provider CLI commands.
  • Create a script in your environment, regarding the availability domains in your tenancy, through the resource discovery feature.
A diagram of a user connected from a local environment to an Oracle Cloud Infrastructure tenancy. The local environment is Linux and has Terraform OCI Provider binaries installed. There is an arrow from the local environment connected to Terraform Registry in the cloud. There is a second arrow from the local environment sending a message to the user's Oracle Cloud Infrastructure tenancy labeled Authenticate?. The third arrow is from the tenancy to the local environment labeled Fetch Data. These arrows suggest that the user has set up their OCI provider variables to be authenticated by their tenancy. The user can then fetch information from the tenancy, by using Terraform OCI provider commands and Terraform Registry. The tenancy displays three availability domains and that is the information that Resource Discovery is fetching to create Terraform scripts in the user's environment.

For more information, see Terraform OCI Resource Discovery Use Cases and Benefits.

Before You Begin

To successfully perform this tutorial, you must have the following:

MacOS or Linux
  • MacOS
  • Linux (Any distribution)
  • Oracle Cloud Infrastructure Cloud Shell:
    • Cloud Shell
      Note

      If you're using Oracle Cloud Infrastructure Cloud Shell, the OCI Terraform Provider is already installed and you don't need to create RSA keys. Go to section 1. Prepare and proceed to Add List Policy.
Windows 10

1. Prepare

Prepare your environment for authenticating and running resource discovery commands. Also, gather the information your account needs to authenticate your commands.

Create RSA Keys

You create RSA keys for API signing in to your Oracle Cloud Infrastructure account.

Note

If you're using Cloud Shell, skip creating the RSA keys. You're already authenticated when you log in to the OCI Console.
  1. Open a terminal window.
  2. Under your home directory, make an .oci directory. 
    mkdir <your-home-directory>/.oci

    Example for Oracle Linux:

    mkdir /home/opc/.oci
    Note

    If you're using Windows Subsystem for Linux (WSL), create the /.oci directory directly in the Linux environment. If you create the /.oci directory in a /mnt folder (Windows file system), you're required to use the chmod command to change permissions for the WSL configuration files.
  3. Generate a 2048-bit private key in a PEM format: 
    openssl genrsa -out <your-home-directory>/.oci/<your-rsa-key-name>.pem 2048
  4. Change permissions, so only you can read and write to the private key file: 
    chmod 600 <your-home-directory>/.oci/<your-rsa-key-name>.pem
  5. Generate the public key: 
    openssl rsa -pubout -in <your-home-directory>/.oci/<your-rsa-key-name>.pem -out $HOME/.oci/<your-rsa-key-name>_public.pem
  6. Copy the public key.
    In the terminal, enter:
    cat <your-home-directory>/.oci/<your-rsa-key-name>_public.pem
  7. Add the public key to your user account.
    • In the OCI Console's top navigation bar, click the Profile menu, and then go to User settings.
    • Click API Keys.
    • Click Add API Key.
    • Select Paste Public Keys.
    • Paste value from previous step, including the lines with BEGIN PUBLIC KEY and END PUBLIC KEY.
    • Click Add.

You have now set up the RSA keys to connect to your OCI account.

Reference

Generating RSA Keys

Add List Policy

If your username is in the Administrators group, then skip this section. Otherwise, ask your administrator to add the following policy to your tenancy:

allow group <the-group-your-username-belongs> to read all-resources in tenancy

With this privilege, you can list all the resources in your tenancy.

Steps to Add the Policy
  1. In the top navigation bar, open the Profile menu.
  2. Click your username.
  3. In the left pane, click Groups.
  4. In a notepad, copy the Group Name that your username belongs.
  5. Open the navigation menu and click Identity & Security. Under Identity, click Policies.
  6. Select <your-tenancy>(root) from the Compartment dropdown.
  7. Click Create Policy.
  8. Fill in the following information:
    • Name: list-resources
    • Description: Allow the group <the-group-your-username-belongs> to list the resources in this tenancy.
    • Compartment: <your-tenancy>(root)
  9. For Policy Builder, click Show manual editor.
  10. Paste in the following policy:
    allow group <the-group-your-username-belongs> to read all-resources in tenancy
  11. Click Create.

Reference: Common Policies

Gather Required Information

Prepare the information you need to authenticate your Terraform OCI Provider commands and copy them into your notepad.

Note

If you're using Cloud Shell, you only need to find the <tenancy-ocid> from the following step.
  1. Collect the following credential information from the Oracle Cloud Infrastructure Console.
    • Tenancy OCID: <tenancy-ocid>
      • Click your Profile menu (your avatar), then click Tenancy:<tenancy-name>, and copy OCID.
    • User OCID: <user-ocid>
      • Click your Profile menu (your avatar), then click User Settings, and copy OCID.
    • Fingerprint: <fingerprint>
      • From your Profile menu (your avatar), click User Settings and then click API Keys.
      • Copy the fingerprint associated with the RSA public key you made in the Create RSA Keys section. The format is: xx:xx:xx...xx.
    • Region: <region-identifier>
  2. Collect the following information from your environment.
    • Private Key Path: <rsa-private-key-path>
      • Path to the RSA private key you made in the Create RSA Keys section. Example: $HOME/.oci/<your-rsa-key-name>.pem.
Install Terraform OCI Provider
Note

If you're using Cloud Shell, skip this section.

Install the latest version of Terraform OCI Provider binaries v4.2+:

  1. From a browser, go to Terraform OCI Provider Releases.
  2. Click the latest link.
  3. Find the link for your environment and then copy the link address. Example for Linux 64 bit: 
    https://releases.hashicorp.com/terraform-provider-oci/4.59.0/terraform-provider-oci_4.59.0_linux_amd64.zip
  4. In your environment, create a temp directory and change to that directory: 
    mkdir temp
    cd temp
  5. Download the Terraform zip file. Example: 
    wget https://releases.hashicorp.com/terraform-provider-oci/4.59.0/terraform-provider-oci_4.59.0_linux_amd64.zip
  6. Unzip the file. Example: 
    unzip terraform-provider-oci_4.59.0_linux_amd64.zip
  7. Move the unzipped folder to /usr/local/bin. Example: 
    sudo mv terraform-provider-oci_v4.59.0_x4 /usr/local/bin
    Note

    Use sudo for the /usr/local/bin directory if it doesn't let you run the commands.
  8. Use a symbolic link to shorten the terraform-provider-oci_<version> command: 
    sudo ln -s <source-executable> <shorter-alias>
    cd /usr/local/bin/
    sudo ln -s terraform-provider-oci_<version> tf-oci

    For this tutorial, use tf-oci for the <shorter-alias>, instead of the <source-executable>.

  9. Go back to your home directory: 
    cd
  10. Check the Terraform OCI provider version: 
    tf-oci

    Example: [INFO] terraform-provider-oci 4.59.0.

    Note

    On MacOS, you need to create a security exception for the executable.

2. Create an Authentication Script

Create a shell script to assign authentication information to OCI provider authentication variables. Your Oracle Cloud Infrastructure account authenticates your OCI provider commands through the values assigned to these parameters.

Add Authentication
Note

If you're using Cloud Shell, you don't need to add authentication. Proceed to section 3. Discover a Resource.
  1. Create an executable script file, somewhere in your PATH, and name it provider-oci.sh. For example, if ~/bin is in your PATH, the steps are as follows: 
    cd ~/bin
    touch provider-oci.sh
    chmod 755 provider-oci.sh
    ls -l

    You get something like this:

    -rwxr-xr-x 1 <your-username> <your-username> 0 <date> provider-oci.sh
  2. Add the following code to provider-oci.sh.

    Replace the fields with brackets, with the information you gathered in Gather Required Information section.

    #!/bin/bash
export TF_VAR_tenancy_ocid="<tenancy-ocid>"
export TF_VAR_user_ocid="<user-ocid>"
export TF_VAR_fingerprint="<fingerprint>"
export TF_VAR_private_key_path="<rsa-private-key-path>"
export TF_VAR_region="<region-identifier>"
  3. Save the provider-oci.sh file.
Explanation
  • Go to Terraform Resource Discovery.
  • Go to the Authentication section.
  • Set up the following environment variables for API Key based authentication:
    • TF_VAR_tenancy_ocid
    • TF_VAR_user_ocid
    • TF_VAR_fingerprint
    • TF_VAR_private_key_path
    • TF_VAR_region
Export Environment Variables

In this section, you run the provider-oci.sh file to add the OCI provider authentication variables to your environment variables.

  1. From your $HOME/bin directory, run the provider-oci.sh file. 
    source provider-oci.sh
  2. Confirm that the OCI provider authentication variables are added to your environment variables. 
    env | sort

    The variables are displayed in alphabetical order. Example:

    ...
TF_VAR_fingerprint=<fingerprint>
TF_VAR_private_key_path=<rsa-private-key-path>
TF_VAR_region=<region-identifier>
TF_VAR_tenancy_ocid=<tenancy-ocid>
TF_VAR_user_ocid=<user-ocid>
...
  3. To load the environment variables, when a new shell starts, append the source ~/bin/provider-oci.sh command to .bashrc. 
    source ~/bin/provider-oci.sh

3. Discover a Resource

Discover the availability domains in your tenancy.

Run Resource Discovery

Every tenancy has a list of availability domains. By discovering the availability domains in your region, you confirm that:

  • Your Oracle Cloud Infrastructure account can authenticate your Oracle Cloud Infrastructure provider commands.
  • You can get information from your account with the resource discovery feature.
  1. In your $HOME directory, create a directory called resource-discovery.
    cd
    mkdir resource-discovery
  2. If you're using Cloud Shell, find the installed OCI Terraform Provider file in the /usr/bin directory.
    ls /usr/bin

    The file name is:

    terraform-provider-oci_<version>

    Sample file name:

    terraform-provider-oci_v5.2.1
  3. Run the following command:
    • In Cloud Shell:
      terraform-provider-oci_<version> -command=export -compartment_id=<tenancy-ocid> -services=availability_domain -output_path=$HOME/resource-discovery
    • On your compute instance or in your local environment:
      tf-oci -command=export -compartment_id=<tenancy-ocid> -services=availability_domain -output_path=$HOME/resource-discovery
    Important

    • Replace the <tenancy-ocid>, with the information you gathered in the Gather Required Information section.
    • If you're using Cloud Shell, replace terraform-provider-oci_<version> with the file name that you found in step 2.

    Sample output:

    ...
INFO <date> [INFO] ===> Generating resource 'oci_identity_availability_domain.export_QnsC-US-ASHBURN-AD-1'
INFO <date> [INFO] ===> Generating resource 'oci_identity_availability_domain.export_QnsC-US-ASHBURN-AD-2'
INFO <date> [INFO] ===> Generating resource 'oci_identity_availability_domain.export_QnsC-US-ASHBURN-AD-3'
...
INFO <date> Found 6 'availability_domain' resources. Generated under '/<home-directory>/resource-discovery/availability_domain.tf'.
INFO <date> === COMPLETED ===
    Note

    • The tenancy OCID is the compartment OCID for the root compartment. Providing a specific <compartment-ocid> or your <tenancy-ocid> outputs the same availability domains.
    • To discover identity resources, you do not need to mention a compartment OCID. In the previous example, you get the same result if you remove the compartment_id from the command. The compartment_id is there for you to learn the syntax for other services.
Troubleshooting
  • error message: Failed to query available provider packages:
    • If you're on a VPN, check your proxy settings.
  • 401 errors - (Service error:NotAuthenticated):
    • You have an incorrect value for one of the following:
      • tenancy OCID
      • user OCID
      • fingerprint
      • RSA private key (the path or the key)
  • no such host:
    • You have an incorrect value for the following:
      • region identifier
Explanation
  • Go to Terraform Resource Discovery.
  • Go to the Usage section.
  • From the two command options in the Usage section, use the one with the compartment_id as a base command:
    terraform-provider-oci -command=export -compartment_id= -output_path=
  • Update the base command:
    • Instead of terraform-provider-oci, use your symbolic link command:

      tf-oci

    • Review the Parameter Description section to add proper values to the parameters in the base command:
      • Use the export command to perform resource discovery:

        -command=export

      • Add the OCID of the compartment that you're discovering its resources:

        -compartment_id=<tenancy-ocid>

      • Create and then specify a directory for the discovered resources:

        Example: -output_path=$HOME/resource-discovery

  • Search for the phrase availability_domain. You get:

    Resources that are dependent on availability domains will be generated under availability_domain.tf file. These include:

    • oci_core_boot_volume
    • oci_file_storage_file_system
    • oci_file_storage_mount_target
    • oci_file_storage_snapshot
  • For services, use:
    • availability_domain (Even though it is not a service, it works with service.)
    • Example: -services=availability_domain
    • You don't need quotations around the service names.
  • Construct the command:

    Example:

    tf-oci -command=export -compartment_id=<tenancy-ocid> -services=availability_domain -output_path=$HOME/resource-discovery
    Note

    The resource discovery command does not create a directory for the discovered resources. Create a directory and specify the path in your command.
Review Discovered Resources
  1. From your $HOME directory, change to the resource-discovery directory. 
    cd resource-discovery
  2. View the contents of the directory. 
    ls

    Sample output:

    availability_domain.tf  provider.tf  vars.tf
  3. Review the Terraform script for availability domains: 
    cat availability_domain.tf

    Sample output:

    ## This configuration was generated by terraform-provider-oci

data oci_identity_availability_domain export_QnsC-US-ASHBURN-AD-1 {
  compartment_id = var.compartment_ocid
  ad_number      = "1"
}
data oci_identity_availability_domain export_QnsC-US-ASHBURN-AD-2 {
  compartment_id = var.compartment_ocid
  ad_number      = "2"
}
data oci_identity_availability_domain export_QnsC-US-ASHBURN-AD-3 {
  compartment_id = var.compartment_ocid
  ad_number      = "3"
}
resource oci_file_storage_file_system export_JennFS ... <some info>
resource oci_file_storage_snapshot export_Snapshot-1 ... <some info>
resource oci_file_storage_mount_target export_MountTarget-xxx ... <some info>
    Note

    The availability_domain.tf Terraform script is using a variable called compartment_ocid. Find the value of this variable, in vars.tf.
  4. Review the variables: 
    cat vars.tf

    Sample output:

    variable compartment_ocid { default = "<tenancy-ocid>" }
variable region { default = "us-ashburn-1" }
  5. Review the provider script information: 
    cat provider.tf

    Sample output:

    provider oci {
        region = var.region
}
    Note

    • The provider.tf denotes that you are using the Terraform OCI provider with provider oci { }.
    • The provider.tf file does not include your authentication information, because you provide authentication information through your environment variables.

Congratulations! Your Oracle Cloud Infrastructure account can now authenticate your Terraform OCI provider commands. And your environment is ready to run the resource discovery commands.

References: