Oracle Cloud Infrastructure Documentation

Terraform: Create Scripts and State Files with Resource Discovery

The OCI resource discovery feature creates Terraform files for existing resources in your compartment. Use it to:

  • Create state files for existing resources in the Console, and then add those resources to a Terraform setup.
  • Duplicate your existing infrastructure in a new tenancy or region.
  • Detect state drift for updated resources.

In this tutorial, you create scripts and state files for resources in your account through the resource discovery feature. Then you use Terraform to manage the resources.

Key tasks include how to:

  • Create a resource through the Console.
  • Create scripts and a state file for the resource through the resource discovery feature.
  • Update the resource with Terraform.
  • Confirm that the resource has been updated in the Console.
A diagram of a user connected from a local environment to an Oracle Cloud Infrastructure tenancy. The local environment is Linux and has Terraform OCI Provider binaries installed. There is an arrow from the local environment connected to Terraform Registry in the cloud. There is a second arrow from the local environment sending a message to the user's Oracle Cloud Infrastructure tenancy labeled Authenticate?. The third arrow is from the tenancy to the local environment labeled Fetch Data, Create Scripts, and Create a State File. These arrows suggest that the user has set up their OCI provider variables to be authenticated by their tenancy. The user can then fetch information from the tenancy, and create scripts and state files, by using Terraform OCI provider commands and Terraform Registry. The tenancy contains a bucket with the name rd-bucket, in a compartment called rd-compartment. Resource Discovery is fetching information about this bucket, to create Terraform scripts and a state file in the user's environment.

For additional information, see:

Before You Begin

To successfully perform this tutorial, you must have the following:

Requirements
  • An Oracle Cloud Infrastructure account. See Signing Up for Oracle Cloud Infrastructure.
  • A MacOS, Linux, or Windows computer.
  • Terraform tutorial resources:
    • Go through all the steps in Terraform Tutorial: Set Up Resource Discovery.
    • Make a note of the OCI Terraform provider command:
      • Cloud Shell: terraform-provider-oci_<version> from /usr/bin
      • Compute instance or a local environment: Use the <shorter-alias> that you created in the previous tutorial to use instead of the command, terraform-provider-oci_<version>.
        • Example: tf-oci
  • Terraform v1.1.3+:
    • If you're using Cloud Shell, you don't need to install Terraform. Terraform is already installed.
    • If you're using a compute instance or a local environment, then in the Terraform Tutorial: Set Up OCI Terraform, go to the Prepare section and follow the Install Terraform steps.

1. Create a Resource

Create a bucket in your tenancy through the Console. This bucket does not have a Terraform script. Later, use resource discovery to create a Terraform script and a state file for the bucket.

Add Compartment Policy

If your username is in the Administrators group, then skip this section. Otherwise, have your administrator add the following policy to your tenancy:

allow group <the-group-your-username-belongs> to manage compartments in tenancy

With this privilege, you can create a compartment for all the resources in your tutorial.

Steps to Add the Policy
  1. In the top navigation bar, open the Profile menu.
  2. Click your username.
  3. In the left pane, click Groups.
  4. In a notepad, copy the Group Name that your username belongs.
  5. Open the navigation menu and click Identity & Security. Under Identity, click Policies.
  6. Select your compartment from the Compartment drop-down.
  7. Click Create Policy.
  8. Fill in the following information:
    • Name: manage-compartments
    • Description: Allow the group <the-group-your-username-belongs> to list, create, update, delete and recover compartments in the tenancy.
    • Compartment: <your-tenancy>(root)
  9. For Policy Builder, click Show manual editor.
  10. Paste in the following policy:
    allow group <the-group-your-username-belongs> to manage compartments in tenancy
  11. Click Create.

Reference: The compartments resource-type in Verbs + Resource-Type Combinations for IAM

Create a Compartment

Create a compartment for the resources that you create in this tutorial.

  1. Sign in to the Oracle Cloud Infrastructure Console.
  2. Open the navigation menu and click Identity & Security. Under Identity, click Compartments.
  3. Click Create Compartment.
  4. Fill in the following information:
    • Name: <your-compartment-name>
    • Description: Compartment for <your-description>.
    • Parent Compartment: <your-tenancy>(root)
  5. Click Create Compartment.

Reference: Create a compartment

Add Resource Policy

If your username is in the Administrators group, then skip this section. Otherwise, have your administrator add the following policy to your tenancy:

allow group <the-group-your-username-belongs> to manage all-resources in compartment <your-compartment-name>

With this privilege, you can manage all resources in your compartment, essentially giving you administrative rights in that compartment.

Steps to Add the Policy
  1. Open the navigation menu and click Identity & Security. Under Identity, click Policies.
  2. Select your compartment from the Compartment drop-down.
  3. Click Create Policy.
  4. Fill in the following information:
    • Name: manage-<your-compartment-name>-resources
    • Description: Allow users to list, create, update, and delete resources in <your-compartment-name>.
    • Compartment: <your-tenancy>(root)
  5. For Policy Builder, select the following choices:
    • Policy use cases: Compartment Management
    • Common policy templates: Let compartment admins manage the compartment
    • Groups: <the-group-your-username-belongs>
    • Location: <your-tenancy>(root)
  6. Click Create.

Reference: Common Policies

Create a Bucket

Create a bucket in your compartment.

  1. Open the navigation menu and click Storage. Under Object Storage & Archive Storage, click Buckets.
  2. In the left panel, for Compartment, select <your-compartment-name>.
  3. Click Create Bucket.
  4. Fill in the following information:
    • Bucket Name: <your-bucket-name>
    • Keep the rest of the defaults.
  5. Click Create.
You have successfully created a private bucket.
Note

Ensure that you create your bucket in <your-compartment-name>.

2. Discover the Resource

Discover the bucket that you created in your compartment.

Discover the Bucket
  1. From your $HOME directory, delete and re-create the resource-discovery directory you created for the Set Up Resource Discovery tutorial. 
    cd
    rm -R resource-discovery
    mkdir resource-discovery
  2. Run the following command: 
    tf-oci -command=export -compartment_name=<your-compartment-name> -services=object_storage -output_path=$HOME/resource-discovery
    Cloud Shell:
    terraform-provider-oci_<version> -command=export -compartment_name=<your-compartment-name> -services=object_storage -output_path=$HOME/resource-discovery
    • Replace <your-compartment-name>, with the name of your compartment.
    Sample output:
    ...
INFO <date> [INFO] ===> Generating resource 'oci_objectstorage_namespace.export_namespace'
INFO <date> [INFO] ===> Generating resource 'oci_objectstorage_bucket.export_<your-bucket-name>'
INFO <date> [INFO] Optional TF attribute 'kms_key_id' not found in source
INFO <date> Found 2 'object_storage' resources. Generated under '/<home-directory>/resource-discovery/object_storage.tf'.
INFO <date> === COMPLETED ===
    Note

    You might get the following message:
    <date> ERROR IN GET: Service error: LifecyclePolicy NotFound. The bucket '<your-bucket-name>' does not define a lifecycle policy.. http status code: 404. Opc request id: iad-1:xxx

    Lacking a lifecycle policy is OK for this tutorial. To plan for the service to automatically archive or delete this bucket, see Lifecycle Policies for Object Storage.

  3. Review the generated Terraform scripts. 
    cd resource-discovery
    ls
    • object_storage.tf
    • provider.tf
    • vars.tf
  4. View the object_storage.tf script. 
    cat object_storage.tf
    ## This configuration was generated by terraform-provider-oci

data oci_objectstorage_namespace export_namespace {
  compartment_id = var.compartment_ocid
}
resource oci_objectstorage_bucket export_<your-bucket-name> {
  access_type    = "NoPublicAccess"
  auto_tiering   = "Disabled"
  compartment_id = var.compartment_ocid
  defined_tags = {
  }
  freeform_tags = {
  }
  #kms_key_id = <<Optional value not found in discovery>>
  metadata = {
  }
  name                  = "<your-bucket-name>"
  namespace             = data.oci_objectstorage_namespace.export_namespace.namespace
  object_events_enabled = "false"
  storage_tier          = "Standard"
  versioning            = "Disabled"
}
You have successfully discovered object storage resources in your compartment.
Explanation
  • Go to Terraform Resource Discovery.
  • Go to the Usage section.
  • From the two command options in the Usage section, use the one with the compartment_name as a base command:
    terraform-provider-oci -command=export -compartment_name=<xxx> -output_path=<xxx>
  • Update the base command:
    • Instead of terraform-provider-oci, use your symbolic link command:

      tf-oci

    • Review the Parameter Description section to add proper values to the parameters in the base command:
      • Use the export command to perform resource discovery:

        -command=export

      • Enter the name of the compartment that you are discovering its resources:

        -compartment_name=<your-compartment-name>

      • Create and then specify a directory for the discovered resources:

        Example: -output_path=$HOME/resource-discovery

  • Add the following parameter from the Parameter Description section to the base command:
    • services
  • From the service names listed for services, use:
    • object_storage
    • Example: -services=object_storage
    • To discover more than one service, separate the service names with a comma.
    • Example: -services=object_storage,core
    • You don't need quotations around the service names.
  • Construct the command:

    Example:

    tf-oci -command=export -compartment_name=<your-compartment-name> -services=object_storage -output_path=$HOME/resource-discovery
    Note

    Troubleshooting
    [ERROR] output_path does not exist: stat /$HOME/resource-discovery: no such file or directory

    The resource discovery command does not create a directory for the discovered resources. Create a directory and specify the path in your command.

    [ERROR] no output directory specified

    The parser stops at the parameter that lacks a dash. Ensure that you add a dash to the beginning of each parameter. For example, if you use services instead of -services, the parser does not reach the output directory.

Generate a State File

To track your resources, Terraform stores the state of your resources in a state file. Then, every time you update a resource with Terraform, Terraform updates the state file with that change.

  1. Run the resource discovery command with the -generate_state flag:
    • In Cloud Shell:
    • terraform-provider-oci_<version> -command=export -compartment_id=<tenancy-ocid> -command=export -compartment_name=<your-compartment-name> -services=object_storage -generate_state -output_path=$HOME/resource-discovery
    • On your compute instance or in your local environment:
      tf-oci -command=export -compartment_name=<your-compartment-name> -services=object_storage -generate_state -output_path=$HOME/resource-discovery
    Sample output:
    ...
Terraform has been successfully initialized!
...
Import successful!

The resources that were imported are shown above. These resources are now in
your Terraform state and will henceforth be managed by Terraform.

INFO <date> [INFO] ===> Generating resource 'oci_objectstorage_namespace.export_namespace'
INFO <date> [INFO] ===> Generating resource 'oci_objectstorage_bucket.export_<your-bucket-name>'
INFO <date> [INFO] Optional TF attribute 'kms_key_id' not found in source
INFO <date> Found 2 'object_storage' resources. Generated under '/<home-directory>/resource-discovery/object_storage.tf'.
INFO <date> === COMPLETED ===
    Note

    Troubleshooting:
    • Error: Failed to query available provider packages:
      • If you are on a VPN, check your proxy settings.
  2. View the contents of the resource-discovery directory. 
    ls
    • object_storage.tf
    • provider.tf
    • terraform.tfstate
    • vars.tf
    Note

    The resource discovery command overwrites the <resource>.tf files every time you run it. If you don't specify a service, it creates a <resource>.tf file for every resource in the specified compartment.
  3. View the generated Terraform state file. 
    cat terraform.tfstate

    Sample output:

    ...
"instances": [
  {
      "bucket_id": "ocid1.bucket.xxx",
      "compartment_id": "ocid1.compartment.xxx",
      "created_by": "ocid1.user.xxx",
      "id": "xxx/<your-bucket-name>",
...

Congratulations! You have created a state file for your bucket resource.

References:

3. Update the Resource

Update the name of your bucket in the object_storage.tf file and then run your Terraform scripts. Your account authenticates the scripts and then Terraform updates the bucket name. Confirm the new bucket name through the Console.

Change Bucket Name
  1. In your editor, open the object_storage.tf file and change the bucket name from <your-bucket-name> to <your-bucket-name>-2.
    Note

    You can only rename empty buckets.
    ## This configuration was generated by terraform-provider-oci

data oci_objectstorage_namespace export_namespace {
  compartment_id = var.compartment_ocid
}
resource oci_objectstorage_bucket export_<your-bucket-name> {
  access_type    = "NoPublicAccess"
  auto_tiering   = "Disabled"
  compartment_id = var.compartment_ocid
  defined_tags = {
  }
  freeform_tags = {
  }
  #kms_key_id = <<Optional value not found in discovery>>
  metadata = {
  }
  name                  = "<your-bucket-name>-2"
  namespace             = data.oci_objectstorage_namespace.export_namespace.namespace
  object_events_enabled = "false"
  storage_tier          = "Standard"
  versioning            = "Disabled"
}
  2. Save the object_storage.tf file.
Initialize

When you create a state file, resource discovery initializes a working directory that includes Terraform configuration files. Make a habit to run this command every time you update your Terraform scripts.

  1. Check the contents of the resource-discovery directory. 
    ls -a

    You have a folder called .terraform that includes the plugins for the oci provider.

  2. Confirm that you have Terraform installed. 
    terraform -v
  3. Run the init command: 
    terraform init

    Example output:

    Initializing the backend...

Initializing provider plugins...
- Using previously-installed hashicorp/oci vx.x.x
...
Terraform has been successfully initialized!
Plan
An execution plan is the list of changes that Terraform plans to apply to your account.
  1. Create an execution plan: 
    terraform plan
  2. Review the execution plan.

    With the command,terraform plan you check whether the changes shown in the execution plan match your expectations, without changing to the real resources. Example output:

    Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
-/+ destroy and then create replacement

Terraform will perform the following actions:

  # oci_objectstorage_bucket.export_<your-bucket-name> must be replaced
-/+ resource "oci_objectstorage_bucket" "export_<your-bucket-name>" {
...
    ~ name = "<your-bucket-name>" -> "<your-bucket-name>-2" # forces replacement
        namespace = "<your-tenancy>"
        ...

Plan: 1 to add, 0 to change, 0 to destroy.
Apply

Update the Bucket

  1. Run your Terraform scripts:
    terraform apply
  2. When prompted for confirmation, enter yes, for the bucket name to be replaced.

    Example output:

    Apply complete! Resources: 1 added, 0 changed, 0 destroyed.

Find New Bucket Name in Console

  1. Open the navigation menu and click Storage. Under Object Storage & Archive Storage, click Buckets.
  2. In the left panel, for Compartment, select <your-compartment-name>.
  3. In the list of buckets, check the bucket name.

    The name of the bucket is now <your-bucket-name>-2.