Manage VM Clusters

Learn how to manage your VM clusters on Exadata Database Service on Cloud@Customer.

About Managing VM Clusters on Exadata Database Service on Cloud@Customer

The VM cluster provides a link between your Exadata Database Service on Cloud@Customer infrastructure and Oracle Databases you deploy.

The VM cluster contains an installation of Oracle Clusterware, which supports databases in the cluster. In the VM cluster definition, you also specify the number of enabled CPU cores, which determines the amount of CPU resources that are available to your databases

Before you can create any databases on your Exadata Cloud@Customer infrastructure, you must create a VM cluster network, and you must associate it with a VM cluster.

Note

Avoid entering confidential information when assigning descriptions, tags, or friendly names to your cloud resources through the Oracle Cloud Infrastructure Console, API, or CLI.

Overview of VM Cluster Node Subsetting

VM Cluster Node Subsetting enables you to allocate a subset of database servers to new and existing VM clusters to enable maximum flexibility in the allocation of compute (CPU, memory, local storage) resources.

With VM Cluster Node Subsetting, you can:
  • Create a smaller VM cluster to host databases that have low resource and scalability requirements or to host a smaller number of databases that require isolation from the rest of the workload.
  • Expand or shrink an existing VM cluster by adding and removing nodes to ensure optimal utilization of available resources.
Consider reviewing the points below that will assist you in subsetting VM cluster nodes.
  • VM Cluster Node Subsetting capability is available for new and existing VM clusters in Gen2 Exadata Cloud@Customer service.
  • All VMs across a VM cluster will have the same resource allocation per VM irrespective of whether the VM was created during cluster provisioning or added later by extending an existing VM cluster.
  • Any VM cluster should have a minimum of 2 VMs even with the node subsetting capability. We currently do not support clusters with a single VM.
  • Each VM cluster network is pre-provisioned with IP addresses for every DB Server in the infrastructure. One cluster network can only be used by a single VM cluster and is validated to ensure the IP addresses do not overlap with other cluster networks. Adding or removing VMs to the cluster does not impact the pre-provisioned IP addresses assigned to each DB server in the associated cluster network.
  • You can host a maximum of 8 VMs on X8M and above generation of DB Servers. X7 and X8 generations can only support a maximum of 6 and 5 VMs per DB Server, respectively.
  • Exadata Infrastructures with X8M and above generation of DB Servers can support a maximum of 16 VM clusters across all DB Servers. X7 and X8 generation Exadata Infrastructure DB Servers can only support a maximum of 12 and 10 VM clusters, respectively.

    Maximum number of clusters across the infrastructure depends on the resources available per DB server and is subject to the per DB Server maximum VM limit.

Introduction to Scale Up or Scale Down Operations

With the Multiple VMs per Exadata system (MultiVM) feature release, you can scale up or scale down your VM cluster resources.

Scaling Up or Scaling Down the VM Cluster Resources

You can scale up or scale down the memory, local disk size (/u02), ASM Storage, and CPUs.

Note

Oracle doesn't stop billing when a VM or VM Cluster is stopped. To stop billing for a VM Cluster, lower the OCPU count to zero.

Scaling up or down of these resources requires thorough auditing of existing usage and capacity management by the customer DB administrator. Review the existing usage to avoid failures during or after a scale down operation. While scaling up, consider how much of these resources are left for the next VM cluster you are planning to create. Exadata Cloud@Customer Cloud tooling calculates the current usage of memory, local disk, and ASM storage in the VM cluster, adds headroom to it, and arrives at a "minimum" value below which you cannot scale down, and expects that you specify the value below this minimum value.

Note

  • When creating or scaling a VM Cluster, setting the number of OCPUs to zero will shut down the VM Cluster and eliminate any billing for that VM Cluster, but the hypervisor will still reserve the minimum 2 OCPUs for each VM. These reserved OCPUs cannot be allocated to any other VMs, even though the VM to which they are allocated is shut down. The Control Plane does not account for reserved OCPUs when showing maximum available OCPU, so you should account for these reserved OCPU when performing any subsequent scaling operations to ensure the operation can acquire enough OCPUs to successfully complete the operation.
  • For memory and /u02 scale up or scale down operations, if the difference between the current value and the new value is less than 2%, then no change will be made to that VM. This is because memory change involves rebooting the VM, and /u02 change involves bringing down the Oracle Grid Infrastructure stack and un-mounting /u02. Productions customers will not resize for such a small increase or decrease, and hence such requests are a no-op.
  • You can scale the VM Cluster resources even if any of the DB servers in the VM Cluster are down:
    • If a DB server is down and scaling is performed, the VMs on that server will not be automatically scaled to the new OCPUs when the DB server and the VMs come back online. It's your responsibility to ensure that all the VMs in the cluster have the same OCPU values.
    • Even if the DB server is down, billing does not stop for the VM Cluster that has the VMs on that DB server.

Calculating the Minimum Required Memory

Cloud tooling provides dbaasapi to identify the minimum required memory. As root user, you have to run dbaasapi and pass a JSON file with sample content as follows. The only parameter that you need to update in the input.json is new_mem_size, which is the new memory to which you want the VM cluster to be re-sized.

cat input.json
{
"object": "db",
"action": "get",
"operation": "precheck_memory_resize",
"params": {
"dbname": "grid",
"new_mem_size" : "30 gb",
"infofile": "/tmp/result.json"
},
"outputfile": "/tmp/info.out",
"FLAGS": ""
}

dbaasapi -i input.json

cat /tmp/result.json
{
"is_new_mem_sz_allowed" : 0,
"min_req_mem" : 167
}

The result indicates that 30 GB is not sufficient and the minimum required memory is 167 GB, and that is the maximum you can reshape down to. On a safer side, you must choose a value greater than 167 GB, as there could be fluctuations of that order between this calculation and the next reshape attempt.

Calculating the ASM Storage

Use the following formula to calculate the minimum required ASM storage:

  • For each disk group, for example, DATA, RECO, note the total size and free size by running the asmcmd lsdg command on any Guest VM of the VM cluster.
  • Calculate the used size as (Total size - Free size) / 3 for each disk group. The /3 is used because the disk groups are triple mirrored.
  • DATA:RECO ratio is:

    80:20 if Local Backups option was NOT selected in the user interface.

    40:60 if Local Backups option was selected in the user interface.

  • Ensure that the new total size as given in the user interface passes the following conditions:

    Used size for DATA * 1.15 <= (New Total size * DATA % )

    Used size for RECO * 1.15 <= (New Total size * RECO % )

Example 5-3 Calculating the ASM Storage

  1. Run the asmcmd lsdg command in the Guest VM:
    • Without SPARSE:
      /u01/app/19.0.0.0/grid/bin/asmcmd lsdg
      ASMCMD>
      State   Type Rebal Sector Logical_Sector Block AU     Total_MB   Free_MB    Req_mir_free_MB   Usable_file_MB   Offline_disks    Voting_files   Name
      MOUNTED HIGH N        512     512        4096 4194304 12591936   10426224   1399104           3009040           0                       Y      DATAC5/
      MOUNTED HIGH N        512     512        4096 4194304 3135456    3036336    348384            895984            0                       N      RECOC5/
      ASMCMD>
    • With SPARSE:
      /u01/app/19.0.0.0/grid/bin/asmcmd lsdg
      ASMCMD>
      State   Type Rebal Sector Logical_Sector Block AU       Total_MB   Free_MB   Req_mir_free_MB   Usable_file_MB   Offline_disks    Voting_files   Name
      MOUNTED HIGH N        512     512        4096 4194304   12591936   10426224  1399104           3009040            0                       Y     DATAC5/
      MOUNTED HIGH N        512     512        4096 4194304   3135456    3036336   348384            895984             0                       N     RECOC5/
      MOUNTED HIGH N        512     512        4096 4194304   31354560   31354500  3483840           8959840            0                       N     SPRC5/
      ASMCMD>
    Note

    The listed values of all attributes for SPARSE diskgroup (SPRC5) present the virtual size. In Exadata DB Systems and Exadata Cloud@Customer, we use the ratio of 1:10 for physicalSize:virtualSize. Hence, for all purposes of our calculation we must use 1/10th of the values displayed above in case of SPARSE for those attributes.

  2. Used size for a disk group = (Total_MB - Free_MB) /3
    • Without SPARSE:

      Used size for DATAC5 = (12591936 - 10426224 ) / 3 = 704.98 GB

      Used size for RECO5 = (3135456 - 3036336 ) / 3 = 32.26 GB

    • With SPARSE:

      Used size for DATAC5 = (12591936 - 10426224 ) / 3 ~= 704.98 GB

      Used size for RECO5 = (3135456 - 3036336 ) /3 ~= 32.26 GB

      Used size for SPC5 = (1/10 * (31354560 - 31354500)) / 3 ~= 0 GB

  3. Storage distribution among diskgroups
    • Without SPARSE:

      DATA:RECO ratio is 80:20 in this example.

    • With SPARSE:

      DATA RECO: SPARSE ratio is 60:20:20 in this example.

  4. New requested size should pass the following conditions:
    • Without SPARSE: (For example, 5 TB in user interface.)

      5 TB = 5120 GB ; 5120 *.8 = 4096 GB; 5120 *.2 = 1024 GB

      For DATA: (704.98 * 1.15 ) <= 4096 GB

      For RECO: (32.36 * 1.15) <= 1024 GB

    • With SPARSE: (For example, 8 TB in the user interface.)

      8 TB = 8192 GB; 8192 *.6 = 4915 GB; 8192 *.2 = 1638 GB; 8192 *.2 = 1638 GB

      For DATA: (704.98 * 1.15 ) <= 4915 GB

      For RECO: (32.36 * 1.15) <= 1638 GB

      For SPR: (0 * 1.15) <= 1638 GB

Above resize will go through. If above conditions are not met by the new size, then resize will fail the precheck.

Estimating How Much Local Storage You Can Provision to Your VMs

X8-2 and X7-2 Systems

You specify how much space is provisioned from local storage to each VM. This space is mounted at location /u02, and is used primarily for Oracle Database homes. The amount of local storage available will vary with the number of virtual machines running on each physical node, as each VM requires a fixed amount of storage for the root file systems, GI homes, and diagnostic log space. Refer to the tables below to see the maximum amount of space available to provision to local storage (/u02) across all VMs.

  • Total space available for VM images (X7 All Systems): 1237 GB
  • Total space available for VM images (X8 All Systems): 1037 GB
  • Fixed storage per VM: 137 GB

Table 5-5 Space allocated to VMs

#VMs Fixed Storage All VMs (GB) X8-2 Space for ALL /u02 (GB) X7-2 Space for ALL /u02 (GB)

1

137

900

1100

2

274

763

963

3

411

626

826

4

548

489

689

5

685

352

552

6

822

N/A

415

For an X8-2, to get the max space available for the nth VM, take the number in the table above and subtract anything previously allocated for /u02 to the other VMs. So if you allocated 60 GB to VM1, 70 GB to VM2, 80 GB to VM3, 60 GB to VM4 (total 270 GB) in an X8-2, the maximum available for VM 5 would be 352 - 270 = 82 GB.

In ExaC@C Gen 2, we require a minimum of 60 GB per /u02, so with that minimum size there is a maximum of 5 VMs in X8-2 and 6 VMs in X7-2.

X8M-2 Systems

The maximum number of VMs for an X8M-2 will be 8, regardless of whether there is local disk space or other resources available.

For an X8M-2 system, the fixed consumption per VM is 160 GB.

Total space available to all VMs on an ExaC@C X8M databases node is 2500 GB. Although there is 2500 GB per database node, with a single VM, you can allocate a maximum of 900 GB local storage. Similarly, for the second VM, there is 1800 GB local storage available given the max limit of 900 GB per VM. With the third VM, the amount of space available is 2500 - (160Gb * 3) = 2020 GB. And so on for 4 and more VMs.

  • Total space available for VM images (X8M Base System): 1237 GB
  • Total space available for VM images (X8M Qtr/Half/Full Racks): 2500 GB
  • Fixed storage per VM: 160 GB

Table 5-6 Space allocated to VMs

#VMs Fixed Storage All VMs (GB) X8M-2 Base System Space for All /u02 (GB) X8M-2 Quarter/Half/Full Rack Space for All /u02 (GB)

1

160

900

900*

2

320

740

1800*

3

480

580

2020

4

640

420

1860

5

800

N/A

1700

6

960

N/A

1540

7

1120

N/A

1380

8

1280

N/A

1220

*Space is limited by 900 GB max per VM

For an X8M-2, to get the max space available for the nth VM, take the number in the table above and subtract anything previously allocated for /u02 to the other VMs. So, for a quarter and larger rack, if you allocated 60 GB to VM1, 70 GB to VM2, 80 GB to VM3, 60 GB to VM4 (total 270 GB) in an X8M-2, the maximum available for VM 5 would be 1700 - 270 = 1430 GB. However, the per VM maximum is 900 GB, so that would take precedent and limits VM5 to 900 GB.

X9M-2 Systems

  • Total Available for VM Images (Base System): 1077 GB
  • Total Available for VM Images (Qtr/Half/Full Racks): 2243 GB
  • Fixed overhead per VM: 184 GB

Table 5-7 Space allocated to VMs

#VMs Fixed Storage All VMs (GB) X9M-2 Base System Space All /u02 (GB) X9M-2 Qtr/Half/Full Racks All /u02 (GB)

1

184

892

900*

2

368

708

1800*

3

552

524

1691

4

736

340

1507

5

920

N/A

1323

6

1104

N/A

1139

7

1288

N/A

955

8

1472

N/A

771

*Space is limited by 900 GB max per VM

Scaling Local Storage Down

Scale Down Local Space Operation Guidelines

Scale down operation expects you to input local space value that you want each node to scale down to.

  • Resource Limit Based On Recommended Minimums

    Scale down operation must meet 60 GB recommended minimum size requirement for local storage.

  • Resource Limit Based On Current Utilization

    The scale down operation must leave 15% buffer on top of highest local space utilization across all nodes in the cluster.

The lowest local space per node allowed is higher of the above two limits.

Run df –kh command on each node to find out the node with the highest local storage.

You can also use the utility like cssh to issue the same command from all hosts in a cluster by typing it just once.

Lowest value of local storage each node can be scaled down to would be = 1.15x (highest value of local space used among all nodes).

Using the Console to Manage VM Clusters on Exadata Cloud@Customer

Learn how to use the console to create, edit, and manage your VM Clusters on Oracle Exadata Cloud@Customer.

Using the Console to Create a VM Cluster

To create your VM cluster, be prepared to provide values for the fields required for configuring the infrastructure.

To create a VM cluster, ensure that you have:

  • Active Exadata infrastructure is available to host the VM cluster.
  • A validated VM cluster network is available for the VM cluster to use.
  1. Open the navigation menu. Under Oracle Database, click Exadata Database Service on Cloud@Customer.
  2. Choose the Region that contains your Exadata infrastructure.
  3. Click VM Clusters.
  4. Click Create VM Cluster.
  5. Provide the requested information on the Create VM Cluster page:
    1. Choose a compartment: From the list of available compartments, choose the compartment that you want to contain the VM cluster.
    2. Provide the display name: The display name is a user-friendly name that you can use to identify the VM cluster. The name doesn't need to be unique because an Oracle Cloud Identifier (OCID) uniquely identifies the VM cluster.
    3. Select Exadata Database Service on Cloud@Customer Infrastructure: From the list, choose the Exadata infrastructure to host the VM cluster. You are not able to create a VM cluster without available and active Exadata infrastructure.
    4. Select a VM Cluster Network: From the list, choose a VM cluster network definition to use for the VM cluster. You must have an available and validated VM cluster network before you can create a VM cluster.
    5. Choose the Oracle Grid Infrastructure version: From the list, choose the Oracle Grid Infrastructure release that you want to install on the VM cluster.

      The Oracle Grid Infrastructure release determines the Oracle Database releases that can be supported on the VM cluster. You cannot run an Oracle Database release that is later than the Oracle Grid Infrastructure software release.

    6. Configure VM Cluster:
      • Click Select DB Servers for VM placement to allocate VM resources.
      • On the Select DB Servers dialog, select a minimum of two database servers for VM placement. Maximum resources available for allocation per VM are based on the number of database servers selected.
        Note

        • DB Servers, which already have 8 VMs running on them are not available for selection.
        • When calculating maximum local storage resources across selected DB Servers, the reserved local storage needed by the system to host a VM based on hardware generation is deducted from the DB Server with the least resources.

          For example, if the local storage available across selected DB servers is 823 GB for DB Server 3 and 813 GB for DB Server 4, then the minimum across selected servers is 813 GB and the maximum available for resource allocation is 813 GB - 184 GB (reserved local storage for hosting VM on X8M DB servers) = 629 GB.

          For more information, see Estimating How Much Local Storage You Can Provision to Your VMs.

      • Click Save Changes.
    7. Specify the OCPU count per VM: Specify the OCPU count for each individual VM. The minimum value is 2 OCPUs per VM (for a live VM condition), unless you are specifying zero OCPUs (for a shutdown VM condition).

      If you specify a value of zero, then the VM cluster virtual machines are all shut down at the end of the cluster creation process. In this case, you can later start the virtual machines by scaling the OCPU resources. See Using the Console to Scale the Resources on a VM Cluster.

      The value for OCPU count for the whole VM Cluster will be calculated automatically based upon the per VM OCPU count you have specified and the number of physical Database Servers configured for the system. There is one VM created on each physical Database Server available.

      OCPU: An Oracle Compute Unit (OCPU) provides CPU capacity equivalent of one physical core of an Intel Xeon processor with hyperthreading enabled. Each OCPU corresponds to two hardware execution threads, known as vCPUs.

      See, Oracle Platform as a Service and Infrastructure as a Service – Public Cloud Service DescriptionsMetered & Non-Metered.
    8. Requested OCPU count for the VM Cluster: Displays the total number of CPU cores allocated to the VM cluster based on the value you specified in the Specify the OCPU count per VM field. This field is not editable.
    9. Specify the memory per VM (GB): Specify the memory for each individual VM. The value must be a multiple of 1 GB and is limited by the available memory on the Exadata infrastructure.
    10. Requested memory for the VM Cluster (GB): Displays the total amount of memory allocated to the VM cluster based on the value you specified in the Specify the memory per VM (GB) field. This field is not editable.
    11. Specify the local file system size per VM (GB): Specify the local file system size for each individual VM. The value must be a multiple of 1 GB and is limited by the available size of the file system on the X8-2 and X7-2 infrastructures.

      Note that the minimum size of local system storage must be 60 GB. In addition to the 60 GB, each node of the VM must have at least 137 GB free for miscellaneous VM files. Each time when you create a new VM cluster, the space remaining out of the total available space is utilized for the new VM cluster.

      For more information and instructions to specify the size for each individual VM, see Introduction to Scale Up or Scale Down Operations.

    12. Reserved local storage per VM (GB): Displays the local storage size reserved internally for root file systems, Oracle Grid Infrastructure Homes, and diagnostic logs. This field is not editable.
    13. Configure the Exadata Storage: The following settings define how the Exadata storage is configured for use with the VM cluster. These settings cannot be changed after creating the VM cluster.
      • Specify Usable Exadata Storage: Specify the size for each individual VM. The minimum recommended size is 2 TB.
      • Allocate Storage for Exadata Snapshots: Check this option to create a sparse disk group, which is required to support Exadata snapshot functionality. Exadata snapshots enable space-efficient clones of Oracle databases that can be created and destroyed very quickly and easily.
      • Allocate Storage for Local Backups: Check this option to configure the Exadata storage to enable local database backups. If you select this option, more space is allocated to the RECO disk group to accommodate the backups. If you do not select this option, you cannot use local Exadata storage as a backup destination for any databases in the VM cluster.

      Table 5-8 Storage Allocation

      Storage Allocation DATA Disk Group RECO Disk Group SPARSE Disk Group

      Exadata Snapshots: No

      Enable Backups on Local Exadata Storage: No

      80%

      20%

      0% (The SPARSE disk group is not created.)

      Exadata Snapshots: No

      Enable Backups on Local Exadata Storage: Yes

      40%

      60%

      0% (The SPARSE disk group is not created.)

      Allocate Storage for Exadata Snapshots: Yes

      Enable Backups on Local Exadata Storage: No

      60%

      20%

      20%

      Allocate Storage for Exadata Snapshots: Yes

      Enable Backups on Local Exadata Storage: Yes

      35%

      50%

      15%

    14. Add SSH Key: Specify the public key portion of an SSH key pair that you want to use to access the VM cluster virtual machines. You can upload a file containing the key, or paste the SSH key string.

      To provide multiple keys, upload multiple key files or paste each key into a separate field. For pasted keys, ensure that each key is on a single, continuous line. The length of the combined keys cannot exceed 10,000 characters.

    15. Choose a license type:
      • Bring Your Own License (BYOL): Select this option if your organization already owns Oracle Database software licenses that you want to use on the VM cluster.
      • License Included: Select this option to subscribe to Oracle Database software licenses as part of Exadata Database Service on Cloud@Customer.
    16. Diagnostic Notification:

      Select the Enable option. Enabling diagnostic notification will allow you and Oracle Support to identify, investigate, track, and resolve guest VM issues quickly and effectively. For more information, see Overview of Database Service Events.

    17. Show Advanced Options:
      • Time zone: The default time zone for the Exadata Infrastructure is UTC, but you can specify a different time zone. The time zone options are those supported in both the Java.util.TimeZone class and the Oracle Linux operating system.
        Note

        If you want to set a time zone other than UTC or the browser-detected time zone, then select the Select another time zone option, select a Region or country, and then select the corresponding Time zone.

        If you do not see the region or country you want, then select Miscellaneous, and then select an appropriate Time zone.

      • Tags: Optionally, you can apply tags. If you have permission to create a resource, you also have permission to apply free-form tags to that resource. To apply a defined tag, you must have permission to use the tag namespace. For more information about tagging, see Resource Tags. If you are not sure if you should apply tags, skip this option (you can apply tags later) or ask your administrator.
  6. Click Create VM Cluster.

    The VM Cluster Details page is now displayed. While the creation process is running, the state of the VM cluster is Pending. When the VM cluster creation process completes, the state of the VM cluster changes to Available.

Using the Console to Enable or Disable Diagnostics Notification

You can enable or disable diagnostics collection for your Guest VMs after provisioning the VM cluster.

Enabling diagnostics collection at the VM cluster level applies the configuration to all the resources such as DB home, Database, and so on under the VM cluster.

  1. Open the navigation menu. Under Oracle Database, click Exadata Database Service on Cloud@Customer.
  2. Choose the Region that contains your Exadata infrastructure.
  3. Click VM Clusters.
  4. Click the name of the VM cluster you want to enable or disable diagnostic data collection.
  5. On the VM Cluster Details page, under General Information, enable or disable Diagnostic Notification.

Using the Console to Add VMs to a Provisioned Cluster

To add virtual machines to a provisioned cluster, use this procedure.

Consider reviewing the points below that will assist you in adding VMs to a provisioned cluster.
  • The same Guest OS Image version running on the existing provisioned VMs in the cluster is used to provision new VMs added to extend the VM cluster. However, any customizations made to the Guest OS Image on the existing VMs must be manually applied to the newly added VM.
  • For VM clusters running a Guest OS Image version older than a year, you must update the Guest OS Image version before adding a VM to extend the cluster.
  • Adding a VM to a cluster will not automatically extend any database which is part of a Data Guard configuration (either primary or standby) to the newly provisioned VM.
  • For databases not part of a Data Guard configuration, only databases that are running on all VMs in the existing cluster will be added to the newly provisioned VM. Any database running on a subset of VMs will not extend automatically to run on the newly added VM.
When you attempt to add a VM to a VM cluster, you might encounter the error [FATAL] [INS-32156] Installer has detected that there are non-readable files in oracle home. To resolve the issue, follow the steps outlined in Adding a VM to a VM Cluster Fails before you try adding a cluster node.
  1. Open the navigation menu. Under Oracle Database, click Exadata Database Service on Cloud@Customer.

    VM Clusters is selected by default.

  2. Choose your Compartment.

    A list of VM Clusters is displayed for the chosen Compartment.

  3. Click the name of a VM cluster where you want to add virtual machines.
  4. In the VM Cluster Details page, under Resources, click Virtual Machines, and then click Add Virtual Machines.
  5. On the Add Virtual Machines dialog, select additional DB servers on which to add the VM.

    You cannot unselect existing DB Servers. The maximum resources available per VM get updated based on the newly added DB servers.

  6. To view a list of DB Servers that have more than 2 VM Custers, click Show More.

    Clicking the link opens an overlay panel to list all the VM Clusters running on a particular DB Server.

To extend the database instance for Data Guard-enabled databases for the newly added VMs, see Nodelist is not Updated for Data Guard-Enabled Databases.

Using the Console to View a List of DB Servers on an Exadata Infrastructure

To view a list of database server hosts on an Oracle Exadata Cloud@Customer system, use this procedure.

  1. Open the navigation menu. Under Oracle Database, click Exadata Database Service on Cloud@Customer.
  2. Under Infrastructure, click Exadata Infrastructure.
  3. In the list of Exadata Infrastructures, click the display name of the infrastructure you wish to view details.
  4. Under Resources, click DB Servers.
  5. In the list of DB Servers, click the name of the DB Server that you wish to view details.

    DB Server lists VMs from each cluster hosted on them along with resources allocated to them.

Using the Console to Remove a VM from a VM Cluster

To remove a virtual machine from a provisioned cluster, use this procedure.

Note

Terminating a VM from a cluster requires the removal of any database which is part of a Data Guard configuration (either primary or standby) from the VM to proceed with the terminate flow. For more information on manual steps, see My Oracle Support note 2811352.1.
  1. Open the navigation menu. Under Oracle Database, click Exadata Database Service on Cloud@Customer.
  2. Choose the Region and Compartment that contains the VM cluster for which you want to scale the CPU resources.
  3. Click VM Clusters.
  4. Click the name of the VM cluster for which you want to remove a virtual machine.
  5. Under Resources, click Virtual Machines.
  6. In the list of virtual machines, click the Actions icon (three dots) for a virtual machine, and then click Terminate.
  7. On the Terminate Virtual Machine dialog, enter the name of the virtual machine, and then click Terminate.

    VM removed from the cluster. VM Cluster Details page displays the updated resource allocation details under VM Cluster Resource Allocation.

Using the Console to Update the License Type on a VM Cluster

To modify licensing, be prepared to provide values for the fields required for modifying the licensing information.

  1. Open the navigation menu. Under Oracle Database, click Exadata Database Service on Cloud@Customer.
  2. Choose the Region and Compartment that contains the VM cluster for which you want to update the license type.
  3. Click VM Clusters.
  4. Click the name of the VM cluster for which you want to update the license type.

    The VM Cluster Details page displays information about the selected VM cluster.

  5. Click Update License Type.
  6. In the dialog box, choose one of the following license types and then click Save Changes.
    • Bring Your Own License (BYOL): Select this option if your organization already owns Oracle Database software licenses that you want to use on the VM cluster.
    • License Included: Select this option to subscribe to Oracle Database software licenses as part of Exadata Database Service on Cloud@Customer.

    Updating the license type does not change the functionality or interrupt the operation of the VM cluster.

Using the Console to Add SSH Keys After Creating a VM Cluster

  1. Open the navigation menu. Under Oracle Database, click Exadata Cloud@Customer.
  2. Choose the Region that contains your Exadata infrastructure.
  3. Click VM Clusters.
  4. Click the name of the VM cluster that you want to add SSH key(s).
  5. In the VM Cluster Details page, click Add SSH Keys.
  6. In the ADD SSH Keys dialog, choose any one of the methods:
    • Generate SSH key pair: Select this option if you want the Control Plane to generate public/private key pairs for you.

      Click Save Private Key and Save Public Key to download and save SSH Key pair.

    • Upload SSH key files: Select this option to upload the file that contains SSH Key pair.
    • Paste SSH keys: Select this option to paste the SSH key string.

      To provide multiple keys, click Another SSH Key. For pasted keys, ensure that each key is on a single, continuous line. The length of the combined keys cannot exceed 10,000 characters.

  7. Click Save Changes.

Using the Console to Scale the Resources on a VM Cluster

Starting in Exadata Database Service on Cloud@Customer Gen2, you can scale up or down multiple resources at the same time. You can also scale up or down resources one at a time.

Scale down resources under the following circumstances:
  • Use Case 1: If you have allocated all of the resources to one VM cluster, and if you want to create multiple VM clusters, then there wouldn't be any resources available to allocate to the new clusters. Therefore, scale down the resources as needed to then create additional VM clusters.
  • Use Case 2: If you want to allocate different resources based on the workload, then scale down or scale up accordingly. For example, you may want to run nightly batch jobs for reporting/ETL and scale down the VM once the job is over.
You can scale down the following resources in any combinations:
  • OCPU
  • Memory
  • Local storage
  • Exadata storage

Each scale down operation can take approximately 15 minutes to complete. If you run multiple scale down operations, then all the operations are performed in a series. For example, if you scale down memory and local storage from the Console, then scaling down happens one after the other. Scaling down local storage and memory takes more time than the time taken to scale down OCPU and Exadata storage.

  1. Open the navigation menu. Under Oracle Database, click Exadata Database Service on Cloud@Customer.
  2. Choose the Region and Compartment that contains the VM cluster for which you want to scale the CPU resources.
  3. Click VM Clusters.
  4. Click the name of the VM cluster for which you want to scale the CPU resources.

    The VM Cluster Details page displays information about the selected VM cluster.

  5. Click Scale Up/Down.
  6. In the dialog box, adjust any or all of the following:
    • OCPU Count:

      The OCPU Count value must be a multiple of the number of virtual machines so that every virtual machine has the same number of CPU cores enabled.

      If you set the OCPU Count to zero, then the VM cluster virtual machines are all shut down. If you change from a zero setting, then the VM cluster virtual machines are all started. Otherwise, modifying the number of enabled CPU cores is an online operation, and virtual machines are not rebooted because of this operation. See also System Configuration.

      Note

      If you have explicitly set the CPU_COUNT database initialization parameter, that setting is not affected by modifying the number of CPU cores that are allocated to the VM cluster. Therefore, if you have enabled the Oracle Database instance caging feature, the database instance does not use extra CPU cores until you alter the CPU_COUNT setting. If CPU_COUNT is set to 0 (the default setting), then Oracle Database continuously monitors the number of CPUs reported by the operating system and uses the current count.
    • Memory:

      Specify the memory for each individual VM. The value must be a multiple of 1 GB and is limited by the available memory on the Exadata infrastructure.

      When you scale up or down the memory, the associated virtual machines are rebooted in a rolling manner one virtual machine at a time to minimize the impact on the VM cluster.

    • Local file system size:

      Specify the size for each individual VM. The value must be a multiple of 1 GB and is limited by the available size of the file system on the Exadata infrastructure.

      When you scale up or down the local file system size, the associated virtual machines are rebooted in a rolling manner one virtual machine at a time to minimize the impact on the VM cluster.

      Reserved local storage per VM (GB): Displays the size reserved internally for root file systems, Oracle Grid Infrastructure Homes, and diagnostic logs.

    • Usable Exadata storage size:

      Specify the total amount of Exadata storage that is allocated to the VM cluster. This storage is allocated evenly from all of the Exadata Storage Servers. The minimum recommended size is 2 TB.

      You may reduce the Exadata storage allocation for a VM cluster. However, you must ensure that the new amount covers the existing contents, and you should also allow for anticipated data growth.

      Note

      When you downsize, the new size must be at least 15% more than the currently used size.

      Modifying the Exadata storage allocated to the VM cluster is an online operation. Virtual machines are not rebooted because of this operation.

  7. . Click Save Changes.

Using the Console to Stop, Start, or Reboot a VM Cluster Virtual Machine

Use the console to stop, start, or reboot a virtual machine.

  1. Open the navigation menu. Under Oracle Database, click Exadata Database Service on Cloud@Customer.
  2. Choose the Region and Compartment that is associated with the VM cluster that contains the virtual machine that you want to stop, start, or reboot.
  3. Click VM Clusters.
  4. Click the name of the VM cluster that contains the virtual machine that you want to stop, start, or reboot.

    The VM Cluster Details page displays information about the selected VM cluster.

  5. In the Resources list, click Virtual Machines.

    The list of virtual machines is displayed.

  6. In the list of nodes, click the Actions icon (three dots) for a node, and then click one of the following actions:
    1. Start: Restarts a stopped node. After the node is restarted, the Stop action is enabled.
    2. Stop: Shuts down the node. After the node is stopped, the Start action is enabled.
    3. Reboot: Shuts down the node, and then restarts it.

Using the Console to Check the Status of a VM Cluster Virtual Machine

Review the health status of a VM cluster virtual machine.

  1. Open the navigation menu. Under Oracle Database, click Exadata Database Service on Cloud@Customer.
  2. Choose the Region and Compartment that is associated with the VM cluster that contains the virtual machine that you are interested in.
  3. Click VM Clusters.
  4. Click the name of the VM cluster that contains the virtual machine that you are interested in.

    The VM Cluster Details page displays information about the selected VM cluster.

  5. In the Resources list, click Virtual Machines.

    The list of virtual machines displays. For each virtual machine in the VM cluster, the name, state, and client IP address are displayed.

  6. In the node list, find the virtual machine that you are interested in and check its state.

    The color of the icon and the associated text it indicates its status.

    • Available: Green icon. The node is operational.
    • Starting: Yellow icon. The node is starting because of a start or reboot action in the Console or API.
    • Stopping: Yellow icon. The node is stopping because of a stop or reboot action in the Console or API.
    • Stopped: Yellow icon. The node is stopped.
    • Failed: Red icon. An error condition prevents the continued operation of the virtual machine.

Using the Console to Move a VM Cluster to Another Compartment

To change the compartment that contains your VM cluster on Exadata Database Service on Cloud@Customer, use this procedure.

When you move a VM cluster, the compartment change is also applied to the virtual machines and databases that are associated with the VM cluster. However, the compartment change does not affect any other associated resources, such as the Exadata infrastructure, which remains in its current compartment.

  1. Open the navigation menu. Under Oracle Database, click Exadata Database Service on Cloud@Customer.
  2. Choose the Region and Compartment that contains the VM cluster that you want to move.
  3. Click VM Clusters.
  4. Click the name of the VM cluster that you want to move.

    The VM Cluster Details page displays information about the selected VM cluster.

  5. Click Move Resource.
  6. In the resulting dialog, choose the new compartment for the VM cluster, and click Move Resource.

Using the Console to Terminate a VM Cluster

Before you can terminate a VM cluster, you must first terminate the databases that it contains.

Terminating a VM cluster removes it from the Cloud Control Plane. In the process, the virtual machines and their contents are destroyed.
  1. Open the navigation menu. Under Oracle Database, click Exadata Database Service on Cloud@Customer.
  2. Choose the Region and Compartment that contains the VM cluster that you want to terminate.
  3. Click VM Clusters.
  4. Click the name of the VM cluster that you want to terminate.

    The VM Cluster Details page displays information about the selected VM cluster.

  5. Click Terminate.
  6. In the resulting dialog, enter the name of the VM cluster, and click Terminate VM Cluster to confirm the action.

Using the API to Manage Exadata Cloud@Customer VM Clusters

Review the list of API calls to manage your Exadata Database Service on Cloud@Customer VM cluster networks and VM clusters.

For information about using the API and signing requests, see "REST APIs" and "Security Credentials". For information about SDKs, see "Software Development Kits and Command Line Interface".

Use these API operations to manage Exadata Database Service on Cloud@Customer VM cluster networks and VM clusters:

VM cluster networks:
  • GenerateRecommendedVmClusterNetwork
  • CreateVmClusterNetwork
  • DeleteVmClusterNetwork
  • GetVmClusterNetwork
  • ListVmClusterNetworks
  • UpdateVmClusterNetwork
  • ValidateVmClusterNetwork
VM clusters:
  • CreateVmCluster
  • DeleteVmCluster
  • GetVmCluster
  • ListVmClusters
  • UpdateVmCluster

For the complete list of APIs, see "Database Service API".