You can scale up or scale down the memory, local disk size (/u02), ASM Storage, and CPUs.

Scaling up or down of these resources requires thorough auditing of existing usage and capacity management by the customer DB administrator. Review the existing usage to avoid failures during or after a scale down operation. While scaling up, consider how much of these resources are left for the next VM cluster you are planning to create. Exadata Cloud@Customer Cloud tooling calculates the current usage of memory, local disk, and ASM storage in the VM cluster, adds headroom to it, and arrives at a "minimum" value below which you cannot scale down, and expects that you specify the value below this minimum value.

You can scale the database server memory up and down in a VM Cluster. Scaling memory requires a rolling restart of the database servers to take effect. For memory scaling to succeed, the databases must autostart in the Open state.

Changing the memory in a VM Cluster will affect the large pages (HugePages) settings for the VMs in that cluster. When a VM is initially created, each VM's operating system is configured with 50% of the memory allocated to the VM for large pages, and databases are configured to use that memory for their SGA. Oracle recommends you not modify the large pages configuration unless you understand the implication of any changes you make. Improper configurations can prevent all databases from starting, and even prevent the VM from booting.

Although modifying the large pages configuration is not recommended, it is permitted. However, any changes made may be overridden by cloud automation if the VM's memory is resized later. During a memory resize operation, cloud automation will attempt to maintain the large pages memory as a percentage of total memory, with a maximum limit of 60%. If large pages are configured to use more than 60% of the total memory, cloud automation will automatically resize it to this 60% limit.

EXACLOUD: Requested memory is insufficient. The new hugepage count is <<>>, which is less than the minimum required for the VM. Not proceeding with the change.

This process ensures there is enough conventional memory for the VM to boot. Before proceeding with the resize, automation performs a precheck to determine the current large pages usage by running database instances. If the precheck indicates that there will not be enough large pages memory after the resize to support the existing databases, the resize will fail, and the process will not continue.

Use the following formula to calculate the minimum required ASM storage:

• For each disk group, for example, DATA, RECO, note the total size and free size by running the asmcmd lsdg command on any Guest VM of the VM cluster.
• Calculate the used size as (Total size - Free size) / 3 for each disk group. The /3 is used because the disk groups are triple mirrored.

• DATA:RECO ratio is:

  80:20 if Local Backups option was NOT selected in the user interface.

  40:60 if Local Backups option was selected in the user interface.

• Ensure that the new total size as given in the user interface passes the following conditions:

  Used size for DATA * 1.15 <= (New Total size * DATA % )

  Used size for RECO * 1.15 <= (New Total size * RECO % )

VM Images include the files necessary to boot and run the VM and its operating system, as well as space for Oracle Homes stored in /u02. To estimate how much additional local storage space beyond the minimum can be allocated to any file system associated with a VM, subtract the size of the VM images for all VMs on a server from the total available space. If you have not modified the default VM Image size by expanding any file systems, use the VM Image size (default and minimum) below. If you have or plan to modify your VM Image size, you must use the OCI console and "Scale VM Cluster" action to check the allocated and available for an existing VM Cluster as expanding some non-/u02 file systems will consume more incremental storage than was added to the file system. This information is also available in the "Configure VM Cluster" action while creating a new VM Cluster.

X8-2 and X7-2 Systems

• Total space available for VM images (X7 All Systems): 1237 GB
• Total space available for VM images (X8 All Systems): 1037 GB
• VM Image size (default and minimum) including /u02: 244 GB
• Default (minimum) /u02: 60 GB

X8M-2 Systems

• Total space available for VM images (X8M Base System): 1237 GB
• Total space available for VM images (X8M Elastic): 2500 GB
• VM Image size (default and minimum) including /u02: 244 GB
• Default (minimum) /u02: 60 GB

X10M and X9M-2 Systems

• Total Available for VM Images (Base System X9M): 1077 GB
• Total Available for VM Images (Elastic): 2243 GB
• VM Image size (default and minimum) including /u02: 244 GB
• Default (minimum) /u02: 60 GB

Scale Local Space Operation Guidelines

You can scale local storage by modifying the size of many of the individual file systems in a VM. By default, the file systems are created at their minimum size. You can increase the size of the file systems as required. However, note that you can only shrink /u02. Other file systems can only be increased in size. The maximum supported size of any file system is 900 GB.

The storage consumed by all file systems is greater than the sum of the file system sizes. Refer to the calculations displayed in the OCI console to see the effects on free local storage when resizing a file system.

Using the OCI Console or API, you can increase or decrease the size of the following local file systems:

• /u02

Using the OCI Console or API, you can increase the size of following local filesystems:

• /
• /u01
• /tmp
• /var
• /var/log
• /var/log/audit
• /home

However, you cannot resize the following local file systems:

• /crashfiles
• /boot
• /acfs01
• /u01/app/19.0.0.0/grid

For more information about resizing these file systems, see Estimating How Much Local Storage You Can Provision to Your VMs.

Resource Limit Based On Current Utilization

• Any scale-down operation must leave 15% buffer on top of highest local space utilization across all nodes in the cluster.
• The lowest local space per node allowed is higher of the above two limits.
• Run the df –kh command on each node to find out the node with the highest local storage.
• You can also use the utility like cssh to issue the same command from all hosts in a cluster by typing it just once.
• Lowest value of local storage each node can be scaled down to would be = 1.15x (highest value of local space used among all nodes).

ACFS File Systems

If requested by support, you can also resize the /acfs01 file system. This file system is used by the system to stage software. It uses Exadata storage and is not subject to the limits described above for /u02. It is a shared file system visible from all nodes in the cluster, and can be online resized from the command line of any VM.

• Default size: The default size of /acfs01 is 100 GB.
• Scaling /acfs01: You can scale acfs01 as user grid from any VM via the /sbin/acfsutil command. No reboot is required. The resize operation will not affect the availability of the database service running in the VM Cluster. The following command issued by the grid user will increase the size of /acfs01 by 100 GB: /sbin/acfsutil size +100 GB /acfs01.
• You can create additional ACFS file systems if required. These will also consume storage from the Exadata Storage diskgroups and may be shared across all VMs in the cluster. Refer to the ACFS documentation for more information.

To create your VM cluster, be prepared to provide values for the fields required for configuring the infrastructure.

1. Open the navigation menu. Under Oracle Database, click Exadata Database Service on Cloud@Customer.
2. Choose the Region that contains your Exadata infrastructure.
3. Click VM Clusters.
4. Click Create VM Cluster.
5. Provide the requested information on the Create VM Cluster page:
   1. Choose a compartment: From the list of available compartments, choose the compartment that you want to contain the VM cluster.
   2. Provide the display name: The display name is a user-friendly name that you can use to identify the VM cluster. The name doesn't need to be unique because an Oracle Cloud Identifier (OCID) uniquely identifies the VM cluster.
   3. Select Exadata Database Service on Cloud@Customer Infrastructure: From the list, choose the Exadata infrastructure to host the VM cluster. You are not able to create a VM cluster without available and active Exadata infrastructure.
   4. Select a VM Cluster Network: From the list, choose a VM cluster network definition to use for the VM cluster. You must have an available and validated VM cluster network before you can create a VM cluster.
   5. Choose the Oracle Grid Infrastructure version: From the list, choose the Oracle Grid Infrastructure release (19c and 23ai) that you want to install on the VM cluster.

      The Oracle Grid Infrastructure release determines the Oracle Database releases that can be supported on the VM cluster. You cannot run an Oracle Database release that is later than the Oracle Grid Infrastructure software release.

      Note
      
      Minimum requirements for provisioning a VM Cluster with Grid Infrastructure 23ai:

      • Exadata Guest VM running Exadata System Software 23.1.8
      • Exadata Infrastructure running Exadata System Software 23.1.x
   6. Choose an Exadata image version:
      • Exadata infrastructure with Oracle Linux 7 and Exadata image version 22.1.10.0.0.230422:
        • The Change image button is not enabled.
        • The Oracle Grid Infrastructure version defaults to 19.0.0.0.0.
        • The Exadata guest version will be the same as that of the host OS.
      • Exadata infrastructure with Oracle Linux 8 and Exadata image version 23.1.3.0.0.230613:
        • The Exadata guest version defaults to the latest (23.1.3.0).
        • The Oracle Grid Infrastructure version defaults to 19.0.0.0.0
        • The Change image button is enabled.
        • Click Change image.

          The resulting Change image panel displays the list of available major versions of Exadata image (23.1.3.0 and 22.1.3.0).

          The most recent release for each major version is indicated by "(latest)".

        • Slide Display all available versions.

          Six past versions including the latest versions of Exadata images 23.1.3.0 and 22.1.3.0 are displayed.

        • Choose a version.
        • Click Save Changes.
   7. Configure VM Cluster:
      • Click Select DB Servers for VM placement to allocate VM resources.
      • On the Select DB Servers dialog, select a minimum of one database server for VM placement. If you require a high availability database service that remains available during maintenance and unplanned outages, select at least two database servers. Maximum resources available for allocation per VM are based on the number of database servers selected.
        Note
        
        

        • DB Servers, which already have 8 VMs running on them are not available for selection.
        • When calculating maximum local storage resources across selected DB Servers, the reserved local storage needed by the system to host a VM based on hardware generation is deducted from the DB Server with the least resources.

          For example, if the local storage available across selected DB servers is 823 GB for DB Server 3 and 813 GB for DB Server 4, then the minimum across selected servers is 813 GB and the maximum available for resource allocation is 813 GB - 184 GB (reserved local storage for hosting VM on X8M DB servers) = 629 GB.

          For more information, see Estimating How Much Local Storage You Can Provision to Your VMs.

      • Click Save Changes.
   8. Specify the OCPU count per VM: Specify the OCPU count to be provisioned for each VM in this cluster. The minimum value is 2 OCPUs per VM (for a live VM condition), unless you are specifying zero OCPUs (for a shutdown VM condition).

      If you specify a value of zero, then the VM cluster virtual machines are all shut down at the end of the cluster creation process. In this case, you can later start the virtual machines by scaling the OCPU resources. See Using the Console to Scale the Resources on a VM Cluster.

      The value for OCPU count for the whole VM Cluster will be calculated automatically based upon the per VM OCPU count you have specified, and the number of physical Database Servers configured for the VM Cluster.

      OCPU: An Oracle Compute Unit (OCPU) provides CPU capacity equivalent of one physical core of an Intel Xeon processor with hyperthreading enabled. Each OCPU corresponds to two hardware execution threads, known as vCPUs.

      See, Oracle Platform as a Service and Infrastructure as a Service – Public Cloud Service DescriptionsMetered & Non-Metered.
   9. Requested OCPU count for the VM Cluster: Displays the total number of CPU cores allocated to the VM cluster based on the value you specified in the Specify the OCPU count per VM field. This field is not editable.
   10. Specify the memory per VM (GB): Specify the memory for each individual VM. The value must be a multiple of 1 GB and is limited by the available memory on the Exadata infrastructure.
   11. Requested memory for the VM Cluster (GB): Displays the total amount of memory allocated to the VM cluster based on the value you specified in the Specify the memory per VM (GB) field. This field is not editable.
   12. Specify the local file system size per VM (GB): Specify the local file system size for each individual VM. The value must be a multiple of 1 GB and is limited by the available size of the file system on the X8-2 and X7-2 infrastructures.

       Note that the minimum size of local system storage must be 60 GB. Each time when you create a new VM cluster, the space remaining out of the total available space is utilized for the new VM cluster.

       For more information and instructions to specify the size for each individual VM, see Introduction to Scale Up or Scale Down Operations.

       1. Click Show additional local file systems configuration options.
       2. Resize the /, /u01, /tmp, /var, /var/log, /var/log/audit, and /home file systems as needed.
          Note
          
          

          • You can only expand these file systems and cannot reduce the size once expanded.
          • Due to backup partitions and mirroring, the / and /var file systems will consume twice the space they were allocated, which is indicated in the read-only Total allocated storage for / (GB) due to mirroring and Total allocated storage for /var (GB) due to mirroring fields.
          • After creating the VM Cluster, check the Exadata Resources section on the Exadata Infrastructure Details page to check the file size allocated to the local storage (/u02) and local storage (additional file systems).
   13. Reserved local storage per VM (GB): Displays the local storage size reserved internally for root file systems, Oracle Grid Infrastructure Homes, and diagnostic logs. This field is not editable.
   14. Configure the Exadata Storage: The following settings define how the Exadata storage is configured for use with the VM cluster. These settings cannot be changed after creating the VM cluster.
       Automatic Storage Management (ASM)

       • Specify Usable Exadata Storage: Specify the size for each individual VM. The minimum recommended size is 2 TB.
       • Allocate Storage for Exadata Snapshots: Check this option to create a sparse disk group, which is required to support Exadata snapshot functionality. Exadata snapshots enable space-efficient clones of Oracle databases that can be created and destroyed very quickly and easily.
       • Allocate Storage for Local Backups: Check this option to configure the Exadata storage to enable local database backups. If you select this option, more space is allocated to the RECO disk group to accommodate the backups. If you do not select this option, you cannot use local Exadata storage as a backup destination for any databases in the VM cluster.

       Table 5-15 Storage Allocation

       Storage Allocation	DATA Disk Group	RECO Disk Group	SPARSE Disk Group
       Exadata Snapshots: No Enable Backups on Local Exadata Storage: No	80%	20%	0% (The SPARSE disk group is not created.)
       Exadata Snapshots: No Enable Backups on Local Exadata Storage: Yes	40%	60%	0% (The SPARSE disk group is not created.)
       Allocate Storage for Exadata Snapshots: Yes Enable Backups on Local Exadata Storage: No	60%	20%	20%
       Allocate Storage for Exadata Snapshots: Yes Enable Backups on Local Exadata Storage: Yes	35%	50%	15%

   15. Add SSH Key: Specify the public key portion of an SSH key pair that you want to use to access the VM cluster virtual machines. You can upload a file containing the key, or paste the SSH key string.

       To provide multiple keys, upload multiple key files or paste each key into a separate field. For pasted keys, ensure that each key is on a single, continuous line. The length of the combined keys cannot exceed 10,000 characters.

   16. Choose a license type:
       • Bring Your Own License (BYOL): Select this option if your organization already owns Oracle Database software licenses that you want to use on the VM cluster.
       • License Included: Select this option to subscribe to Oracle Database software licenses as part of Exadata Database Service on Cloud@Customer.
   17. Diagnostics Collection:

       By enabling diagnostics collection and notifications, Oracle Cloud Operations and you will be able to identify, investigate, track, and resolve guest VM issues quickly and effectively. Subscribe to Events to get notified about resource state changes. For more information, see Getting Started with Events.

       Note
       
       You are opting in with the understanding that the list of events, metrics, and log files collected can change in the future. You can opt out of this feature at any time.

       • Enable Diagnostic Events: Allow Oracle to collect and publish critical, warning, error, and information events to me.
       • Enable Health Monitoring: Allow Oracle to collect health metrics/events such as Oracle Database up/down, disk space usage, and so on, and share them with Oracle Cloud operations. You will also receive notification of some events.

       • Enable Incident Logs and Trace Collection: Allow Oracle to collect incident logs and traces to enable fault diagnosis and issue resolution.

         All three checkboxes are selected by default. You can leave the default settings as is or clear the checkboxes as needed. You can view the Diagnostic Collection settings on the VM Cluster Details page under General Information >> Diagnostics Collection.

         • Enabled: When you choose to collect diagnostics, health metrics, incident logs, and trace files (all three options).

         • Disabled: When you choose not to collect diagnostics, health metrics, incident logs, and trace files (all three options).

         • Partially Enabled: When you choose to collect diagnostics, health metrics, incident logs, and trace files ( one or two options).
   18. Show Advanced Options:
       • Time zone: The default time zone for the Exadata Infrastructure is UTC, but you can specify a different time zone. The time zone options are those supported in both the Java.util.TimeZone class and the Oracle Linux operating system.
         Note
         
         

         If you want to set a time zone other than UTC or the browser-detected time zone, then select the Select another time zone option, select a Region or country, and then select the corresponding Time zone.

         If you do not see the region or country you want, then select Miscellaneous, and then select an appropriate Time zone.

       • Tags: Optionally, you can apply tags. If you have permission to create a resource, you also have permission to apply free-form tags to that resource. To apply a defined tag, you must have permission to use the tag namespace. For more information about tagging, see Resource Tags. If you are not sure if you should apply tags, skip this option (you can apply tags later) or ask your administrator.
6. Optionally, you can save the resource configuration as a stack.
   • To save the resource configuration as a Stack:
     1. Click Save as Stack.
     2. In the resulting Save as Stack dialog, provide the following details:
        1. Name: (Optional) Provide an easy to remember descriptive name.
        2. Description: (Optional) Enter a short description.
        3. Compartment: Select a compartment where this Stack will reside.
        4. Tags: Add tags.
     3. Click Save.

        After saving the Stack, the system displays a banner with a link to the saved Stack.

     4. Click the link to open the Stack in the Resource Manager Service console.

        See, Resource Manager and Terraform.

   • To view the details of a Stack:
     1. Open the navigation menu. Under Developer Services, click Resource Manager.
     2. Click Stacks.
     3. Click the name of the Stack that you want to view details.

        Or, click the Actions menu (three dots), and select the View stack details option.

7. Click Create VM Cluster.

   The VM Cluster Details page is now displayed. While the creation process is running, the state of the VM cluster is Pending. When the VM cluster creation process completes, the state of the VM cluster changes to Available.

   The Exadata Database Storage section on the VM Cluster Details page shows the type of storage configured, which, in this case, is ASM.

To create your Exascale VM cluster, be prepared to provide values for the fields required for configuring the infrastructure.

1. Open the navigation menu. Under Oracle Database, click Exadata Database Service on Cloud@Customer.
2. Choose the Region that contains your Exadata infrastructure.
3. Click VM Clusters.
4. Click Create VM Cluster.
5. Provide the requested information on the Create VM Cluster page:
   1. Choose a compartment: From the list of available compartments, choose the compartment that you want to contain the VM cluster.
   2. Provide the display name: The display name is a user-friendly name that you can use to identify the VM cluster. The name doesn't need to be unique because an Oracle Cloud Identifier (OCID) uniquely identifies the VM cluster.
   3. Select Exadata Database Service on Cloud@Customer Infrastructure: From the list, choose the Exadata infrastructure to host the VM cluster. You are not able to create a VM cluster without available and active Exadata infrastructure.
   4. Select a VM Cluster Network: From the list, choose a VM cluster network definition to use for the VM cluster. You must have an available and validated VM cluster network before you can create a VM cluster.
   5. Choose the Oracle Grid Infrastructure version: From the list, choose the Oracle Grid Infrastructure release (19c and 23ai) that you want to install on the VM cluster.

      The Oracle Grid Infrastructure release determines the Oracle Database releases that can be supported on the VM cluster. You cannot run an Oracle Database release that is later than the Oracle Grid Infrastructure software release.

      Note
      
      Minimum requirements for provisioning a VM Cluster with Grid Infrastructure 23ai:

      • Exadata Guest VM running Exadata System Software 23.1.8
      • Exadata Infrastructure running Exadata System Software 23.1.x
   6. Choose an Exadata image version:
      • Exadata infrastructure with Oracle Linux 7 and Exadata image version 22.1.10.0.0.230422:
        • The Change image button is not enabled.
        • The Oracle Grid Infrastructure version defaults to 19.0.0.0.0.
        • The Exadata guest version will be the same as that of the host OS.
      • Exadata infrastructure with Oracle Linux 8 and Exadata image version 23.1.3.0.0.230613:
        • The Exadata guest version defaults to the latest (23.1.3.0).
        • The Oracle Grid Infrastructure version defaults to 19.0.0.0.0
        • The Change image button is enabled.
        • Click Change image.

          The resulting Change image panel displays the list of available major versions of Exadata image (23.1.3.0 and 22.1.3.0).

          The most recent release for each major version is indicated by "(latest)".

        • Slide Display all available versions.

          Six past versions including the latest versions of Exadata images 23.1.3.0 and 22.1.3.0 are displayed.

        • Choose a version.
        • Click Save Changes.
   7. Configure VM Cluster:
      • Click Select DB Servers for VM placement to allocate VM resources.
      • On the Select DB Servers dialog, select a minimum of one database server for VM placement. If you require a high availability database service that remains available during maintenance and unplanned outages, select at least two database servers. Maximum resources available for allocation per VM are based on the number of database servers selected.
        Note
        
        

        • DB Servers, which already have 8 VMs running on them are not available for selection.
        • When calculating maximum local storage resources across selected DB Servers, the reserved local storage needed by the system to host a VM based on hardware generation is deducted from the DB Server with the least resources.

          For example, if the local storage available across selected DB servers is 823 GB for DB Server 3 and 813 GB for DB Server 4, then the minimum across selected servers is 813 GB and the maximum available for resource allocation is 813 GB - 184 GB (reserved local storage for hosting VM on X8M DB servers) = 629 GB.

          For more information, see Estimating How Much Local Storage You Can Provision to Your VMs.

      • Click Save Changes.
   8. Specify the OCPU count per VM: Specify the OCPU count to be provisioned for each VM in this cluster. The minimum value is 2 OCPUs per VM (for a live VM condition), unless you are specifying zero OCPUs (for a shutdown VM condition).

      If you specify a value of zero, then the VM cluster virtual machines are all shut down at the end of the cluster creation process. In this case, you can later start the virtual machines by scaling the OCPU resources. See Using the Console to Scale the Resources on a VM Cluster.

      The value for OCPU count for the whole VM Cluster will be calculated automatically based upon the per VM OCPU count you have specified, and the number of physical Database Servers configured for the VM Cluster.

      OCPU: An Oracle Compute Unit (OCPU) provides CPU capacity equivalent of one physical core of an Intel Xeon processor with hyperthreading enabled. Each OCPU corresponds to two hardware execution threads, known as vCPUs.

      See, Oracle Platform as a Service and Infrastructure as a Service – Public Cloud Service DescriptionsMetered & Non-Metered.
   9. Requested OCPU count for the VM Cluster: Displays the total number of CPU cores allocated to the VM cluster based on the value you specified in the Specify the OCPU count per VM field. This field is not editable.
   10. Specify the memory per VM (GB): Specify the memory for each individual VM. The value must be a multiple of 1 GB and is limited by the available memory on the Exadata infrastructure.
   11. Requested memory for the VM Cluster (GB): Displays the total amount of memory allocated to the VM cluster based on the value you specified in the Specify the memory per VM (GB) field. This field is not editable.
   12. Specify the local file system size per VM (GB): Specify the local file system size for each individual VM. The value must be a multiple of 1 GB and is limited by the available size of the file system on the X8-2 and X7-2 infrastructures.

       Note that the minimum size of local system storage must be 60 GB. Each time when you create a new VM cluster, the space remaining out of the total available space is utilized for the new VM cluster.

       For more information and instructions to specify the size for each individual VM, see Introduction to Scale Up or Scale Down Operations.

       1. Click Show additional local file systems configuration options.
       2. Resize the /, /u01, /tmp, /var, /var/log, /var/log/audit, and /home file systems as needed.
          Note
          
          

          • You can only expand these file systems and cannot reduce the size once expanded.
          • Due to backup partitions and mirroring, the / and /var file systems will consume twice the space they were allocated, which is indicated in the read-only Total allocated storage for / (GB) due to mirroring and Total allocated storage for /var (GB) due to mirroring fields.
          • After creating the VM Cluster, check the Exadata Resources section on the Exadata Infrastructure Details page to check the file size allocated to the local storage (/u02) and local storage (additional file systems).
   13. Reserved local storage per VM (GB): Displays the local storage size reserved internally for root file systems, Oracle Grid Infrastructure Homes, and diagnostic logs. This field is not editable.
   14. Configure the Exadata Storage: The following settings define how the Exadata storage is configured for use with the VM cluster. These settings cannot be changed after creating the VM cluster.
       Exascale - Exascale database storage vault:

       • Create new storage vault:
         • Storage vault name: Enter a descriptive name for the vault. Click the change compartment link and choose a compartment if you want to create this vault in a different compartment.
         • Storage capacity for databases: Enter the storage capacity for the databases within the minimum and maximum values displayed on the screen.
       • Select existing storage vault: Select a vault that resides in the compartment of your choice.
   15. Add SSH Key: Specify the public key portion of an SSH key pair that you want to use to access the VM cluster virtual machines. You can upload a file containing the key, or paste the SSH key string.

       To provide multiple keys, upload multiple key files or paste each key into a separate field. For pasted keys, ensure that each key is on a single, continuous line. The length of the combined keys cannot exceed 10,000 characters.

   16. Choose a license type:
       • Bring Your Own License (BYOL): Select this option if your organization already owns Oracle Database software licenses that you want to use on the VM cluster.
       • License Included: Select this option to subscribe to Oracle Database software licenses as part of Exadata Database Service on Cloud@Customer.
   17. Diagnostics Collection:

       By enabling diagnostics collection and notifications, Oracle Cloud Operations and you will be able to identify, investigate, track, and resolve guest VM issues quickly and effectively. Subscribe to Events to get notified about resource state changes. For more information, see Getting Started with Events.

       Note
       
       You are opting in with the understanding that the list of events, metrics, and log files collected can change in the future. You can opt out of this feature at any time.

       • Enable Diagnostic Events: Allow Oracle to collect and publish critical, warning, error, and information events to me.
       • Enable Health Monitoring: Allow Oracle to collect health metrics/events such as Oracle Database up/down, disk space usage, and so on, and share them with Oracle Cloud operations. You will also receive notification of some events.

       • Enable Incident Logs and Trace Collection: Allow Oracle to collect incident logs and traces to enable fault diagnosis and issue resolution.

         All three checkboxes are selected by default. You can leave the default settings as is or clear the checkboxes as needed. You can view the Diagnostic Collection settings on the VM Cluster Details page under General Information >> Diagnostics Collection.

         • Enabled: When you choose to collect diagnostics, health metrics, incident logs, and trace files (all three options).

         • Disabled: When you choose not to collect diagnostics, health metrics, incident logs, and trace files (all three options).

         • Partially Enabled: When you choose to collect diagnostics, health metrics, incident logs, and trace files ( one or two options).
   18. Show Advanced Options:
       • Time zone: The default time zone for the Exadata Infrastructure is UTC, but you can specify a different time zone. The time zone options are those supported in both the Java.util.TimeZone class and the Oracle Linux operating system.
         Note
         
         

         If you want to set a time zone other than UTC or the browser-detected time zone, then select the Select another time zone option, select a Region or country, and then select the corresponding Time zone.

         If you do not see the region or country you want, then select Miscellaneous, and then select an appropriate Time zone.

       • Tags: Optionally, you can apply tags. If you have permission to create a resource, you also have permission to apply free-form tags to that resource. To apply a defined tag, you must have permission to use the tag namespace. For more information about tagging, see Resource Tags. If you are not sure if you should apply tags, skip this option (you can apply tags later) or ask your administrator.
6. Optionally, you can save the resource configuration as a stack.
   • To save the resource configuration as a Stack:
     1. Click Save as Stack.
     2. In the resulting Save as Stack dialog, provide the following details:
        1. Name: (Optional) Provide an easy to remember descriptive name.
        2. Description: (Optional) Enter a short description.
        3. Compartment: Select a compartment where this Stack will reside.
        4. Tags: Add tags.
     3. Click Save.

        After saving the Stack, the system displays a banner with a link to the saved Stack.

     4. Click the link to open the Stack in the Resource Manager Service console.

        See, Resource Manager and Terraform.

   • To view the details of a Stack:
     1. Open the navigation menu. Under Developer Services, click Resource Manager.
     2. Click Stacks.
     3. Click the name of the Stack that you want to view details.

        Or, click the Actions menu (three dots), and select the View stack details option.

7. Click Create VM Cluster.

   The VM Cluster Details page is now displayed. While the creation process is running, the state of the VM cluster is Pending. When the VM cluster creation process completes, the state of the VM cluster changes to Available.

   The Exadata Database Storage section on the VM Cluster Details page shows the type of storage configured, which, in this case, is Exascale.

You can enable, partially enable, or disable diagnostics collection for your Guest VMs after provisioning the VM cluster. Enabling diagnostics collection at the VM cluster level applies the configuration to all the resources such as DB home, Database, and so on under the VM cluster.

1. Open the navigation menu. Under Oracle Database, click Exadata Database Service on Cloud@Customer.
2. Choose the Region that contains your Exadata infrastructure.
3. Click VM Clusters.
4. Click the name of the VM cluster you want to enable or disable diagnostic data collection.
5. On the VM Cluster Details page, under General Information, enable, partially enable, or disable Diagnostics Collection.
6. Click Edit.

   Edit Diagnostics Collection Settings window is displayed.

7. Select or clear the checkboxes and then click Save Changes.

To add virtual machines to a provisioned cluster, use this procedure.

1. Open the navigation menu. Under Oracle Database, click Exadata Database Service on Cloud@Customer.

   VM Clusters is selected by default.

2. Choose your Compartment.

   A list of VM Clusters is displayed for the chosen Compartment.

3. Click the name of a VM cluster where you want to add virtual machines.
4. In the VM Cluster Details page, under Resources, click Virtual Machines, and then click Add Virtual Machines.
5. On the Add Virtual Machines dialog, select additional DB servers on which to add the VM.

   You cannot unselect existing DB Servers. The maximum resources available per VM get updated based on the newly added DB servers.

   DB Server Statuses include In this VM cluster, Network not configured, Available to add, and Insufficient resources. You can only add DB servers with the Available to add status.

   DB servers that do not have a network configured are not available to add. To configure the network, edit the VM Cluster Network of the associated infrastructure. For more information, see Using the Console to Add Another DB Server to the VM Cluster Network.

6. Select the DB servers with the Available to add status and then click Save Changes.

   The statuses of the DB servers change to Allocated.

   Note
   
   You cannot remove an allocated DB server.

To view a list of database server hosts on an Oracle Exadata Cloud@Customer system, use this procedure.

1. Open the navigation menu. Under Oracle Database, click Exadata Database Service on Cloud@Customer.
2. Under Infrastructure, click Exadata Infrastructure.
3. In the list of Exadata Infrastructures, click the display name of the infrastructure you wish to view details.
4. Under Resources, click DB Servers.
5. In the list of DB Servers, click the name of the DB Server that you wish to view details.

   DB Server lists VMs from each cluster hosted on them along with resources allocated to them.

To remove a virtual machine from a provisioned cluster, use this procedure.

1. Open the navigation menu. Under Oracle Database, click Exadata Database Service on Cloud@Customer.
2. Choose the Region and Compartment that contains the VM cluster for which you want to scale the CPU resources.
3. Click VM Clusters.
4. Click the name of the VM cluster for which you want to remove a virtual machine.
5. Under Resources, click Virtual Machines.
6. In the list of virtual machines, click the Actions icon (three dots) for a virtual machine, and then click Remove.
7. On the Terminate Virtual Machine dialog, enter the name of the virtual machine, and then click Remove.

   VM removed from the cluster. VM Cluster Details page displays the updated resource allocation details under VM Cluster Resource Allocation.

To modify licensing, be prepared to provide values for the fields required for modifying the licensing information.

1. Open the navigation menu. Under Oracle Database, click Exadata Database Service on Cloud@Customer.
2. Choose the Region and Compartment that contains the VM cluster for which you want to update the license type.
3. Click VM Clusters.
4. Click the name of the VM cluster for which you want to update the license type.

   The VM Cluster Details page displays information about the selected VM cluster.

5. Click Update License Type.
6. In the dialog box, choose one of the following license types and then click Save Changes.
   • Bring Your Own License (BYOL): Select this option if your organization already owns Oracle Database software licenses that you want to use on the VM cluster.
   • License Included: Select this option to subscribe to Oracle Database software licenses as part of Exadata Database Service on Cloud@Customer.

   Updating the license type does not change the functionality or interrupt the operation of the VM cluster. Customers are permitted to change the license type for a VM Cluster at most once per month.

1. Open the navigation menu. Under Oracle Database, click Exadata Cloud@Customer.
2. Choose the Region that contains your Exadata infrastructure.
3. Click VM Clusters.
4. Click the name of the VM cluster that you want to add SSH key(s).
5. In the VM Cluster Details page, click Add SSH Keys.
6. In the ADD SSH Keys dialog, choose any one of the methods:
   • Generate SSH key pair: Select this option if you want the Control Plane to generate public/private key pairs for you.

     Click Save Private Key and Save Public Key to download and save SSH Key pair.

   • Upload SSH key files: Select this option to upload the file that contains SSH Key pair.
   • Paste SSH keys: Select this option to paste the SSH key string.

     To provide multiple keys, click Another SSH Key. For pasted keys, ensure that each key is on a single, continuous line. The length of the combined keys cannot exceed 10,000 characters.

7. Click Save Changes.

Starting in Oracle Exadata Database Service on Cloud@Customer, you can scale up or down multiple resources at the same time. You can also scale up or down resources one at a time.

1. Open the navigation menu. Under Oracle Database, click Exadata Database Service on Cloud@Customer.
2. Choose the Region and Compartment that contains the VM cluster for which you want to scale the CPU resources.
3. Click VM Clusters.
4. Click the name of the VM cluster for which you want to scale the CPU resources.

   The VM Cluster Details page displays information about the selected VM cluster.

5. Click Scale Up/Down.
6. In the dialog box, adjust any or all of the following:
   • OCPU Count:

     The OCPU Count value must be a multiple of the number of virtual machines so that every virtual machine has the same number of CPU cores enabled.

     If you set the OCPU Count to zero, then the VM cluster virtual machines are all shut down. If you change from a zero setting, then the VM cluster virtual machines are all started. Otherwise, modifying the number of enabled CPU cores is an online operation, and virtual machines are not rebooted because of this operation. See also System Configuration.

     Note
     
     If you have explicitly set the CPU_COUNT database initialization parameter, that setting is not affected by modifying the number of CPU cores that are allocated to the VM cluster. Therefore, if you have enabled the Oracle Database instance caging feature, the database instance does not use extra CPU cores until you alter the CPU_COUNT setting. If CPU_COUNT is set to 0 (the default setting), then Oracle Database continuously monitors the number of CPUs reported by the operating system and uses the current count.
   • Memory:

     Specify the memory for each individual VM. The value must be a multiple of 1 GB and is limited by the available memory on the Exadata infrastructure.

     When you scale up or down the memory, the associated virtual machines are rebooted in a rolling manner one virtual machine at a time to minimize the impact on the VM cluster.

   • Local file system size:

     Specify the size for each individual VM. The value must be a multiple of 1 GB and is limited by the available size of the file system on the Exadata infrastructure.

     When you scale up or down the local file system size, the associated virtual machines are rebooted in a rolling manner one virtual machine at a time to minimize the impact on the VM cluster.

     1. Click Show additional local file systems configuration options.
     2. Resize the /, /u01, /tmp, /var, /var/log, /var/log/audit, and /home file systems as needed.
        Note
        
        

        • You can only expand these file systems and cannot reduce the size once expanded.
        • Due to backup partitions and mirroring, the / and /var file systems will consume twice the space they were allocated, which is indicated in the read-only Total allocated storage for / (GB) due to mirroring and Total allocated storage for /var (GB) due to mirroring fields.
     3. After creating the VM Cluster, check the Exadata Resources section on the Exadata Infrastructure Details page to check the file size allocated to the local storage (/u02) and local storage (additional file systems).

     Reserved local storage per VM (GB): Displays the size reserved internally for root file systems, Oracle Grid Infrastructure Homes, and diagnostic logs.

   • Usable Exadata storage size:

     Specify the total amount of Exadata storage that is allocated to the VM cluster. This storage is allocated evenly from all of the Exadata Storage Servers. The minimum recommended size is 2 TB.

     You may reduce the Exadata storage allocation for a VM cluster. However, you must ensure that the new amount covers the existing contents, and you should also allow for anticipated data growth.

     Note
     
     When you downsize, the new size must be at least 15% more than the currently used size.

     Modifying the Exadata storage allocated to the VM cluster is an online operation. Virtual machines are not rebooted because of this operation.

7. . Click Save Changes.

Use the console to stop, start, or reboot a virtual machine.

1. Open the navigation menu. Under Oracle Database, click Exadata Database Service on Cloud@Customer.
2. Choose the Region and Compartment that is associated with the VM cluster that contains the virtual machine that you want to stop, start, or reboot.
3. Click VM Clusters.
4. Click the name of the VM cluster that contains the virtual machine that you want to stop, start, or reboot.

   The VM Cluster Details page displays information about the selected VM cluster.

5. In the Resources list, click Virtual Machines.

   The list of virtual machines is displayed.

6. In the list of nodes, click the Actions icon (three dots) for a node, and then click one of the following actions:
   1. Start: Restarts a stopped node. After the node is restarted, the Stop action is enabled.
   2. Stop: Shuts down the node. After the node is stopped, the Start action is enabled.
   3. Reboot: Shuts down the node, and then restarts it.

Review the health status of a VM cluster virtual machine.

1. Open the navigation menu. Under Oracle Database, click Exadata Database Service on Cloud@Customer.
2. Choose the Region and Compartment that is associated with the VM cluster that contains the virtual machine that you are interested in.
3. Click VM Clusters.
4. Click the name of the VM cluster that contains the virtual machine that you are interested in.

   The VM Cluster Details page displays information about the selected VM cluster.

5. In the Resources list, click Virtual Machines.

   The list of virtual machines displays. For each virtual machine in the VM cluster, the name, state, and client IP address are displayed.

6. In the node list, find the virtual machine that you are interested in and check its state.

   The color of the icon and the associated text it indicates its status.

   • Available: Green icon. The node is operational.
   • Starting: Yellow icon. The node is starting because of a start or reboot action in the Console or API.
   • Stopping: Yellow icon. The node is stopping because of a stop or reboot action in the Console or API.
   • Stopped: Yellow icon. The node is stopped.
   • Failed: Red icon. An error condition prevents the continued operation of the virtual machine.

To change the compartment that contains your VM cluster on Oracle Exadata Database Service on Cloud@Customer, use this procedure.

1. Open the navigation menu. Under Oracle Database, click Exadata Database Service on Cloud@Customer.
2. Choose the Region and Compartment that contains the VM cluster that you want to move.
3. Click VM Clusters.
4. Click the name of the VM cluster that you want to move.

   The VM Cluster Details page displays information about the selected VM cluster.

5. Click Move Resource.
6. In the resulting dialog, choose the new compartment for the VM cluster, and click Move Resource.

Before you can terminate a VM cluster, you must first terminate the databases that it contains.

1. Open the navigation menu. Under Oracle Database, click Exadata Database Service on Cloud@Customer.
2. Choose the Region and Compartment that contains the VM cluster that you want to terminate.
3. Click VM Clusters.
4. Click the name of the VM cluster that you want to terminate.

   The VM Cluster Details page displays information about the selected VM cluster.

5. Click Terminate.
6. In the resulting dialog, enter the name of the VM cluster, and click Terminate VM Cluster to confirm the action.

An administrator must grant you secure access to the virtual machine console on the Exadata Database Service on Cloud@Customer system through an IAM policy.

This access is required whether you're using the Console or the REST API with an SDK, CLI, or other tools. If you get a message that you don’t have permission or are unauthorized, verify with your administrator what type of access you have and which compartment to work in.

To create virtual machine console connections, an administrator needs to grant user access to read and manage virtual machine console connections through an IAM policy. The resource name for virtual machine console connections is dbnode-console-connection. The resource name for virtual machine is db-nodes. The following policies grant users the ability to create virtual machine console connections:

Allow group <group_name> to manage dbnode-console-connection in tenancy
Allow group <group_name> to read db-nodes in tenancy

You must install an SSH client and create SSH key pairs.

• Ports to Open for Control Plane Connectivity
  
• Install an SSH Client and a Command-line Shell (Microsoft Windows)
  
• Create SSH Key Pairs
  
• Sign in to a Virtual Machine From the Serial Console
  
• Connect Through Firewalls
  

channel 0: open failed: administratively prohibited: console access is limited to one connection at a time

1. Open the navigation menu. Under Oracle Database, click Exadata Database Service on Cloud@Customer.
2. Click the VM Cluster that you're interested in.
3. In the resulting VM Cluster Details page, click the name of the virtual machine that you're interested in.

   Under Resources, Console connection is selected by default.

4. Click Create serial console access.
5. In the resulting Create serial console access window, you have three options for adding the SSH key
   • Generate a key pair for me: You can have Oracle Cloud Infrastructure generate an SSH key pair to use. If you are using PowerShell or PuTTY to connect to the instance from a Windows client, you cannot use the generated SSH key pair without first converting it to a .ppk file.

   • Upload public key file: Browse to a public key file on your computer. If you followed the steps in Creating SSH Key Pairs in the Prerequisites section to create a key pair, use this option to navigate to the .pub file.
   • Paste public key: Paste the content of your public key file into the text box.
6. Click Create console connection.

   When the console connection has been created and is available, the state changes to Active.

After you create the console connection for the virtual machine, you can connect to the serial console using a Secure Shell (SSH) connection. When making an SSH connection to the serial console, you must use an RSA key. You can use the same SSH key for the serial console that was used when you launched the instance, or you can use a different SSH key.

When you are finished with the serial console and have terminated the SSH connection, you should delete the serial console connection. If you do not disconnect from the session, Oracle Cloud Infrastructure terminates the serial console session after 24 hours and you must reauthenticate to connect again.

• Validate Server Host Keys
  
• Connect from Mac OS X and Linux Operating Systems
  
• Connect from Windows Operating Systems
  

You can connect to the serial console quickly and easily using the Cloud Shell integration. Cloud Shell is a web browser-based terminal accessible from the Console. The Cloud Shell integration automatically creates the instance console connection and a temporary SSH key. The only prerequisite for connecting to the serial console from Cloud Shell is granting users the correct permissions. For an introductory walkthrough of using Cloud Shell, see Using Cloud Shell.

When you are finished with the serial console and have terminated the SSH connection, you should delete the serial console connection. If you do not disconnect from the session, Oracle Cloud Infrastructure terminates the serial console session after 24 hours and you must re-authenticate to connect again.

• To connect to the serial console using Cloud Shell
  

You can capture and display recent serial console data for a Virtual Machine. The data includes configuration messages that occur when the Virtual Machine boots, such as kernel and BIOS messages, and is useful for checking the status of the Virtual Machine or diagnosing and troubleshooting problems.

The console history captures up to a megabyte of the most recent serial console data for the specified Virtual Machine. Note that the raw console data, including multi-byte characters, is captured.

The console history is a point-in-time record. To troubleshoot a malfunctioning Virtual Machine using an interactive console connection, use a serial console connection.

• Managing Console History Data
  

After you are connected with an instance console connection, you can perform various tasks, such as:

• Edit system configuration files.
• Add or reset the SSH keys for the opc user.
• Reset the password for the opc user.

These tasks require you to boot into a Bash shell in maintenance mode.

• To boot into maintenance mode
  
• To edit the system configuration files
  
• To add or reset the SSH key for the opc user
  
• To reset the password for the opc user
  

• To exit the serial console connection
  
• To delete the serial console connection for a Virtual Machine