Known Issues
The following lists describe the known issues with Oracle Cloud Infrastructure.
Announcements
Currently, there are no known Announcements issues.
Anomaly Detection
Currently, there are no known issues with the Anomaly Detection service.
API Gateway
Details: The default Oracle Cloud Infrastructure Resolver resolves public URL endpoints (and URL endpoints with public hostnames) to IP addresses. Additionally, a subnet can be configured with a custom DNS server that resolves internal hostname (and private hostname) URL endpoints to IP addresses. However, API gateways you create with the API Gateway service do not inherit custom DNS servers from subnets. Instead API gateways use the default Oracle Cloud Infrastructure Resolver, which does not resolve internal/private hostname URL endpoints.
Due to this restriction, if you create an API gateway that has an internal/private hostname URL endpoint as the HTTP or HTTPS URL back end, calls to the API will fail because the hostname cannot be resolved to an IP address.
Workaround:
We're working on a resolution. In the meantime, if you want to create an API gateway that has an internal/private URL endpoint as the HTTP or HTTPS URL back end, you must specify the host's IP address in the URL rather than the hostname. In addition, if the back end is an HTTPS URL, you must also select the Disable SSL Verification option in the Console (or include isSSLVerifyDisabled: true
in the API deployment specification JSON file).
Direct link to this issue: API gateways do not inherit custom DNS servers from subnets
Application Performance Monitoring
Details: In Synthetic Monitoring, the Browser and Scripted Browser monitors might fail to run against applications that use frames.
Workaround: We are aware of the issue and working on a resolution. For Scripted Browser monitors, you can work around this issue by replacing index=<frame-index>
with either id=<id-of-frame>
or name=<name-of-frame>
in the .side
script.
For example, if this script is the original version:
{
"id": "23956f51-8812-40e6-ac91-1d608871ee4c",
"comment": "",
"command": "selectFrame",
"target": "index=0",
"targets": [
["index=0"]
],
"value": ""
}
The following script would be the modified version:
{
"id": "23956f51-8812-40e6-ac91-1d608871ee4c",
"comment": "",
"command": "selectFrame",
"target": "id=frame1",
"targets": [
["id=frame1"]
],
"value": ""
}
Direct link to this issue: Browser and Scripted Browser monitors might not run applications that use frames
Artifact Registry
For known issues with Artifact Registry, see Known Issues.
Audit
Currently, there are no known Audit issues.
Automated CEMLI Execution
For known issues with Automated CEMLI Execution, see Known Issues.
Bastion
For known issues with Bastion, see Known Issues.
Big Data
For known issues with Big Data Service, see Known Issues.
Billing and Cost Management
For known issues with Billing and Cost Management, see Known Issues for Billing and Cost Management.
Block Volume
Details: When you try to enable cross-region replication for a volume configured
to use a Vault encryption key, the following error message occurs: Edit Volume
Error: You cannot enable cross-region replication for volume
<volume_ID> as it uses a Vault encryption
key.
Workaround: We're working on a resolution. Cross-region replication is not supported for volumes encrypted with a customer-managed key. As a workaround to enable replication, unassign the Vault encryption key from the volume. In this scenario, the volume is encrypted with an Oracle-managed key.
Direct link to this issue: Cross-region replication not supported for volumes encrypted with customer-managed keys
Details: To achieve the optimal performance level for volumes configured for ultra high performance, the volume attachment must be multipath-enabled. Multipath-enabled attachments to VM instances are only supported for instances based on shapes with 16 or greater OCPUs.
If you have an instance with fewer than 16 OCPUs, you can resize it so that it has 16 or more OCPUs to support multipath-enabled attachments. This step will not work for instances where the original number of OCPUs was less than 8 and the volume attachment is paravirtualized. In this scenario, after the volume is detached and reattached, the volume attachment will still not be multipath-enabled even though the instance now supports multipath-enabled attachments.
Workaround: As a workaround, we recommend that you create a new instance based on a shape with 16 or more OCPUs, and then attach the volume to the new instance.
Direct link to this issue: Paravirtualized volume attachment not multipath-enabled after instance is resized
Details: When you attempt to attach the maximum number of block volumes to a smaller VM.Standard.A1.Flex instance, in some cases, the volumes might fail to attach. This happens because of limitations with the underlying physical host configuration.
Workaround: We're working on a resolution. As a workaround, we recommend that you increase the size of the VM by resizing the VM, and then try attaching the volumes again.
Direct link to this issue: Attaching the maximum number of block volumes to smaller VM.Standard.A1.Flex instances might fail
Details: The copy_boot_volume_backup_and_wait_for_state and copy_volume_backup_and_wait_for_state from the BlockStorage Client Composite operations throw a 404/NotAuthorizedOrNotFound when copying a backup from one region to another. For more information see: https://github.com/oracle/oci-python-sdk/issues/344.
Workaround: Instead of using the composite operations, use two different clients for this operation; one client in the Source Region to send the request for copying the backup from Region A to Region B, and a second client in Destination region to wait for the backup to become available. See example here: https://github.com/oracle/oci-python-sdk/blob/master/examples/copy_volume_backup_example.py
Direct link to this issue: Python SDK composite operations throw a 404 NotAuthorizedOrNotFound error even though operation is a success
Details: When you schedule volume and volume group backups using a backup policy that is enabled for cross-region copy for volumes that are encrypted using Vault service encryption keys, the encryption keys are not copied with the volume backup to the destination region. The volume backup copies in the destination region are instead encrypted using Oracle-provided keys.
Workaround: We're working on a resolution. As a workaround, you can manually copy volume backups and volume group backups across regions, either manually or using a script, and specify the key management key ID in the target region for the copy operation. For more information about manual cross region copy, see Copying a Volume Backup Between Regions.
Direct link to this issue: Vault encryption keys not copied to destination region for scheduled cross region backup copies
Details: When you attach a Windows boot volume as a data volume to another instance, when you try to connect to the volume using the steps described in Connecting to a Volume the volume fails to attach and you may encounter the following error:
Connect-IscsiTarget : The target has already been logged in via an iSCSI session.
Workaround: You need to append the following to the Connect-IscsiTarget
command copied from the Console:
-IsMultipathEnabled $True
Direct link to this issue: Attaching a Windows boot volume as a data volume to another instance fails
Details: When calling GetInstance, the bootVolumeSizeInGBs
attribute of InstanceSourceViaImageDetails is null.
Workaround: We're working on a resolution. To work around this issue, call GetBootVolume, and use the sizeInGBs
attribute of BootVolume.
Direct link to this issue: bootVolumeSizeInGBs attribute is null
Blockchain Platform
For known issues with Blockchain Platform, see Known Issues.
Certificates
For known issues with Certificates, see Known Issues.
Classic Migration
For known issues with Classic Migration Service, see Known Issues.
Cloud Guard
For known issues with Cloud Guard, see Known Issues for Cloud Guard.
Cloud Shell
Details: Due to some issues with one of its dependencies, the Go SDK feature which allows customers to automatically use new realms which might be unknown to the SDK is not functioning from within Cloud Shell.
can not create client, bad configuration: failed to get security token: failed to renew security token: failed to get security token: failed to call: Post "https://<endpoint>/v1/x509": dial tcp: lookup <endpoint> on 127.0.0.11:53: server misbehaving
panicked while retrying operation. Panic was: runtime error: invalid memory address or nil pointer dereference
Workaround: To resolve this issue, enable resolving regions using the instance metadata service for Go SDK. For more information, see: Adding Regions
Direct link to this issue: Go SDK cannot automatically find some regions while running in Cloud Shell
Compartment Quotas
For known issues with Compartment Quotas, see Known Issues for Compartment Quotas.
Compliance Documents
For known issues with Compliance Documents, see Known Issues for Compliance Documents.
Compute
For known issues with Compute, see Known Issues for Compute.
Console
Details: When you try to access the Console using Firefox, the Console page never loads in the browser. This problem is likely caused by a corrupted Firefox user profile.
Workaround: Create a new Firefox user profile as follows:
- Ensure that you are on the latest version of Firefox. If not, update to the latest version.
- Create a new user profile and remove your old user profile. See Mozilla Support for instructions to create and remove user profiles: https://support.mozilla.org/en-US/kb/profile-manager-create-and-remove-firefox-profiles.
- Open Firefox with the new profile.
Alternatively, you can use one of the other Supported Browsers.
Direct link to this issue: Bug in the Firefox browser can cause the Console not to load
Console Dashboards
For known issues with Console Dashboards, see Known Issues for Console Dashboards.
Container Engine for Kubernetes
Details: The properties of new worker nodes starting in a node pool do not reflect the latest changes to the node pool's properties. The likely cause is use of the deprecated quantityPerSubnet and subnetIds attributes when using the UpdateNodePoolDetails API operation to update node pool properties.
- Start using the nodeConfigDetails attribute when using the UpdateNodePoolDetails API operation. First, scale the node pool to 0 using quantityPerSubnet. Then stop using the subnetIds and quantityPerSubnet attributes, and use the nodeConfigDetails attribute instead.
- Contact Oracle Support to restart the back-end component responsible for synchronization (the tenant-agent component).
Direct link to this issue: Worker node properties out-of-sync with updated node pool properties
Details: When you launch the Kubernetes Dashboard, in some situations you might encounter "net/http: TLS handshake timeout" and "connection reset by peer" error messages in your web browser. This issue has only been observed in newly created clusters running Kubernetes version 1.11. For details about a related Kubernetes issue, see https://github.com/kubernetes/dashboard/issues/3038.
Workaround:
-
In a terminal window, enter:
$ kubectl -n kube-system port-forward svc/kubernetes-dashboard 8443:443
- In your web browser, go to
https://localhost:8443
Direct link to this issue: Unable to launch Kubernetes Dashboard
Details: When you use a Kubeconfig token version 2.0.0 to access Helm/Tiller versions prior to version 2.11, you will receive one of the following errors:
Error: Unauthorized
-
Error: could not get Kubernetes client: exec plugin: invalid apiVersion "client.authentication.k8s.io/v1beta1"
Workaround: Upgrade Helm/Tiller as follows:
-
In a terminal window, download a Kubeconfig token version 1.0.0 by entering the following command:
$ oci ce cluster create-kubeconfig --token-version=1.0.0 --cluster-id=<cluster_ocid>
-
Identify the region key to use to specify the Oracle Cloud Infrastructure Registry registry in the cluster's region (see Availability by Region). For example, if the cluster is in US East (Ashburn),
iad
is the region key to use to specify the registry in that region. -
Upgrade Tiller by entering the following command:
$ helm init --upgrade -i <region-key>.ocir.io/odx-oke/oke-public/tiller:v2.14.3
where
<region-key>
is the key that you identified in the previous step. -
In a browser, navigate to https://helm.sh/docs/using_helm/#installing-the-helm-client and follow the instructions to download and install the Helm client binary.
-
Having upgraded Helm/Tiller, download a Kubeconfig token version 2.0.0 by entering the following command:
$ oci ce cluster create-kubeconfig --token-version=2.0.0 --cluster-id=<cluster_ocid>
Direct link to this issue: Unable to access in-cluster Helm
Details: The Container Engine for Kubernetes 1.8.0 release included a security improvement to improve cipher strength on the kubelet running on customer worker nodes. New worker nodes created between August 20, 2019 and September 16, 2019 include this configuration. The new set of ciphers does not allow connections to the kubelet via http/2. This restriction impacts the Metric Server, and also the Horizontal Pod Autoscaler which depends on the Metrics Server.
Workaround:
For each existing worker node in turn:
-
Prevent new pods from starting and delete existing pods on the worker node by entering
kubectl drain <node_name>
. For more information:- about using kubectl, see Accessing a Cluster Using Kubectl
- about the drain command, see drain in the Kubernetes documentation
Recommended: Leverage pod disruption budgets as appropriate for your application to ensure that there's a sufficient number of replica pods running throughout the drain operation.
- Delete the worker node (for example, by terminating it in the Console).
- Wait for a replacement worker node to start.
The replacement worker nodes include include new settings to enable communication with the kubelet.
Direct link to this issue: Some Kubernetes features (for example, the Metrics Server) cannot communicate with the kubelet via http/2
Details: When a new pod starts on a worker node in a cluster, in some situations the pod fails to mount all volumes attached to the node due to timeouts and you see a message similar to the following:
Unable to mount volumes for pod "<pod_name>(<pod_uid>)": timeout expired waiting for volumes to attach or mount for pod "<namespace>"/"<pod_name>". list of unmounted volumes=[<failed_volume>]. list of unattached volumes=[<… list of volumes >]
One possible cause identified for this issue is if the pod spec includes an fsGroup
field in the securityContext
field. If the container is running on a worker node as a non-root user, setting the fsGroup
field in the securityContext
can cause timeouts due to the number of files to which Kubernetes must make ownership changes (see https://github.com/kubernetes/kubernetes/issues/67014).
If the pod spec does not include an fsgroup
field in the securityContext
, the cause is unknown.
Workarounds:
If the pod spec includes the fsgroup
field in the securityContext
and the container is running a non-root user, consider the following workarounds:
- Remove the
fsgroup
field from thesecurityContext
. - Use the
supplementalGroups
field in thesecurityContext
(instead offsgroup
), and setsupplementalGroups
to the volume identifier. - Change the pod spec so that the container runs as root.
If the pod spec does not include the fsgroup
field in the securityContext
, or if the container is already running as root, you have to restart or replace the worker node. For example, by stopping and starting the instance, by rebooting the instance, or by terminating the instance so that a new instance is started. Follow the instructions in Stopping and Starting an Instance or Terminating an Instance as appropriate to use the Console or the API. Alternatively you can use CLI commands, such as the following example to terminate an instance:
$ INSTANCE_OCID=$(kubectl get node <name> -ojsonpath='{.spec.providerID}')
$ oci compute instance terminate --instance-id $INSTANCE_OCID
where <name>
is the worker node name, derived from the Private IP Address property of the instance (for example, 10.0.10.5
).
Direct link to this issue: Kubernetes pods fail to mount volumes due to timeouts
Details: When using the OS Management service to manage operating system updates and patches on Oracle Cloud Infrastructure instances, there are some situations in which cluster node pools created by Container Engine for Kubernetes fail to come online.
Workaround:
There are two possible workarounds:
- Workaround 1: If you want to use OS Management to manage Oracle Cloud Infrastructure instances, enable Oracle Enterprise Linux in OS Management. See Managing Software Sources.
- Workaround 2: If you don't want to use OS Management to manage Oracle Cloud Infrastructure instances, make sure there are no policies that allow OS Management to run. Specifically, remove the policy that grants a dynamic group of instances access to the OS Management service. See Setting Up Policies for OS Management.
Direct link to this issue: OS Management causes Kubernetes cluster node pools to fail
Details: If node pools have master nodes running Kubernetes version 1.19 (or later) and worker nodes running Kubernetes version 1.18 (or earlier), mounting block volumes attached to the cluster using the FlexVolume volume plugin might not work as expected. For example, you might see:
- A
FailedMount
warning message in the events of a pod running on a worker node, even though the block volume has been attached successfully. - A
Volume not attached according to node status for volume
error message in the logs of the kubelet running on a worker node.
Workaround:
- If there isn't already a node pool in the cluster with worker nodes running Kubernetes version 1.19 (or later), add such a node pool now.
- Remove the affected worker node that is running Kubernetes version 1.18 (or earlier), as follows:
- Prevent new pods from starting and delete existing pods on the affected worker node by entering
kubectl drain <node_name>
. For more information:- about using kubectl, see Accessing a Cluster Using Kubectl
- about the drain command, see drain in the Kubernetes documentation
- Delete the affected worker node (for example, by terminating it in the Console).
- Prevent new pods from starting and delete existing pods on the affected worker node by entering
Direct link to this issue: Volume mount issues in node pools with master nodes running Kubernetes version 1.19 (or later) and worker nodes running Kubernetes version 1.18 (or earlier)
localhost
doesn't route correctly. For example:
/ # nslookup www.oracle.com
;; reply from unexpected source: 10.244.0.167#53, expected 10.96.5.5#53
;; reply from unexpected source: 10.244.0.167#53, expected 10.96.5.5#53
;; reply from unexpected source: 10.244.0.167#53, expected 10.96.5.5#53
;; connection timed out; no servers could be reached
sudo /usr/sbin/lsmod | grep br_netfilter
If no results are returned, then the Bridge Netfilter kernel module is not enabled. The Bridge Netfilter kernel module is required to masquerade VxLAN traffic for Kubernetes pods.
sudo modprobe br_netfilter
sudo sh -c 'echo "br_netfilter" > /etc/modules-load.d/br_netfilter.conf'
Direct link to this issue: Issues resolving with DNS (nslookup, dig, or curl)
Details: When using VCN-native pod networking, the source client IP address of inbound requests to a pod might not be preserved as expected. Instead, inbound requests received via a Kubernetes service of type LoadBalancer that has externalTrafficPolicy: Local
set in the manifest file might be shown as originating from the worker node's IP address.
Workaround: For inbound TCP requests received via a Kubernetes service of type LoadBalancer that has the oci.oraclecloud.com/load-balancer-type: "lb"
annotation in the manifest file, obtain the source client IP address from the X-Forwarded-For
or X-Real-IP
header.
Direct link to this issue: Source client IP is not preserved for traffic through a LoadBalancer Service using externalTrafficPolicy: Local
Details: When using VCN-native pod networking, pods might be unable to communicate over the network if you have specified a bare metal shape for worker nodes in one or more of the node pools in the cluster.
Workaround: Specify a VM shape for worker nodes in every node pool in the cluster when using VCN-native pod networking.
Direct link to this issue: Pod network connectivity issues on bare metal instances
Details: When using VCN-native pod networking, the maximum number of pods per worker node in a node pool might be limited to 31, regardless of the number of OCPUs you specify for the flexible shape you have selected for the node pool.
Workaround: If you want more than 31 pods per worker node in a node pool, select a different shape for worker nodes in the node pool.
Direct link to this issue: Incorrect maximum pods per node limit for flexible shapes
Details: When using VCN-native pod networking, pods running on worker nodes connected to a pod subnet with a CIDR block outside the first CIDR block specified for the VCN might be unable to communicate with Kubernetes services.
Workaround: Create pod subnets with CIDR blocks within the first CIDR block specified for the VCN.
Direct link to this issue: Pod network connectivity issues on VCNs with added CIDR blocks
Details: When using the Node Doctor script to troubleshoot node issues, the script might display an exception error message similar to the following:
FileNotFoundError: [Errno 2] No such file or directory: ’/home/opc/vendor/pip…
The Node Doctor script will probably continue to run and, having displayed the message Generating node doctor bundle
, produce troubleshooting output.
Workaround: We are aware of the issue and working on a resolution. In the meantime, if the Node Doctor script displays the message Generating node doctor bundle
, note that the troubleshooting output is still valid.
If you do not want the Node Doctor script to display the FileNotFoundError: [Errno 2]...
exception error message, update the Node Doctor script by entering:
node-doctor.sh --update
For more information about the Node Doctor script and how to update it, see Node Troubleshooting Using the Node Doctor Script.
Direct link to this issue: Node Doctor script displays FileNotFoundError: [Errno 2] exception
Details: If a cluster uses VCN-native pod networking and has both a workload pod and the CoreDNS pod running on the same worker node, DNS resolution sometimes fails because traffic is incorrectly NATed.
Workaround: We are aware of the issue and working on a resolution.
In the meantime, if you encounter this issue, run the kube-proxy daemonset on worker nodes in ipvs mode (rather than in the default iptables mode).
To move workloads to worker nodes running the kube-proxy daemonset in ipvs mode, do one of the following
- Create a new nodepool using the CLI. Specify that the kube-proxy daemonset is to run in ipvs mode on new worker nodes in the new node pool, using
--node-metadata '{ "oke-kubeproxy-proxy-mode": "ipvs"}'
. For example:oci ce node-pool create \ --cluster-id <cluster-ocid> \ --name <node-pool-name> \ ... --node-metadata '{ "oke-kubeproxy-proxy-mode": "ipvs"}' \ ...
Having created the new node pool, move workloads from the existing node pool to the new node pool. Drain existing worker nodes in the original node pool to prevent new pods starting and to delete existing pods. Then delete the original node pool. When new worker nodes start in the new node pool, they run the kube-proxy daemonset in ipvs mode as you specified. For more information, see Performing an Out-of-Place Worker Node Update by Replacing an Existing Node Pool with a New Node Pool.
- Update an existing node pool using the CLI. Specify that the kube-proxy daemonset is to run in ipvs mode on new worker nodes created in the existing node pool, using
--node-metadata '{ "oke-kubeproxy-proxy-mode": "ipvs"}'
. For example:oci ce node-pool update \ --node-pool-id <node-pool-ocid> \ ... --node-metadata '{ "oke-kubeproxy-proxy-mode": "ipvs"}' \ ...
Having updated the node pool, delete each existing worker node in turn. Select appropriate cordon and drain options to prevent new pods starting and to delete existing pods. Then start a new worker node to take the place of each worker node you delete. When new worker nodes start in the existing node pool, they run the kube-proxy daemonset in ipvs mode as you specified. For more information, see Performing an In-Place Worker Node Update by Updating an Existing Node Pool.
Direct link to this issue: DNS resolution sometimes fails in clusters using VCN-native pod networking
Details: If a cluster uses VCN-native pod networking and has worker nodes running Oracle Linux 8 (OL8), pods sometimes fail to start on one of the worker nodes. The issue has the following characteristics:
- The worker node is running an OL8 image.
- Host-network related pods run as expected on the worker node, but all other pods fail to start.
- The crictl logs contain the message
Adding address to IPVLAN parent device
(indicating that an IP address is being attached to the worker node's secondary VNIC), followed by the error messageError adding secondary VNIC IP
. - Running the Linux
ip address
command on the worker node shows that one (or more) secondary VNICs does not have an attached IP address. - All (or most) other worker nodes are operating as expected.
A likely cause identified for this issue is related to the NetworkManager, which manages network devices and connections. In some cases, the NetworkManager detaches the IP address attached to one or more of the worker node's secondary VNICs, causing the OCI VCN-Native Pod Networking CNI plugin to fail.
Workaround: We are aware of the issue and working on a resolution.
In the meantime, if you encounter this issue, delete the affected worker node (see Deleting Worker Nodes). Assuming you delete the worker node without scaling down the node pool, the deleted worker node is replaced and the replacement worker node operates as expected.
Direct link to this issue: Pods sometimes fail to start on a worker node running Oracle Linux 8, in clusters using VCN-native pod networking
Container Instances
For known issues with Container Instances, see Known Issues for Container Instances.
Data Catalog
For known issues with Data Catalog, see Known Issues.
Data Flow
For known issues with Data Flow, see Known Issues.
Data Integration
For known issues with Data Integration, see Known Issues.
Data Labeling
For known issues with Data Labeling, see Known Issues.
Data Science
Currently, there are no known issues with the Data Science service.
Data Transfer
Currently, there are no known Data Transfer issues.
Database
Details: Existing PDBs do not appear in a newly created database and it may take up to a few hours before they appear in the Console. This includes the default PDB for a new database and existing PDBs for cloned or restored databases. In case of an in-place restore to an older version, the PDB list is updated similarly and may have some delay.
Workaround: None
Direct link to this issue: Existing PDBs in a new database
Details: Creating and cloning a PDB in the primary database is not allowed via console or the API.
Workaround: You can use sqlplus to create or clone PDBs in the Primary database and they will be synced later in OCI console.
Direct link to this issue: PDB in existing Data Guard configuration
Details: Using the Database Service API to migrate a file-based TDE wallet to a customer-managed key-based TDE wallet on Oracle Database 12c release 1 (12.1.0.2) fails with the following error:
[FATAL] [DBAAS-11014] - Required patches (30128047) are not present in the Oracle home <ORACLE_HOME>
ACTION: Apply the required patches (30128047) and re-try the operation
--skip_patch_check
true
flag to skip the validation of the patch for bug 30128047. Ensure that
you have applied the patch for bug 31527103 in the Oracle home and then run the
following dbaascli
command:nohup /var/opt/oracle/dbaascli/dbaascli tde file_to_hsm --dbname <database_name> --kms_key_ocid <kms_key_ocid> --skip_patch_check true &
In the preceding command,
<
kms_key_ocid>
refers to the
OCID of the customer-managed key you are using.
Details: Using the Database Service API to migrate a customer-managed key-based TDE wallet to a file-based TDE wallet on Oracle Database 12c release 1 (12.1.0.2) fails with the following error:
[FATAL] [DBAAS-11014] - Required patches (30128047) are not present in the Oracle home <ORACLE_HOME>
ACTION: Apply the required patches (30128047) and re-try the operation
--skip_patch_check
true
flag to skip the validation of the patch for bug 30128047. Ensure that
you have applied the patch for bug 29667994 in the Oracle home and then run following
dbaascli
command:nohup /var/opt/oracle/dbaascli/dbaascli tde hsm_to_file --dbname <database_name> --skip_patch_check true &
Details: Using the Database Service API to migrate a file-based TDE wallet to customer-managed key-based TDE wallet on Oracle Database 12c release 2 (12.2.0.1) fails with the following error:
[FATAL] [DBAAS-11014] - Required patches (30128047) are not present in the Oracle home <ORACLE_HOME>
ACTION: Apply the required patches (30128047) and re-try the operation
Workaround: Migrate a file-based TDE wallet to a customer-managed key-based TDE wallet, as follows:
- Determine whether the database has encrypted UNDO or TEMP tablespaces in any of the
Autonomous Databases or in CDB$ROOT, as follows:Run the following query from CDB$ROOT, to list all encrypted tablespaces contained within all Autonomous Databases:
SQL> select tstab.con_id, tstab.name from v$tablespace tstab, v$encrypted_tablespaces enctstab where tstab.ts# = enctstab.ts# and encryptedts = 'YES';
In then NAME column of the result of the query, search for the names of UNDO and TEMP tablespaces. If there are encrypted UNDO or TEMP tablespaces, then proceed to the next step.
- Unencrypt UNDO or TEMP tablespaces, as follows:
If an UNDO tablespace is encrypted
Unencrypt existing UNDO tablespaces, as follows:SQL> alter tablespace <undo_tablespace_name> encryption online decrypt;
Repeat this procedure for all encrypted UNDO tablespaces.
If a TEMP tablespace is encrypted- Check the default TEMP tablespace, as
follows:
SQL> select property_value from database_properties where property_name = 'DEFAULT_TEMP_TABLESPACE';
If the default TEMP tablespace is not encrypted but other TEMP tablespaces are encrypted, then drop the other TEMP tablespaces, as follows:SQL> drop tablespace <temp_tablespace_name>;
Skip the remainder of the steps in this procedure.
If the default TEMP tablespace is encrypted, then proceed with the remaining steps to create and set an unencrypted default TEMP tablespace.
- Set the
encrypt_new_tablespaces
parameter to DDL, as follows:SQL> alter system set "encrypt_new_tablespaces" = DDL scope = memory;
- Create a TEMP tablespace with the specifications of the current TEMP
tablespace, as
follows:
SQL> create temporary tablespace <temp_tablespace_name> TEMPFILE size 5000M;
- Set the new TEMP tablespace as the default TEMP tablespace for the
database, as
follows:
SQL> alter database default temporary tablespace <temp_tablespace_name>;
- Drop existing TEMP tablespaces, as
follows:
SQL> drop tablespace <temp_tablespace_name>;
Repeat this procedure for all encrypted TEMP tablespaces.
The database is now running with default UNDO and TEMP tablespaces that are not encrypted and any older TEMP and UNDO tablespaces are also decrypted.
Setencrypt_new_tablespaces
to its original value, as follows:SQL> alter system set "encrypt_new_tablespaces" = cloud_only;
Proceed with keystore migration to customer-managed keys.
- Check the default TEMP tablespace, as
follows:
- Once you confirm that there are no UNDO or TEMP tablespaces encrypted in any of the
pluggable databases or in CDB$ROOT, use the DBAASCLI utility with the
--skip_patch_check true
flag to skip the validation of the patch for bug 30128047. Ensure that you have applied the patch for bug 31527103 in the Oracle home and then run followingdbaascli
command:nohup /var/opt/oracle/dbaascli/dbaascli tde file_to_hsm --dbname <database_name> --kms_key_ocid <kms_key_ocid> --skip_patch_check true &
In the preceding command,
<
kms_key_ocid>
refers to the OCID of the customer-managed key you are using.
Details: Using the Database Service API to migrate a customer-managed key-based TDE wallet to a file-based TDE wallet on Oracle Database 12c release 2 (12.2.0.1) fails with the following error:
[FATAL] [DBAAS-11014] - Required patches (30128047) are not present in the Oracle home <ORACLE_HOME>
ACTION: Apply the required patches (30128047) and re-try the operation
Workaround: Migrate a customer-managed key-based TDE wallet to a file-based TDE wallet, as follows:
- Determine whether the database has encrypted UNDO or TEMP tablespaces in any of the
Autonomous Databases or in CDB$ROOT, as follows:Run the following query from CDB$ROOT, to list all encrypted tablespaces contained within all Autonomous Databases:
SQL> select tstab.con_id, tstab.name from v$tablespace tstab, v$encrypted_tablespaces enctstab where tstab.ts# = enctstab.ts# and encryptedts = 'YES';
In then NAME column of the result of the query, search for the names of UNDO and TEMP tablespaces. If there are encrypted UNDO or TEMP tablespaces, then proceed to the next step.
- Unencrypt UNDO or TEMP tablespaces, as follows:
If an UNDO tablespace is encrypted
Unencrypt existing UNDO tablespaces, as follows:SQL> alter tablespace <undo_tablespace_name> encryption online decrypt;
Repeat this procedure for all encrypted UNDO tablespaces.
If a TEMP tablespace is encrypted- Check the default TEMP tablespace, as
follows:
SQL> select property_value from database_properties where property_name = 'DEFAULT_TEMP_TABLESPACE';
If the default TEMP tablespace is not encrypted but other TEMP tablespaces are encrypted, then drop the other TEMP tablespaces, as follows:SQL> drop tablespace <temp_tablespace_name>;
Skip the remainder of the steps in this procedure.
If the default TEMP tablespace is encrypted, then proceed with the remaining steps to create and set an unencrypted default TEMP tablespace.
- Set the
encrypt_new_tablespaces
parameter to DDL, as follows:SQL> alter system set "encrypt_new_tablespaces" = DDL scope = memory;
- Create a TEMP tablespace with the specifications of the current TEMP
tablespace, as
follows:
SQL> create temporary tablespace <temp_tablespace_name> TEMPFILE size 5000M;
- Set the new TEMP tablespace as the default TEMP tablespace for the
database, as
follows:
SQL> alter database default temporary tablespace <temp_tablespace_name>;
- Drop existing TEMP tablespaces, as
follows:
SQL> drop tablespace <temp_tablespace_name>;
Repeat this procedure for all encrypted TEMP tablespaces.
The database is now running with default UNDO and TEMP tablespaces that are not encrypted and any older TEMP and UNDO tablespaces are also decrypted.
Setencrypt_new_tablespaces
to its original value, as follows:SQL> alter system set "encrypt_new_tablespaces" = cloud_only;
Proceed with keystore migration to customer-managed keys.
- Check the default TEMP tablespace, as
follows:
- Once you confirm that there are no UNDO or TEMP tablespaces encrypted in any of the
pluggable databases or in CDB$ROOT, use the DBAASCLI utility with the
--skip_patch_check true
flag to skip the validation of the patch for bug 30128047. Ensure that you have applied the patch for bug 29667994 in the Oracle home and then run followingdbaascli
command:nohup /var/opt/oracle/dbaascli/dbaascli tde file_to_hsm --dbname <database_name> --kms_key_ocid <kms_key_ocid> --skip_patch_check true &
In the preceding command,
<
kms_key_ocid>
refers to the OCID of the customer-managed key you are using.
Details: When you change the license type of your Database or DB system from BYOL to license included, or the other way around, you are billed for both types of licenses for the first hour. After that, you are billed according to your updated license type.
Workaround: We're working on a resolution.
Direct link to this issue: Billing issue when changing license type
Details: If you configure your VCN with a service gateway, the private subnet blocks access to the YUM repositories needed to update the OS. This issue affects all types of DB systems.
Workaround: This issue is now resolved. Here is the workaround that was recommended before the issue's resolution:
The service gateway enables access to the Oracle YUM repos if you use the Available Service CIDR Labels called All <region> Services in Oracle Services Network. However, you still might have issues accessing the YUM services through the service gateway. There's a solution to the issue. For details, see Access issues for instances to Oracle yum services through service gateway.
Direct link to this issue: Service gateway does not currently support OS updates
Bare Metal and Virtual Machine DB Systems Only
Details: Unmanaged backups to Object Storage using the database CLI (dbcli) or RMAN fail with the following errors:
-> Oracle Error Codes found:
-> ORA-19554: error allocating device, device type: SBT_TAPE, device name:
-> ORA-19511: non RMAN, but media manager or vendor specific failure, error text:
-> KBHS-00712: ORA-29024 received from local HTTP service
-> ORA-27023: skgfqsbi: media manager protocol error
In response to policies implemented by two common web browsers regarding Symantec certificates, Oracle recently changed the certificate authority used for Oracle Cloud Infrastructure. The resulting change in SSL certificates can cause backups to Object Storage to fail if the Oracle Database Cloud Backup Module still points to the old certificate.
Workaround for dbcli: Check the log files for the errors listed and, if found, update the backup module.
Review the RMAN backup log files for the errors listed above:
-
Determine the ID of the failed backup job.
dbcli list-jobs
In this example output, the failed backup job ID is "f59d8470-6c37-49e4-a372-4788c984ea59".
root@<node name> ~]# dbcli list-jobs ID Description Created Status ---------------------------------------- --------------------------------------------------------------------------- ----------------------------------- ---------- cbe852de-c0f3-4807-85e8-7523647ec78c Authentication key update for DCS_ADMIN March 30, 2018 4:10:21 AM UTC Success db83fdc4-5245-4307-88a7-178f8a0efa48 Provisioning service creation March 30, 2018 4:12:01 AM UTC Success c1511a7a-3c2e-4e42-9520-f156b1b4cf0e SSH keys update March 30, 2018 4:48:24 AM UTC Success 22adf146-9779-4a2c-8682-7fd04d7520b2 SSH key delete March 30, 2018 4:50:02 AM UTC Success 6f2be750-9823-4ed5-b5ff-8e49f136dd22 create object store:bV0wqIaoLA4xLT4dGjOu March 30, 2018 5:33:38 AM UTC Success 0716f464-1a10-40df-a303-cadee0302b1b create backup config:bV0wqIaoLA4xLT4dGjOu_BC March 30, 2018 5:33:49 AM UTC Success e08b21c3-cd09-4e3a-944c-d1da96cb21d8 update database : hfdb1 March 30, 2018 5:34:04 AM UTC Success 1c3d7c58-79c3-4039-8f48-787057ce7c6e Create Longterm Backup with TAG-DBTLongterm<identity number> for Db:<dbname> March 30, 2018 5:37:11 AM UTC Success f59d8470-6c37-49e4-a372-4788c984ea59 Create Longterm Backup with TAG-DBTLongterm<identity number> for Db:<dbname> March 30, 2018 5:43:45 AM UTC Failure
-
Use the ID of the failed job to obtain the location of the log file to review.
dbcli describe-job -i <failed_job_ID>
Relevant output from the
describe-job
command should look like this:Message: DCS-10001:Internal error encountered: Failed to run Rman statement. Refer log in Node <node_name>: /opt/oracle/dcs/log/<node_name>/rman/bkup/<db_unique_name>/rman_backup/<date>/rman_backup_<date>.log.
Update the Oracle Database Cloud Backup Module:
-
Determine the Swift object store ID and user the database is using for backups.
-
Run the
dbcli list-databases
command to determine the ID of the database. -
Use the database ID to determine the backup configuration ID (
backupConfigId
).dbcli list-databases dbcli describe-database -i <database_ID> -j
-
Using the backup configuration ID you noted from the previous step, determine the object store ID (
objectStoreId
).dbcli list-backupconfigs dbcli describe-backupconfig –i <backupconfig_ID> -j
-
Using the object store ID you noted from the previous step, determine the object store user (
userName
).dbcli list-objectstoreswifts dbcli describe-objectstoreswift –i <objectstore_ID> -j
-
-
Using the object store credentials you obtained from step 1, update the backup module.
dbcli update-objectstoreswift –i <objectstore_ID> -p –u <user_name>
Workaround for RMAN: Check the RMAN log files for the error messages listed. If found, log on to the host as the oracle user, and use your Swift credentials to reinstall the backup module.
java -jar <opc_install.jar_path> -opcId '<swift_user_ID>' -opcPass '<auth_token>' -container <objectstore_container> -walletDir <wallet_directory> -configfile <config_file> -host https://swiftobjectstorage.<region_name>.oraclecloud.com/v1/<object_storage_namespace> -import-all-trustcerts
For a multi-node DB system, perform the workaround on all nodes in the cluster.
See Oracle Database Cloud Backup Module documentation for details on using this command.
Direct link to this issue: Backing up to Object Storage using dbcli or RMAN fails due to certificate change
Details: The SDKs released on October 18, 2018 introduce code-breaking changes to the database size and the database edition attributes in the database backup APIs.
Workaround: Refer to the following language-specific documentation for more details about the breaking changes, and update your existing code as applicable:
Language | Version | Documentation |
---|---|---|
Java | 1.2.49 | https://github.com/oracle/oci-java-sdk/releases |
Python | 2.0.6 | https://github.com/oracle/oci-python-sdk/releases |
Ruby | 2.3.9 | https://github.com/oracle/oci-ruby-sdk/releases |
Go | 2.7.0 | https://github.com/oracle/oci-go-sdk/releases |
Direct link to this issue: Breaking changes in Database service SDKs
Details: Backup and restore operations might not work in your DB system when you use the Console or the API.
Workaround: Install the Oracle Database Cloud Backup Module, and then contact Oracle Support Services for further instructions.
To install the Oracle Database Cloud Backup Module:
-
SSH to the DB system, and log in as opc.
ssh -i <SSH_key> opc@<DB_system_IP address> login as: opc
Alternatively, you can use opc@<DB_system_hostname> to log in.
- Download the Oracle Database Cloud Backup Module from http://www.oracle.com/technetwork/database/availability/oracle-cloud-backup-2162729.html.
- Extract the contents of opc_installer.zip to a target directory, for example, /home/opc.
- In your tenancy, create a temporary user, and grant them privileges to access the tenancy's Object Storage.
- For this temporary user, create an Working with Auth Tokens and note down the password.
-
Verify that credentials work by running the following curl command:
curl -v -X HEAD -u <user_id>:'<auth_token>' https://swiftobjectstorage.<region_name>.oraclecloud.com/v1/<object_storage_namespace>
See https://cloud.oracle.com/infrastructure/storage/object-storage/faq for the correct region to use.
The command should return either the HTTP 200 or the HTTP 204 No Content success status response code. Any other status code indicates a problem connecting to Object Storage.
-
Run the following command:
java -jar opc_install.jar -opcid <user_id> -opcPass '<auth_token>' -libDir <target_dir> -walletDir <target_dir> -host https://swiftobjectstorage.<region_name>.oraclecloud.com/v1/<object_storage_namespace> -configFile config.txt
Note that <target_dir> is the directory to which you extracted opc_installer.zip in step 3.
This command might take a few minutes to complete because it downloads libopc.so and other files. Once the command completes, you should see several files (including libopc.so) in your target directory.
-
Change directory to your target directory, and copy the lipopc.so and opc_install.jar files into the /opt/oracle/oak/pkgrepos/oss/odbcs directory.
cp libopc.so /opt/oracle/oak/pkgrepos/oss/odbcs
cp opc_install.jar /opt/oracle/oak/pkgrepos/oss/odbcs
(You might have to use sudo with the copy commands to run them as root.)
-
Run the following command to check whether the directory indicated exists:
ls /opt/oracle/dcs/commonstore/pkgrepos/oss/odbcs
If this directory exists, perform the following steps:
- Back up the files in the
/opt/oracle/dcs/commonstore/pkgrepos/oss/odbcs
directory. -
Run these two commands to replace the existing
libopc.so
andopc_install.jar
files in that directory:cp libopc.so /opt/oracle/dcs/commonstore/pkgrepos/oss/odbcs cp opc_install.jar /opt/oracle/dcs/commonstore/pkgrepos/oss/odbcs
- Back up the files in the
-
Verify the version of
opc_install.jar
.java -jar /opt/oracle/oak/pkgrepos/oss/odbcs/opc_install.jar |grep -i build
If /opt/oracle/dcs/commonstore/pkgrepos/oss/odbcs exists, also run the following command:
java -jar /opt/oracle/dcs/commonstore/pkgrepos/oss/odbcs/opc_install.jar |grep -i build
Both commands should return the following output:
Oracle Database Cloud Backup Module Install Tool, build MAIN_2017-08-16.
- (Optional) Delete the temporary user and the target directory you used to install the backup module.
After you complete the procedure, contact Oracle Support or your tenant administrator for further instructions. You must provide the OCID of the DB system for which you would like to enable backups.
Direct link to this issue: Unable to use Managed Backups in your DB System
Details: Memory limitations of host machines running the VM.Standard1.1 shape can cause failures for automatic database backup jobs managed by Oracle Cloud Infrastructure (jobs managed by using either the Console or the API). You can change the systems' memory parameters to resolve this issue.
Workaround: Change the systems' memory parameters as follows:
-
Switch to the oracle user in the operating system.
[opc@hostname ~]$ sudo su - oracle
-
Set the environment variable to login to the database instance. For example:
[oracle@hostname ~]$ . oraenv ORACLE_SID = [oracle] ? orcl
-
Start SQL*Plus.
[oracle@hostname ~]$ sqlplus / as sysdba
-
Change the initial memory parameters as follows:
SQL> ALTER SYSTEM SET SGA_TARGET = 1228M scope=spfile; SQL> ALTER SYSTEM SET PGA_AGGREGATE_TARGET = 1228M; SQL> ALTER SYSTEM SET PGA_AGGREGATE_LIMIT = 2457M; SQL> exit
-
Restart the database instance.
[oracle@hostname ~]$ srvctl stop database -d db_unique_name -o immediate [oracle@hostname ~]$ srvctl start database -d db_unique_name -o open
Direct link to this issue: Managed Automatic Backups fail on the VM.Standard1.1 shape due to a process crash
Details: On High Performance and Extreme Performance DB systems, Data Pump utility operations that use compression and/or parallelism might fail and return the error ORA-00439: feature not enabled. This issue affects database versions 12.1.0.2.161018 and 12.1.0.2.170117.
Workaround: Apply patch 25579568 or 25891266 to Oracle Database homes for database versions 12.1.0.2.161018 or 12.1.0.2.170117, respectively. Alternatively, use the Console to apply the April 2017 patch to the DB system and database home.
Determining the Version of a Database in a Database Home
To determine the version of a database in a database home, run either $ORACLE_HOME/OPatch/opatch lspatches
as the oracle user or dbcli list-dbhomes
as the root user.
Direct link to this issue: Oracle Data Pump operations return "ORA-00439: feature not enabled"
Details: You might get a "Secure Connection Failed" error message when you try to connect to the EM Express console from your 1-node DB system because the correct permissions were not applied automatically.
Workaround: Add read permissions for the asmadmin group on the wallet directory of the DB system, and then retry the connection:
-
SSH to the DB system host, log in as opc, sudo to the grid user.
[opc@dbsysHost ~]$ sudo su - grid [grid@dbsysHost ~]$ . oraenv ORACLE_SID = [+ASM1] ? The Oracle base has been set to /u01/app/grid
-
Get the location of the wallet directory, shown in red below in the command output.
[grid@dbsysHost ~]$ lsnrctl status | grep xdb_wallet (DESCRIPTION=(ADDRESS=(PROTOCOL=tcps)(HOST=dbsysHost.sub04061528182.dbsysapril6.oraclevcn.com)(PORT=5500))(Security=(my_wallet_directory=/u01/app/oracle/admin/dbsys12_phx3wm/xdb_wallet))(Presentation=HTTP)(Session=RAW))
-
Return to the opc user, switch to the oracle user, and change to the wallet directory.
[opc@dbsysHost ~]$ sudo su - oracle [oracle@dbsysHost ~]$ cd /u01/app/oracle/admin/dbsys12_phx3wm/xdb_wallet
-
List the directory contents and note the permissions.
[oracle@dbsysHost xdb_wallet]$ ls -ltr total 8 -rw------- 1 oracle asmadmin 3881 Apr 6 16:32 ewallet.p12 -rw------- 1 oracle asmadmin 3926 Apr 6 16:32 cwallet.sso
-
Change the permissions:
[oracle@dbsysHost xdb_wallet]$ chmod 640 /u01/app/oracle/admin/dbsys12_phx3wm/xdb_wallet/*
-
Verify that read permissions were added.
[oracle@dbsysHost xdb_wallet]$ ls -ltr total 8 -rw-r----- 1 oracle asmadmin 3881 Apr 6 16:32 ewallet.p12 -rw-r----- 1 oracle asmadmin 3926 Apr 6 16:32 cwallet.sso
Direct link to this issue: Unable to connect to the EM Express console from your 1-node DB system
Exadata DB Systems Only
Details: Backup operations to Object Storage using the Exadata backup utility (bkup_api) or RMAN fail with the following errors:
* DBaaS Error trace:
-> API::ERROR -> KBHS-00715: HTTP error occurred 'oracle-error'
-> API::ERROR -> ORA-19511: non RMAN, but media manager or vendor specific failure, error text:
-> API::ERROR -> ORA-19554: error allocating device, device type: SBT_TAPE, device name:
-> API::ERROR -> ORA-27023: skgfqsbi: media manager protocol error
-> API::ERROR Unable to verify the backup pieces
-> Oracle Error Codes found:
-> ORA-19554: error allocating device, device type: SBT_TAPE, device name:
-> ORA-19511: non RMAN, but media manager or vendor specific failure, error text:
-> KBHS-00712: ORA-29024 received from local HTTP service
-> ORA-27023: skgfqsbi: media manager protocol error
In response to policies implemented by two common web browsers regarding Symantec certificates, Oracle recently changed the certificate authority used for Oracle Cloud Infrastructure. The resulting change in SSL certificates can cause backups to Object Storage to fail if the Oracle Database Cloud Backup Module still points to the old certificate.
Before using the applicable workaround in this section, follow the steps in Updating Tooling on an Exadata Cloud Service Instance to ensure the latest version of
dbaastools_exa
is
installed on the system.Workaround for bkup_api: Check the log files for the errors listed above, and if found, reinstall the backup module.
Use the following command to check the status of the failed backup:
/var/opt/oracle/bkup_api/bkup_api bkup_status --dbname=<database_name>
Run the following command to reinstall the backup module:
/var/opt/oracle/ocde/assistants/bkup/bkup -dbname=<database_name>
Workaround for RMAN: Check the RMAN log files for the error messages listed. If found, log on to your host as the oracle user, and reinstall the backup module using your Swift credentials.
java -jar <opc_install.jar_path> -opcId '<Swift_user_ID>' -opcPass '<auth_token>' -container <objectstore_container> -walletDir <wallet_directory> -configfile <config_file> -host https://swiftobjectstorage.<region_name>.oraclecloud.com/v1/<object_storage_namespace> -import-all-trustcerts
Perform this workaround on all nodes in the cluster.
See Oracle Database Cloud Backup Module documentation for details on using this command.
Direct link to this issue: Backing up to Object Storage using bkup_api or RMAN fails due to certificate change
Details: With the release of the shared Database Home feature for Exadata DB systems, the Console now also synchronizes and displays information about databases that are created and managed by using the dbaasapi
and dbaascli
utilities. However, databases with Data Guard configured do not display correct information in the Console under the following conditions:
- If Data Guard was enabled by using the Console, and then a change is made to the primary or standby database by using
dbaascli
(such as moving the database to a different home), the result is not reflected in the Console. - If Data Guard was configured manually, the Console does not show a Data Guard association between the two databases.
Workaround: We are aware of the issue and working on a resolution. In the meantime, Oracle recommends that you manage your Data Guard enabled databases by using either only the Console or only command line utilities.
Direct link to this issue: Console information not synced for Data Guard enabled databases when using dbaascli
Details: This is a clusterware issue that occurs only when the Oracle GI version is 12.2.0.1 without any bundle patch. The problem is caused by corruption of a voting disk after you offline then online the disk.
Workaround: Determine the version of the GI, and whether the voting disk is corrupted. Repair the disk, if applicable, and then apply the latest GI bundle.
-
Verify the GI version is 12.2.0.1 without any bundle patch applied:
[root@rmstest-udaau1 ~]# su - grid [grid@rmstest-udaau1 ~]$ . oraenv ORACLE_SID = [+ASM1] ? +ASM1 The Oracle base has been set to /u01/app/grid [grid@rmstest-udaau1 ~]$ $ORACLE_HOME/OPatch/opatch lsinventory Oracle Interim Patch Installer version 12.2.0.1.6 Copyright (c) 2018, Oracle Corporation. All rights reserved. Oracle Home : /u01/app/12.2.0.1/grid Central Inventory : /u01/app/oraInventory from : /u01/app/12.2.0.1/grid/oraInst.loc OPatch version : 12.2.0.1.6 OUI version : 12.2.0.1.4 Log file location : /u01/app/12.2.0.1/grid/cfgtoollogs/opatch/opatch2018-01-15_22-11-10PM_1.log Lsinventory Output file location : /u01/app/12.2.0.1/grid/cfgtoollogs/opatch/lsinv/lsinventory2018-01-15_22-11-10PM.txt -------------------------------------------------------------------------------- Local Machine Information:: Hostname: rmstest-udaau1.exaagclient.sretest.oraclevcn.com ARU platform id: 226 ARU platform description:: Linux x86-64 Installed Top-level Products (1): Oracle Grid Infrastructure 12c 12.2.0.1.0 There are 1 products installed in this Oracle Home. There are no Interim patches installed in this Oracle Home. -------------------------------------------------------------------------------- OPatch succeeded.
-
Check the
/u01/app/grid/diag/crs/<hostname>/crs/trace/ocssd.trc
file for evidence that the GI failed to start due to voting disk corruption:ocssd.trc 2017-01-17 23:45:11.955 : CSSD:3807860480: clssnmvDiskCheck:: configured Sites = 1, Incative sites = 1, Mininum Sites required = 1 2017-01-17 23:45:11.955 : CSSD:3807860480: (:CSSNM00018:)clssnmvDiskCheck: Aborting, 2 of 5 configured voting disks available, need 3 ...... . 2017-01-17 23:45:11.956 : CSSD:3807860480: clssnmCheckForNetworkFailure: skipping 31 defined 0 2017-01-17 23:45:11.956 : CSSD:3807860480: clssnmRemoveNodeInTerm: node 4, slcc05db08 terminated. Removing from its own member and connected bitmaps 2017-01-17 23:45:11.956 : CSSD:3807860480: ################################### 2017-01-17 23:45:11.956 : CSSD:3807860480: clssscExit: CSSD aborting from thread clssnmvDiskPingMonitorThread 2017-01-17 23:45:11.956 : CSSD:3807860480: ################################### 2017-01-17 23:45:11.956 : CSSD:3807860480: (:CSSSC00012:)clssscExit: A fatal error occurred and the CSS daemon is terminating abnormally ------------ 2017-01-19 19:00:32.689 : CSSD:3469420288: clssnmFindVF: Duplicate voting disk found in the queue of previously configured disks queued(o/192.168.10.18/PCW_CD_02_slcc05cel10|[66223efc-29254fbb-bf901601-21009 cbd]), found(o/192.168.10.18/PCW_CD_02_slcc05cel10|[66223efc-29254fbb-bf901601-21009c bd]), is not corrupted 2017-01-19 19:01:06.467 : CSSD:3452057344: clssnmvVoteDiskValidation: Voting disk(o/192.168.10.19/PCW_CD_02_slcc05cel11) is corrupted
-
You can also use SQL*Plus to confirm that the voting disks are corrupted:
-
Log in as the grid user, and set the environment to
ASM
.[root@rmstest-udaau1 ~]# su - grid [grid@rmstest-udaau1 ~]$ . oraenv ORACLE_SID = [+ASM1] ? +ASM1 The Oracle base has been set to /u01/app/grid
-
Log in to SQL*Plus as
SYSASM
.$ORACLE_HOME/bin/sqlplus / as sysasm
-
Run the following two queries:
SQL> select name, voting_file from v$asm_disk where VOTING_FILE='Y' and group_number !=0; SQL> select CC.name, count(*) from x$kfdat AA JOIN (select disk_number, name from v$asm_disk where VOTING_FILE='Y' and group_number !=0) CC ON CC.disk_number = AA.NUMBER_KFDAT where AA.FNUM_KFDAT= 1048572 group by CC.name;
If the system is healthy, the results should look like the following example.
Query 1 Results
NAME VOTING_FILE ------------------------------ --------------- DBFSC3_CD_02_SLCLCX0788 Y DBFSC3_CD_09_SLCLCX0787 Y DBFSC3_CD_04_SLCLCX0786 Y
Query 2 Results
NAME COUNT(*) ------------------------------ --------------- DBFSC3_CD_02_SLCLCX0788 8 DBFSC3_CD_09_SLCLCX0787 8 DBFSC3_CD_04_SLCLCX0786 8
In a healthy system, every voting disk returned in the first query should also be returned in the second query and the counts for all the disks should be non-zero. Otherwise, one or more of your voting disks are corrupted.
-
-
If a voting disks is corrupted, offline the grid disk that contains the voting disk. The cells will automatically move the bad voting disk to the other grid disk and online that voting disk.
-
The following command offlines a grid disk named
DATAC01_CD_05_SCAQAE08CELADM13
.SQL> alter diskgroup DATAC01 offline disk DATAC01_CD_05_SCAQAE08CELADM13; Diskgroup altered.
-
Wait 30 seconds and then rerun the two queries in step 3c to verify that the voting disk migrated to the new grid disk and that it is healthy.
-
Verify the grid disk you offlined is now online:
SQL> select name, mode_status, voting_file from v$asm_disk where name='DATAC01_CD_05_SCAQAE08CELADM13';
The
mode_status
should beONLINE
, and thevoting_file
should NOT beY
.
Repeat steps 4a through 4c for each remaining grid disk that contains a corrupt voting disk.Note
If the CRS does not start because of the voting disk corruption, start it using Exclusive mode before you execute the command in step 4.
crsctl start crs -excl
-
-
If you are using Oracle GI version 12.2.0.1 without any bundle patch, you must upgrade the GI version to the latest GI bundle, whether or not a voting disk was corrupted.
See Patching Oracle Grid Infrastructure and Oracle Databases Using dbaascli for instructions on how to use the dbaascli utility to perform patching operations for Oracle Grid Infrastructure and Oracle Database on Exadata Database Service on Dedicated Infrastructure.
Direct link to this issue: Grid Infrastructure does not start after offlining and onlining a disk
Details: Exadata DB systems launched on June 15, 2018 or later automatically include the ability to create, list, and delete databases by using the Console, API, or Oracle Cloud Infrastructure CLI. However, systems provisioned before this date require extra steps to enable this functionality.
Attempts to use this functionality without the extra steps result in the following error messages:
- On creating a database - "Create Database is not supported on this Exadata DB system. To enable this feature, please contact Oracle Support."
- On terminating a database - "DeleteDbHome is not supported on this Exadata DB system. To enable this feature, please contact Oracle Support."
Workaround: You need to install the Exadata agent on each node of the Exadata DB system.
First, create a service request for assistance from Oracle Support Services. Oracle Support will respond by providing you with a preauthenticated URL for an Oracle Cloud Infrastructure Object Storage location where you can obtain the agent.
Before you install the Exadata agent:
- Upgrade the tooling (dbaastools rpm) to latest version on all the nodes on the Exadata DB system. See Updating Tooling on an Exadata Cloud Service Instance.
- Ensure that the system is configured to access Oracle Cloud Infrastructure Object Storage with the required security lists for the region in which the DB system was created. For more information about connectivity to Oracle Cloud Infrastructure Object Storage, see Prerequisites for Backups on Exadata Cloud Service.
To install the Exadata agent:
- Log on to the node as root.
-
Run the following commands to install the agent:
[root@<node_n>~]# cd /tmp [root@<node_n>~]# wget https://objectstorage.<region_name>.oraclecloud.com/p/1q523eOkAOYBJVP9RYji3V5APlMFHIv1_6bAMmxsS4E/n/dbaaspatchstore/b/dbaasexadatacustomersea1/o/backfill_agent_package_iwwva.tar [root@<node_n>~]# tar -xvf /tmp/backfill_agent_package_*.tar -C /tmp [root@<node_n>~]# rpm -ivh /tmp/dbcs-agent-2.5-3.x86_64.rpm
Example output:
[root@<node_n>~]# rpm -ivh dbcs-agent-2.5-3.x86_64.rpm Preparing... ########################################### [100%] Checking for dbaastools_exa rpm on the system Current dbaastools_exa version = dbaastools_exa-1.0-1+18.1.4.1.0_180725.0000.x86_64 dbaastools_exa version dbaastools_exa-1.0-1+18.1.4.1.0_180725.0000.x86_64 is good. Continuing with dbcs-agent installation 1:dbcs-agent ########################################### [100%] initctl: Unknown instance: initctl: Unknown instance: initzookeeper start/running, process 85821 initdbcsagent stop/waiting initdbcsadmin stop/waiting initdbcsagent start/running, process 85833 initdbcsadmin start/running, process 85836
-
Confirm that the agent is installed and running.
[root@<node_n>~]# rpm -qa | grep dbcs-agent dbcs-agent-2.5-0.x86_64 [root@<node_n>~]# initctl status initdbcsagent initdbcsagent start/running, process 97832
- Repeat steps 1 through 3 on the remaining nodes.
After the agent is installed on all nodes, allow up to 30 minutes for Oracle to complete additional workflow tasks such as upgrading the agent to the latest version, rotating the agent credentials, and so on. When the process is complete, you should be able to use the Exadata managed features in the Console, API, or Oracle Cloud Infrastructure CLI.
Direct link to this issue: Managed features not enabled for systems provisioned before June 15, 2018
Details: The patching configuration file (/var/opt/oracle/exapatch/exadbcpatch.cfg
) points to the object store of the us-phoenix-1
region, even if the Exadata DB system is deployed in another region.
This problem occurs if the release version of the database tooling package (dbaastools_exa
) is 17430 or lower.
Workaround: Follow the instructions in Updating Tooling on an Exadata Cloud Service Instance to confirm that the release version of the tooling package is 17430 or lower, and then update it to the latest version.
Direct link to this issue: Patching configuration file points to wrong region
Details: A change in how Oracle Linux 7 handles temporary files can result in the removal of required socket files from the /var/tmp/.oracle
directory. This issue affects only Exadata DB systems running the version 19.1.2 operating system image.
Workaround: Run sudo /usr/local/bin/imageinfo
as the opc user to determine your operating system image version. If your image version is 19.1.2.0.0.190306, follow the instructions in Doc ID 2498572.1 to fix the issue.
Direct link to this issue: Various database workflow failures due to Oracle Linux 7 removal of required temporary files
If you are scaling either regular data storage or recovery area (RECO) storage from a value less than 10,240 GB (10 TB) to a value exceeding 10,240 GB, perform the scaling in two operations. First, scale the system to 10,240 GB. After this first scaling operation is complete and the system is in the "available" state, perform a second scaling operation, specifying your target storage value above 10,240 GB. Attempting to scale from a value less than 10,240 GB to a value higher than 10,240 GB in a single operation can lead to a failure of the scaling operation. For instructions on scaling, see Scale Up the Storage For a Virtual Machine DB System.
Details: When scaling a virtual machine DB system to use a larger system shape, the scaling operation fails if a DB_Cache_nX
parameter is not set to 0 (zero).
Workaround: When scaling a virtual DB system, ensure that all DB_Cache_nX
parameters (for example, DB_nK_CACHE_SIZE
) are set to 0.
Database Migration
For known issues with Database Migration, see Database Migration Known Issues.
Developer Tools
Details: If you are using any of the OCI Java SDK synchronous clients that upload streams of data (for example ObjectStorageClient or DataSafeClient) and you do not define the RetryConfiguration at the client level or request level, you may be affected by silent data corruption.
Workaround: We are actively working on fixing this issue. For more information and possible workarounds, see the issue on GitHub.
Direct link to this issue: Potential data corruption issue in OCI Java SDK with binary data upload using default retries
Details: If you're using versions 2.14.1 and later of the OCI Java SDK, you may encounter performance regressions when using the SDK to call API operations on any of the OCI services. The regression causes a 30% to 60% increase in latency in SDK operations on any of the OCI services.
Workaround: We are actively working on fixing this issue. For more information and possible workarounds, see the issue on GitHub.
Direct link to this issue: Performance regression in OCI Java SDK versions 2.14.1 and later for all API operations
Details: Starting with version 2.0.0, the OCI SDK for Java supports using the
Jersey ApacheConnectorProvider
instead of the Jersey default
HttpUrlConnectorProvider
to allow the Apache HttpClient to make OCI
service calls.
The ApacheConnectorProvider
supports the use of Expect
header by default for some Object Storage service operations (see https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/100). This has been observed to cause performance regression in the
same Object Storage service operations.
Expect
header by
switching back to Jersey Default Connector (see https://docs.oracle.com/iaas/Content/API/SDKDocs/javasdkconfig.htm), or if
you're already using the ApacheConnectorProvider
, you can disable the
Expect
header with the ApacheConnectorProvider
by
doing the following when initializing the client
:final ApacheConnectorProperties apacheConnectorProperties =
ApacheConnectorProperties.builder()
.expectContinue(false) // disable the Expect header
.build();
final ApacheConfigurator configurator =
new ApacheConfigurator.NonBuffering(apacheConnectorProperties);
ObjectStorageClient objectStorageClient =
ObjectStorageClient.builder()
.clientConfigurator(configurator)
.build(provider);
Direct link to this issue: Performance regression with the Apache Connector Provider in OCI SDK for Java
Details: In versions 2.0.0 to 2.13.0 of the OCI Java SDK API, some operations that return a stream of data but don't return the content-length header in the response might return truncated data. This is caused by the SDK prematurely closing the stream before reading all the data.
Workaround:Update the OCI Java SDK client to version 2.13.1 or later. For more information about this issue and workarounds, see https://github.com/oracle/oci-java-sdk/issues/357
Direct link to this issue: Truncated response for operations that return binary data with the OCI Java SDK
Details: Due to some issues with one of its dependencies, the Go SDK feature which allows customers to automatically use new realms which might be unknown to the SDK is not functioning from within Cloud Shell.
can not create client, bad configuration: failed to get security token: failed to renew security token: failed to get security token: failed to call: Post "https://<endpoint>/v1/x509": dial tcp: lookup <endpoint> on 127.0.0.11:53: server misbehaving
panicked while retrying operation. Panic was: runtime error: invalid memory address or nil pointer dereference
Workaround: To resolve this issue, enable resolving regions using the instance metadata service for Go SDK. For more information, see: Adding Regions
Direct link to this issue: Go SDK cannot automatically find some regions while running in Cloud Shell
Details: You may encounter an increase in latency for operations made to some OCI services using the SDKs, Terraform, Ansible, and the CLI. This issue has been confirmed to impact the OCI Streaming service, and likely impacts the Email Delivery, Health Checks, NoSQL Database Cloud, Registry, Generic Artifacts, and Web Application Acceleration and Security services as well. This list is not comprehensive, and it is possible you may encounter the issue against other OCI services as well. The issue has been confirmed to NOT affect the OCI Object Storage and Functions services.
The following SDKs and tools are impacted:
- Go SDK (version 41.2.0 and later)
- .NET SDK (version 14.2.0 and later)
- Java SDK (version 2.0.0 and later)
- Python SDK (version 2.38.4 and later)
- CLI (version 2.25.0 and later)
- PowerShell Modules (version 9.2.0 and later)
- Ansible Modules (version 2.23.0 and later)
- Terraform Provider (version 4.30.0 and later)
Workarounds and more information: We are actively working on fixing this issue. For more information and possible workarounds, please see the following:
Direct link to this issue: Increased latency issues in operations for some OCI services using the SDKS and other tools
Details: The copy_boot_volume_backup_and_wait_for_state and copy_volume_backup_and_wait_for_state from the BlockStorage Client Composite operations throw a 404/NotAuthorizedOrNotFound when copying a backup from one region to another. For more information see: https://github.com/oracle/oci-python-sdk/issues/344.
Workaround: Instead of using the composite operations, use two different clients for this operation; one client in the Source Region to send the request for copying the backup from Region A to Region B, and a second client in Destination region to wait for the backup to become available. See example here: https://github.com/oracle/oci-python-sdk/blob/master/examples/copy_volume_backup_example.py
Direct link to this issue: Python SDK composite operations throw a 404 NotAuthorizedOrNotFound error even though operation is a success
Details: The OCI SDK for TypeScript and JavaScript have a known issue with big numbers greater than JavaScript's Number.MAX_SAFE_INTEGER. Any Numbers greater than Number.MAX_SAFE_INTEGER will result in rounding issue.
Workaround: We are aware of the issue where an API response may send back a number greater than JavaScript's Number.MAX_SAFE_INTERGER. At the moment the number rounding issue is not an impact to calling any APIs.
Direct link to this issue: Potential data rounding issue for big numbers with OCI SDK for TypeScript and JavaScript
Details: When using version 1.25.1 or earlier of the OCI Java SDK clients that
upload streams of data (for example ObjectStorageClient
or
FunctionsInvokeClient
), either synchronously and asynchronously,
and you use a RefreshableOnNotAuthenticatedProvider
(for example, for
Resource Principals or Instance Principals) you may be affected by silent data
corruption.
Workaround: Update the OCI Java SDK client to version 1.25.2 or later. For more information about this issue and workarounds, see Potential data corruption issue for OCI Java SDK on binary data upload with RefreshableOnNotAuthenticatedProvider.
Direct link to this issue: Potential data corruption issue with OCI Java SDK on binary data upload with RefreshableOnNotAuthenticatedProvider
Details: If you are using version 3.2.1.1 or earlier of the OCI HDFS Connector clients and you use a RefreshableOnNotAuthenticatedProvider (e.g. InstancePrincipalsCustomAuthenticator, or generally for Resource Principals or Instance Principals) you may be affected by silent data corruption.
Workaround: Update the OCI HDFS Connector client to version 3.2.1.3 or later. For more information about this issue and workarounds, see Potential data corruption issue for OCI HDFS Connector with RefreshableOnNotAuthenticatedProvider.
Direct link to this issue: Potential data corruption issue with OCI HDFS Connector on binary data upload with RefreshableOnNotAuthenticatedProvider
Details: When using the SDK for Python to perform binary upload operations you may encounter an issue with data corruption if retries are enabled or if you are using UploadManager.upload_file
.
Workaround: We're working on a resolution. For more information about this issue and workarounds, see Potential data corruption issue for PythonSDK retry on binary data upload.
Direct link to this issue: Potential data corruption with SDK for Python on binary upload
Update: The root cause of the issue causing data corruption has been fixed with the release of v2.54.0. Please use oci v2.54.0 or above to avoid data corruption. The behavior of older versions of the OCI Python SDK regarding this issue has been explained below.
Details: Customers using the OCI SDK for Python and
oci.object_storage.UploadManager.upload_stream
in FIPS mode might
be vulnerable to silent data corruption. If the circumstances to produce the issue are
true for your environment, the SDK reports success for the upload operation, but a
0-length object is uploaded.
The resolution differs depending on the state of your environment:
- Using
UploadManager.upload_stream()
in an environment which uses a FIPS-compliant OpenSSL version where the SDK for Python is not operating in FIPS mode as described in Using FIPS validated libraries.To determine if you fall under this scenario:
-
Verify that you are using a FIPS-compliant OpenSSL version by running the command
openssl version
. If "fips" is part of the version name, and you are not operating the SDK in FIPS mode, then you would fall under this scenario. -
If
oci.fips.is_fips_mode()
does not return True, then the SDK is not operating in FIPS mode.
Workaround: Upgrade the OCI SDK for Python to version 2.53.1 or greater and operate the SDK for Python in FIPS mode as described in Using FIPS validated libraries.Important
Not operating the SDK in FIPS mode while using a FIPS-compliant OpenSSL version will still result in data corruption while usingUploadManager.upload_stream()
. -
- Using
UploadManager.upload_stream()
when the SDK for Python is operating in FIPS mode as described in Using FIPS validated libraries and the SDK for Python is v2.53.0 or lower.If
oci.fips.is_fips_mode()
returns True, then the SDK is operating in FIPS mode.Resolution: Upgrade the OCI SDK for Python to version 2.53.1 or greater.
For more information about this issue, see Potential data corruption issue for multipart stream upload for OCI Python SDK on GitHub.
Direct link to this issue: Potential data corruption issue with SDK for Python and upload streams
DNS
Currently, there are no known DNS issues.
Document Understanding
For known issues with Document Understanding, see Known Issues.
Email Delivery
For known issues, see Known Issues for Email Delivery.
Events
Currently, there are no known Events issues.
File Storage
For known issues with File Storage, see Known Issues for File Storage.
Full Stack Disaster Recovery
Details: If you use volume group backups when performing DR operations for compute and storage across different ADs within the same region, back and forth DR transitions will cause the compute and associated block storage (which uses volume group backups) to end up in a different AD each time.
Workaround: This issue does not affect block storage that is replicated using volume group replication.
Details: Auto-tune performance settings for block storage volumes are not carried over during DR operations.
Workaround: For block storage volumes which have auto-tuned performance enabled you must re-enable these settings after FSDR transitions these block storage volumes to another region.
Details: If you perform a failover operation immediately after modifying an FSDR-protected resource, then the resource recovery may fail, or the resource may not be recovered properly. For example, if you change the replication target or other properties for a volume group that you added to a DR protection group, and the primary region suffers an immediate outage thereafter, FSDR may not detect the changes you made to the volume group replication settings, and this will affect recovery of that volume group.
Workaround: Perform a switchover precheck immediately after making any changes to any resources under DR protection.
Details: FSDR uses the Oracle Cloud Agent (OCA) Run Command utility to run local scripts on instances. When you configure a user-defined step to run a local script on a Microsoft Windows instance, then you cannot use the FSDR Run As User feature that allows you to specify a different userid to run local scripts that reside on instances.
Workaround: On Microsoft Windows instances, the script can only run as the default ocarun userid used by the Oracle Cloud Agent Run Command utility. This limitation does not affect Oracle Linux instances.
Details: FSDR uses the Oracle Cloud Agent (OCA) Run Command utility to run local scripts on instances. By default, these scripts are run as the ocarun user.
Workaround: On a Microsoft Windows instance, any local script that you configure to run as a user-defined step in a DR plan must be accessible and executable by this ocarun userid.
Details: When running a local script using a user-defined step in a DR plan, if you do not provide full paths to script interpreters or scripts, then FSDR will throw errors.
Workaround: When you configure a user-defined step in a DR plan to run a local script that resides on an instance, ensure that you provide the full path to any interpreter that may precede the script name, as well as the full path to the script.
/bin/sh /path/to/myscript.sh arg1 arg2
instead of sh myscript.sh arg1 arg2
Details: During DR operations, FSDR attempts to reassign the original private IP assigned to an instance if the CIDR-block of the destination subnet matches the CIDR-block of the source subnet, and if the original private IP of the instance is not already assigned.
If you use FSDR to relocate all the nodes in an OCFS2 cluster, and the private IP for any of the cluster node cannot be reassigned, those cluster nodes will detach from the OCFS2 cluster after the nodes are launched in the standby region.
Workaround: Ensure that the destination subnet's CIDR-block matches the CIDR-block of the source subnet and all private IP addresses required for cluster nodes are available in the destination subnet.
Details: After FSDR relocates an instance to a different region, the resource page of the instance may display the following message for Instance Access:
We are not quite sure how to connect to an instance this uses this image
Workaround: Ignore this message. SSH connections to the instance will work normally if you use the original SSH keyfile to connect to and authenticate the instance.
Details: After FSDR relocates an instance to a different region, the resource page of the instance may display incorrect information for the Image portion of its boot volume.
For example, the Image information column may display the following message: Error loading data
Workaround: This error message is for the display of the Image name but does not affect the operation of the instance or its boot volume.
Functions
Currently, there are no known Functions issues.
GoldenGate
Health Checks
For known issues with Health Checks, see Known Issues for Health Checks.
IAM
For IAM with Identity Domain issues, see Known Issues for IAM with Identity Domains.
For IAM without Identity Domain issues, see Known Issues for IAM without Identity Domains.
Integration
For known issues with Integration Generation 2, see Known Issues.
For known issues with Integration 3, see Known Issues.
Java Management
For details about known issues in the Java Management service, see Known Issues.
Language
Currently, there are no known issues with the Language service.
Load Balancer
For known issues with the Load Balancer service, see Known Issues.
Logging
For known issues with Logging see Known Issues for Logging.
Logging Analytics
Details: The on-demand upload of a zip file which is created on a Windows machine might sometimes fail to upload the log content. The reason for the failure is that the zip created on Windows has the same last modification time as the file's creation time. So, when the file is unzipped, the file's last modification time is set as the file's creation time which might be older than the timestamp of the log entries in the log file. In such a case, the log entries with the timestamp more recent than the file's last modification time are not uploaded.
An example of the issue:
Timestamp on the log entry: 2020-10-12 21:12:06
File last modification time of the log file: 2020-10-10 08:00:00
Workaround: Copy the log files to a new folder and create a zip file. This action makes the file's last modification time more recent than the timestamp of the log entries. Use this new zip file for on-demand upload.
Using the previous example, after the workaround is implemented:
Timestamp on the log entry: 2020-10-12 21:12:06
File last modification time of the log file: 2021-03-31 08:00:00
Direct link to this issue: On-demand upload from a Windows machine using a zip file
Details: Folders containing more than 10,000 files can cause log collection issues (as well as operating system issues).
When large folders are encountered by the Management Agent Logging Analytics plug-in, a message similar to the following example message is added to the Management Agent mgmt_agent.log file:
2020-07-30 14:46:51,653 [LOG.Executor.2388 (LA_TASK_os_file)-61850] INFO - ignore large dir /u01/service/database/logs. set property loganalytics.enable_large_dir to enable.
Resolution: We recommend avoiding large folders.
However, if you want to continue monitoring logs in large folders, then you can enable the property indicated in the mgmt_agent.log file by performing the following action:
sudo -u mgmt_agent echo "loganalytics.enable_large_dir=true" >> INSTALL_DIRECTORY/agent_inst/config/emd.properties
Replace INSTALL_DIRECTORY
with the path to the
agent_inst
folder.
Direct link to this issue: Special handling when monitoring logs in large folders
Managed Access
For known issues with Managed Access, see Known Issues.
Managed Cloud Self Service Platform
For known issues with Managed Cloud Self Service Platform, see Known Issues.
Management Agent
Currently, there are no known Management Agent issues.
Marketplace
For known issues with Marketplace, see Known issues.
Media Flow
For known issues with Media Flow, see Known Issues.
Media Streams
For known issues with Media Streams, see Known Issues.
Monitoring
For known issues with Monitoring, see Known Issues for Monitoring.
Network Load Balancer
For known issues with Network Load Balancer, see Known Issues.
Networking
For known issues with Networking, see Known Issues for Networking.
Notifications
For known issues with Notifications, see Known Issues for Notifications.
Object Storage
Currently, there are no known Object Storage issues.
Operations Insights
Currently, there are no known Operations Insights issues.
Oracle Cloud Marketplace
For known issues with Oracle Cloud Marketplace, see Known Issues.
OS Management
For details about known issues in the OS Management service, see Known Issues.
Partner Portal
For known issues with Partner Portal, see Known Issues.
Process Automation
For details about known issues in the Process Automation service, see Known Issues.
Queue
Currently, there are no known Queue issues.
Registry
Details: Up to now, you might have been using either the tenancy name or the tenancy namespace when logging in to Oracle Cloud Infrastructure Registry and when performing operations on images in the Container Registry.
After September 30, 2019, you will have to use the tenancy namespace rather than the tenancy name when using Oracle Cloud Infrastructure Registry.
Background: After September 30, 2019, you will not be able to:
- Specify the tenancy name when logging in to Oracle Cloud Infrastructure Registry.
- Perform operations on images that include tenancy name in the repository path.
Instead, you will have to use the tenancy namespace rather than the tenancy name when using Oracle Cloud Infrastructure Registry.
A tenancy namespace is an auto-generated and immutable random string of alphanumeric characters. For example, the namespace of the acme-dev tenancy might be ansh81vru1zp. You can see the tenancy namespace on the Container Registry page of the Console.
Note that for some older tenancies, the tenancy namespace might be the same as the tenancy name. If that is the case, no action is required.
On or before September 30, 2019, if the tenancy namespace and the tenancy name are different, you must:
- Start specifying the tenancy namespace when logging in to Oracle Cloud Infrastructure Registry, instead of the tenancy name.
- Start specifying the tenancy namespace when pushing new images to Oracle Cloud Infrastructure Registry, instead of the tenancy name.
- Migrate any existing images in Oracle Cloud Infrastructure Registry that include the tenancy name in the path.
The following workarounds and examples assume:
- tenancy name is acme-dev
- tenancy namespace is ansh81vru1zp
- username is jdoe@acme.com
Workaround for logging into Oracle Cloud Infrastructure Registry: Previously, when you logged in to Oracle Cloud Infrastructure Registry and were prompted for a username, you could have entered it in the format <tenancy-name>/<username>
.
For example:
$ docker login phx.ocir.io
Username: acme-dev/jdoe@acme.com
Password:
On or before September 30, 2019, you must start using the tenancy namespace instead of the tenancy name when logging in to Oracle Cloud Infrastructure Registry. When you are prompted for username, enter it in the format <tenancy-namespace>/<username>
.
For example:
$ docker login phx.ocir.io
Username: ansh81vru1zp/jdoe@acme.com
Password:
Workaround for pushing new images to Oracle Cloud Infrastructure Registry: Previously, when you pushed a new image to Oracle Cloud Infrastructure Registry, you could have specified the tenancy name as part of the repository path in the docker push
command. You could have entered the command in the format:
$ docker push <region-key>.ocir.io/<tenancy-name>/<image-name>:<tag>
For example:
$ docker push phx.ocir.io/acme-dev/helloworld:latest
On or before September 30, 2019, you must start using the tenancy namespace instead of the tenancy name in the docker push
command when you push new images. Enter the command in the format:
$ docker push <region-key>.ocir.io/<tenancy-namespace>/<image-name>:<tag>
For example:
$ docker push phx.ocir.io/ansh81vru1zp/helloworld:latest
Workaround for existing images in Oracle Cloud Infrastructure Registry that include the tenancy name in the repository path: If you have previously pushed images to Oracle Cloud Infrastructure Registry, those existing images could have included the tenancy name as part of the repository path. For example, phx.ocir.io/acme-dev/helloworld:latest
.
After September 30, 2019, you will not be able to perform operations on existing images in the Container Registry that include the tenancy name in the repository path.
So on or before September 30, 2019, for every existing image that contains the tenancy name in the repository path, you must replace tenancy name with tenancy namespace.
To replace tenancy name with tenancy namespace in the repository path of an existing image:
-
Pull the image by entering:
$ docker pull <region-key>.ocir.io/<tenancy-name>/<image-name>:<tag>
For example:
$ docker pull phx.ocir.io/acme-dev/helloworld:latest
-
Use the
docker tag
command to change the repository path by entering:$ docker tag <region-key>.ocir.io/<tenancy-name>/<image-name>:<tag> <region-key>.ocir.io/<tenancy-namespace>/<image-name>:<tag>
For example:
$ docker tag phx.ocir.io/acme-dev/helloworld:latest phx.ocir.io/ansh81vru1zp/helloworld:latest
-
Push the image with the new repository path to the Container Registry by entering:
$ docker push <region-key>.ocir.io/<tenancy-namespace>/<image-name>:<tag>
For example:
$ docker push phx.ocir.io/ansh81vru1zp/helloworld:latest
-
Repeat the above steps for every existing image that has tenancy name in the repository path.
Direct link to this issue: Use Tenancy Namespace instead of Tenancy Name in image tags and Docker login credentials on or before September 30, 2019
Resource Manager
For known issues with Resource Manager see Known Issues for Resource Manager.
Roving Edge Infrastructure
Currently, there are no known Roving Edge Infrastructure issues.
Search
For known issues with Search, see Known Issues for Search.
Search with OpenSearch
For known issues with Search with OpenSearch, see Known Issues.
Security Zones
For known issues with Security Zones, see Known Issues.
Service Connector Hub
For known issues with Service Connector Hub, see Known Issues for Service Connector Hub.
Service Mesh
For known issues with Service Mesh, see Known Issues.
Service Catalog
For known issues with Service Catalog, see Known Issues.
Speech
Currently, there are no known issues with the Speech service.
Storage Gateway
For known issues with Storage Gateway, see Known Issues.
Streaming
For known issues with Streaming, see Streaming Known Issues.
Tagging
For known issues with Tagging, see Known Issues for Tagging
Threat Intelligence
For known issues with Threat Intelligence, see Known Issues.
Traffic Management Steering Policies
Currently, there are no known Traffic Management Steering Policies issues.
Vault
Currently, there are no known Vault service issues.
Vision
For known issues with Vision, see Known Issues.
Visual Builder
For known issues with Visual Builder, see Known Issues for Oracle Visual Builder.
Visual Builder Studio
For known issues with Visual Builder Studio, see Known Issues for Oracle Visual Builder Studio.
Web Application Firewall (WAF)
For known issues with Web Application Firewall, see Web Application Firewall-Known Issues.
Web Application Acceleration
For known issues with Web Application Acceleration, see Known Issues.