Ingest Logs from Other OCI
Services Using Service Connector
You can analyze the logs to troubleshoot issues, monitor health and
performance and observe the operational tasks in Oracle Cloud Infrastructure services by ingesting the logs into Oracle Logging Analytics.
Use the Service Connector to identify your Oracle Cloud Infrastructure service as the source of the logs and Oracle Logging Analytics as
the destination. For information on how the Service Connector Hub works, see
Service Connector Hub
Overview in Oracle Cloud Infrastructure
Documentation.
Note
After the service connector is created, an entity is
automatically created for processing the logs. To ensure proper log
collection, the entity must not be deleted.
Important: Oracle recommends that you use the data
ingestion work flow available in Logging Analytics console to quickly ingest logs from other OCI
services. Go to Logging Analytics
Home or Log Explorer, click Compass, and click
Add Data.
For all types of logs from OCI services except OCI Audit
Logs and IDCS Audit Logs, expand the
section Monitor OCI resources and click Configure
log collection for OCI resources.
In case of OCI Audit Logs or IDCS Audit
Logs, expand the section Security and
Compliance and click the logs of your choice. In
this work flow, all the required resources like policies,
log group, and service connector are automatically
created.
Follow the intuitive steps in the work flow to start ingesting logs. As a
prerequisite, ensure that you have the required permissions to
complete the steps. For a quick walk through of the steps, watch
Video: How to
Quickly Ingest Logs into Logging Analytics from Other OCI
Services in Oracle Cloud Observability and
Management Platform.
Alternatively, you can manually set up the log collection by performing
the following steps:
List of Oracle-defined sources for
collecting logs: For the list of
Oracle-defined sources to collect logs from Oracle Cloud Infrastructure
services, see Oracle-defined Sources and search for sources with title
OCI...
Types of service logs you can collect:
For the types of logs you can collect from the Oracle Cloud Infrastructure
services, the parsers, example log content, fields,
and JSON path, see OCI Parser Details.
Filter logs collected though service
connector: The service connector OCID is
mapped to the field Log Origin. To
view the logs flowing from that service connector to
Oracle Logging Analytics, filter the logs by the
field Log Origin. See Filter Logs by Pinned Attributes and Fields.
Allow Collection of Logs from OCI
Logging Service 🔗
Based on the type of service logs that you want to ingest, you must create
policies to enable Oracle Logging Analytics to get
the information about the resources and create an entity for each resource.
After you create the policy, the entity that is created will be auto-associated with
all the logs collected from that resource. If the policy is not created, then the logs
are still ingested but the entity is not created.
The following permissions are for uploading logs to Oracle Logging Analytics from the service
connector. You are prompted to add these policy statements when you create the service
connector through OCI console. Alternatively, you can manually create the policy that
includes the following policy statements:
allow any-user to {LOG_ANALYTICS_LOG_GROUP_UPLOAD_LOGS} in compartment id <Log_Group_Compartment_OCID> where all {request.principal.type = 'serviceconnector', target.loganalytics-log-group.id = '<Log_Group_OCID>', request.principal.compartment.id = '<Service_Connector_Compartment_OCID>'}
allow group <userGroup> to MANAGE serviceconnectors in tenancy
allow group <userGroup> to READ logging-family in tenancy
In the above policy statements,
Log_Group_Compartment_OCID: The compartment OCID of
the log group in Oracle Logging Analytics where the logs must be stored.
Log_Group_OCID: The OCID of the log group in Oracle Logging Analytics where the
logs must be stored.
Service_Connector_Compartment_OCID: The compartment
OCID of the service connector hub.
Note
If you enabled Oracle Logging Analytics using the
onboarding UI which is available when you navigate to the service for the first time, then
some policies are already created. See Policies Created While Onboarding Logging Analytics.
Policy for Each Type of Service Logs
Oracle Logging Analytics creates
an entity representing the underlying OCI resource when new logs are received through
the service connector. In order to obtain the necessary information from the OCI
resource, you must provide Oracle Logging Analytics with a minimum of read access to the OCI resource.
For example, in order to read information about a VNIC, you can write one of the
following policies:
Policy statement with the READ PRIVILEGE of the OCI resource:
allow service loganalytics to {VNIC_READ} in compartment <specify_compartment>
OR
Policy statement with the read verb for the OCI RESOURCE:
allow service loganalytics to read vnics in compartment <specify_compartment>
The above policy statements restrict the read access to a
compartment. To extend the access to the entire tenancy, you can change the policy
statement accordingly.
The following OCI resources are supported in Oracle Logging Analytics for log collection through the service connector. You can
either create the policy using read verb for the OCI resource or use the read
privilege for the resource as illustrated above.
OCI Resource Description
OCI Resource
Read Privilege
Analytics Cloud Instance
analytics-instances
ANALYTICS_INSTANCE_READ
API Gateway
api-gateways
API_GATEWAY_READ
APM Domain
apm-domains
APM_DOMAIN_READ
Container Engine For Kubernetes
clusters
CLUSTER_READ
Data Flow (Application)
dataflow-application
DATAFLOW_APPLICATION_READ
Data Integration Workspace
dis-workspaces
DIS_WORKSPACE_READ
Data Science Jobs
data-science-jobs
DATA_SCIENCE_JOB_READ
Data Science Model Deployments
data-science-model-deployments
DATA_SCIENCE_MODEL_DEPLOYMENT_READ
DevOps Build Pipeline
devops-build-pipeline
DEVOPS_BUILD_PIPELINE_READ
DevOps Build Pipeline Stage
devops-build-pipeline-stage
DEVOPS_BUILD_PIPELINE_STAGE_READ
DevOps Build Run
devops-build-run
DEVOPS_BUILD_RUN_READ
DevOps Deployment
devops-deployment
DEVOPS_DEPLOY_DEPLOYMENT_READ
DevOps Deployment Pipeline
devops-deploy-pipeline
DEVOPS_DEPLOY_PIPELINE_READ
DevOps Deployment Stage
devops-deploy-stage
DEVOPS_DEPLOY_STAGE_READ
Email Delivery Service
approved-senders
APPROVED_SENDER_READ
Events Service
cloudevents-rules
EVENTRULE_READ
Functions (FN App)
fn-app
FN_APP_READ
Functions (FN Function)
fn-function
FN_FUNCTION_READ
GoldenGate Deployment
goldengate-deployments
GOLDENGATE_DEPLOYMENT_READ
Instance
instances
INSTANCE_READ
IPSec Tunnel
ipsec-connections
IPSEC_CONNECTION_READ
Load Balancer
load-balancers
LOAD_BALANCER_READ
Media Workflow
media-workflow
MEDIA_WORKFLOW_READ
Media Workflow Job
media-workflow-job
MEDIA_WORKFLOW_JOB_READ
Network Firewall
network-firewall
NETWORK_FIREWALL_READ
Object Storage (Bucket)
buckets
BUCKET_READ
OCI Database with PostgreSQL
postgres-db-systems
POSTGRES_DB_SYSTEM_READ
OIC Instance
integration-instance
INTEGRATION_INSTANCE_READ
Operator Control
operator-control-family
-
Service Connector
serviceconnectors
SERVICE_CONNECTOR_READ
VCN - VNIC
vnics
VNIC_READ
Web Application Firewall
web-app-firewall
WEB_APP_FIREWALL_READ
Note
If you enabled Oracle Logging Analytics using the
onboarding UI which is available when you navigate to the service for the first time, then
some policies are already created. See Policies Created While Onboarding Logging Analytics.
Set Up the Service Connector to
Ingest Logs 🔗
Before you set up the service connector to ingest logs, ensure that the
compartment and log group are identified for the logs that you want to ingest.
In the following example, the steps show you how to collect VCN service logs
from Oracle Cloud Infrastructure Logging service:
This is a suggestive step to show you how to enable logs in the Oracle Cloud Infrastructure Logging service.
Go to Oracle Cloud InfrastructureLogging service > Go to Logs.
Click Enable Resource Log to enable VCN
service logs. The dialog box opens.
Select the resource compartment.
Select the service, for example, Virtual Cloud Network
(subnets).
Select the resource, for example, the VCN resource.
Under Configure Log, select the log category, for
example, Flow Logs, and the log name.
Under Log Location, select the compartment and log group
that Oracle Logging Analytics will
refer the logs from.
Click Enable Log.
Set up the service connector by specifying the source service of the
logs and the target as Oracle Logging Analytics. You can either set it up from the source service that has
integrated with Oracle Cloud Infrastructure Service
Connector Hub, for example, Oracle Cloud Infrastructure
Logging service, or directly from Oracle Cloud Infrastructure Service Connector Hub.
Go to Oracle Cloud InfrastructureLogging service > Go to Service Connectors > Click
Create Connector.
Alternatively, go to Oracle Cloud InfrastructureService Connector Hub service > Click Create Service
Connector.
The Create Service Connector page opens.
Enter a name for the connector and provide a description.
Select the resource compartment where the connector resource
must be created.
Under Configure Service Connector, specify
Logging as the Source service, and
Logging Analytics as the Target
service.
Under Configure Source Connection, provide the details of
the logs to collect from the service, for example, the VCN service logs.
Select the compartment name, the log group to which the logs
belong, and the name of the logs that you had configured in step
1.
You can configure the same service connector to collect more logs.
Click Another Log and repeat step 2-d.
Optionally, you can create filters under Configure Task.
Click Create Connector.
After the service connector is created, you can verify that the selected logs
are available in Oracle Logging Analytics.
Allow Cross-Tenancy Log
Collection from OCI Logging Service 🔗
Let Source_Tenant be the tenant of the source service such
as Oracle Cloud Infrastructure Logging from which logs are collected. Let
Target_Tenant be the tenant in which the service connector is
created. The service connector is configured with Oracle Logging Analytics as the target for the logs that are collected from the
source service. It is assumed that the service connector hub and Oracle Logging Analytics are available on the
same target tenant.
Set the following policies to configure the log collection from a tenancy that is
different from the tenancy the service connector is created in.
Policies To Be Added in the Source
Tenant
Here is an example of policy statements which allow any user of the
service connector hub tenancy to have READ access to the Logging
service:
define tenancy <Target_Tenant> as <Target_Tenant_OCID>
define group <Common_User_Group> as <Common_User_Group_OCID>
admit any-user of tenancy <Target_Tenant> to read logging-family IN TENANCY WHERE ALL {request.principal.type = 'serviceconnector'}
admit group <Common_User_Group> of tenancy <Target_Tenant> to read logging-family IN TENANCY
Additionally, the following permissions are required to read the Audit
event logs:
admit group <Common_User_Group> of tenancy <Target_Tenant> to read audit-events in TENANCY
admit any-user of tenancy <Target_Tenant> to read audit-events IN tenancy WHERE ALL {request.principal.type = 'serviceconnector'}
Here is an example of policy statements which allow any user to access
the Logging service through the service connector hub, and the target IAM group
Common_User_Group to have MANAGE access to the
service connector hub:
define tenancy <Source_Tenant> as <Source_Tenant_OCID>
endorse any-user to read logging-family IN tenancy <Source_Tenant> WHERE ALL {request.principal.type = 'serviceconnector'}
endorse group <Common_User_Group> to read logging-family IN tenancy <Source_Tenant>
Additionally, the following permissions are required to read the source
Audit event logs:
endorse group <Common_User_Group> to read audit-events in tenancy <Source_Tenant>
endorse any-user to read audit-events in tenancy <Source_Tenant> WHERE ALL {request.principal.type = 'serviceconnector'}
The following permissions are for uploading logs to Oracle Logging Analytics from the service
connector. Make sure to manually create the policy that includes the following
policy statements:
allow any-user to {LOG_ANALYTICS_LOG_GROUP_UPLOAD_LOGS} in compartment id <Log_Group_Compartment_OCID> where all {request.principal.type = 'serviceconnector', target.loganalytics-log-group.id = '<Log_Group_OCID>', request.principal.compartment.id = '<Service_Connector_Compartment_OCID>'}
allow group <Common_User_Group> to MANAGE serviceconnectors in tenancy
In the above policy statements,
Log_Group_OCID: The OCID of the Oracle Logging Analytics log
group.
Log_Group_Compartment_OCID: The OCID of the
compartment where the Oracle Logging Analytics log group is located.
Service_Connector_Compartment_OCID: The
compartment OCID of the service connector.
Common_User_Group: The user group that creates
the service connector.
Create a Connector Between the Source and Target Tenants
After the required policies are created for the source and target
tenants, create a service connector using CLI. The following example CLI command
specifies Logging as the source and Oracle Logging Analytics as the target for creating the cross-tenancy service
connector: