Scenario: Archiving Logs to Object Storage
Archive log data to a bucket in Object Storage.
This scenario involves creating a connector. The connector processes and moves log data from Logging to Object Storage.
Required IAM Policy
If you're a member of the Administrators group, you already have the required access to execute this scenario. Otherwise, you need access to Monitoring, Notifications, and Object Storage.
The workflow for creating the connector includes a default policy when needed to provide permission for writing to the target service. If you're new to policies, see Getting Started with Policies and Common Policies.
Setting Up This Scenario
Setup is easy in the Console. Alternatively, you can use the Oracle Cloud Infrastructure CLI or API, which lets you execute the individual operations yourself.
Using the Console
This example walks through using the Console to create a connector that receives subnet log data from Logging and then moves this data to a bucket in Object Storage.
For help with troubleshooting, see Troubleshooting Connectors.
- Open the navigation menu and click Analytics & AI. Under Messaging, click Connector Hub.
- Choose the Compartment where you want to create the service connector.
- Select Create connector.
-
On the Create connector page, send subnet log data to a bucket:
- Type a Connector name, such as "Archive Logs." Avoid entering confidential information.
- Select the Resource compartment where you want to store the new connector.
- Under Configure connector, select your source and target services to move log data to a metric:
- Source: Logging
- Target: Object Storage
- Under Configure source connection, select your subnet log:
- Compartment: The compartment containing the log data you want.
- Log group: The log group containing the log data you want.
- Logs: The log object name for the log data you want.
- Under Configure target connection, select the bucket
where you want to archive this log data:
- Select the Compartment where you want to store the received (filtered) log data.
- Select the Bucket you want.
- If prompted to create a policy (required for access to create or update a service connector), select Create.
- Select Create.
Using the CLI
This example walks through using the CLI to create a connector that receives subnet log data from Logging and then moves this data to a bucket in Object Storage.
For information about using the API and signing requests, see REST API documentation and Security Credentials. For information about SDKs, see SDKs and the CLI.
-
Create a connector: Open a command prompt and run the
oci sch service-connector create
command:oci sch service-connector create --display-name "<display_name>" --compartment-id <compartment_OCID> --source [<source_in_JSON>] --target [<targets_in_JSON>]
For help with troubleshooting, see Troubleshooting Connectors.
Using the API
This example walks through using the API to create a connector that receives subnet log data from Logging and then moves this data to a bucket in Object Storage.
For information about using the API and signing requests, see REST API documentation and Security Credentials. For information about SDKs, see SDKs and the CLI.
Use the following operations:
-
CreateServiceConnector: Create a connector.
Example CreateServiceConnector requestPOST /20200909/serviceConnectors Host: service-connector-hub.us-phoenix-1.oraclecloud.com <authorization and other headers> { "compartmentId": "<compartment_OCID>", "description": "My connector description", "displayName": "My Connector", "source": { "kind": "logging", "logSources": [ { "compartmentId": "<compartment_OCID>", "logGroupId": "<log_group_OCID>", "logId": "<log_OCID>" } ] }, "target": { "kind": "objectStorage", "bucketName": "MyBucket", "objectNamePrefix": "myprefix" } }
For help with troubleshooting, see Troubleshooting Connectors.