Scenario: Creating Dimensions for a Monitoring Target

If you're a member of the Administrators group, you already have the required access to complete this scenario. Otherwise, you need access to Monitoring.

The workflow for creating the connector includes a default policy when needed to provide permission for writing to the target service. If you're new to policies, see Getting Started with Policies and Common Policies.

This topic describes the goal of this scenario.

The goal of this scenario is to filter update events for Object Storage buckets. For example, find updates that changed buckets to public access. Finding public buckets can help prevent leakage of secrets. In addition to public access type, this scenario sets up filters for bucket name, compartment name, availability domain, versioning status, and a static value.

This topic describes the tasks involved in setting up this scenario.

This scenario creates a metric from a log using the Connector Hub service. Setting up this scenario involves the following tasks:

• Create a connector to move logs from Logging to a custom metric with dimensions in Monitoring.

  • Log data: Bucket - Update event from audit logs (_Audit log group)

    Note
    
    While this scenario uses the _Audit log group and the bucket update event, you can use the same approach with any log available in your tenancy.

  • Metric namespace: bucket_events

  • Metric name: update

  • Static value buckets-from-connector (dimension name: mytags)

  • Extracted value using path dimensions:

    Note
    
    

    Each new dimension value creates a new metric stream. To avoid generating too many unique metric streams, which could potentially result in throttling, we recommend excluding GUIDs or UUIDs (such as compartment OCIDs) from the dimensions.

    Path property (dimension name)	Value
    bucketName	logContent.data.additionalDetails.bucketName
    compartmentName	logContent.data.compartmentName
    availabilityDomain	logContent.data.availabilityDomain
    publicAccessType	logContent.data.additionalDetails.publicAccessType
    versioning	logContent.data.additionalDetails.versioning

• Filter the moved data with the dimensions (static and extracted values).

You can complete these tasks in the Oracle Cloud Infrastructure Console, CLI, or API.

This topic describes how to use the Console to set up the scenario.

For help with troubleshooting, see Troubleshooting Connectors and Troubleshooting Notifications.

This topic describes how to use the CLI to create the connector and filter the moved data.

For information about using the API and signing requests, see REST API documentation and Security Credentials. For information about SDKs, see SDKs and the CLI.

1. Define the log source for the connector: Save a file named source.json.

   Example source.json file

   {
     "kind": "logging",
     "logSources": [
       {
         "compartmentId": "<compartment_OCID>",
           "logGroupId": "_Audit",
           "logId": null
       }
     ]
   }

2. Define the target (metric and dimensions) for the connector: Save a file named target.json.

   Example target.json file

   {
     "kind": "monitoring",
     "compartmentId": "<compartment_OCID>",
     "metricNamespace": "bucket_events",
     "metric": "update",
     "dimensions": [
       {"name": "mytags", "dimensionValue": {"kind": "static", "value": "buckets-from-connector"}},
       {"name": "bucketName", "dimensionValue": {"kind": "jmesPath", "path": "logContent.data.additionalDetails.bucketName"}},
       {"name": "compartmentName", "dimensionValue": {"kind": "jmesPath", "path": "logContent.data.compartmentName"}},
       {"name": "availabilityDomain", "dimensionValue": {"kind": "jmesPath", "path": "logContent.data.availabilityDomain"}},
       {"name": "publicAccessType", "dimensionValue": {"kind": "jmesPath", "path": "logContent.data.additionalDetails.publicAccessType"}},
       {"name": "versioning", "dimensionValue": {"kind": "jmesPath", "path": "logContent.data.additionalDetails.versioning"}}
     ]
   }

3. Create a connector: Open a command prompt and run the oci sch service-connector create command:

   Example command for creating a connector

   oci sch service-connector create
    --compartment-id <compartment_OCID>
    --display-name "Bucket - Update"
    --source file://source.json
    --target file://target.json

4. Query the metric data using the custom dimensions: Open a command prompt and run the oci monitoring metric-data summarize-metrics-data command:

   Example command for querying metric data using custom dimensions

   oci monitoring metric-data summarize-metrics-data
    --compartment-id <compartment_OCID>
    --namespace bucket_events
    --query-text update[1m]{mytags = "buckets-from-connector", bucketName = "bucket-20210722-1148", coompartmentName = "development", availabilityDomain = "PHX-AD-3", publicAccessType = "ObjectRead", versioning = "Enabled"}.count()

For help with troubleshooting, see Troubleshooting Connectors and Troubleshooting Notifications.

This topic describes how to use the API to create the connector and filter the moved data.

For information about using the API and signing requests, see REST API documentation and Security Credentials. For information about SDKs, see SDKs and the CLI.

Use the following operations:

1. CreateServiceConnector: Create a connector.

   Example CreateServiceConnector request

   POST /20200909/serviceConnectors
   Host: service-connector-hub.us-phoenix-1.oraclecloud.com
   <authorization and other headers>
   {
     "compartmentId": "<compartment_OCID>",
     "displayName": "Bucket - Update",
     "source": {
       "kind": "logging",
       "logSources": [
         {
           "compartmentId": "<compartment_OCID>",
           "logGroupId": "_Audit",
           "logId": null
         }
       ]
     },
     "target": {
       "compartmentId": "<compartment_OCID>",
       "dimensions": [
         {"name": "mytags", "dimensionValue": {"kind": "static", "value": "buckets-from-connector"}},
         {"name": "bucketName", "dimensionValue": {"kind": "jmesPath", "path": "logContent.data.additionalDetails.bucketName"}},
         {"name": "compartmentName", "dimensionValue": {"kind": "jmesPath", "path": "logContent.data.compartmentName"}},
         {"name": "availabilityDomain", "dimensionValue": {"kind": "jmesPath", "path": "logContent.data.availabilityDomain"}},
         {"name": "publicAccessType", "dimensionValue": {"kind": "jmesPath", "path": "logContent.data.additionalDetails.publicAccessType"}},
         {"name": "versioning", "dimensionValue": {"kind": "jmesPath", "path": "logContent.data.additionalDetails.versioning"}}
       ]
       "kind": "monitoring",
       "metric": "update",
       "metricNamespace": "bucket_events"
     }
   }

2. SummarizeMetricsData: Query the metric data by dimensions.

   Example SummarizeMetricsData request

   POST /20180401/metrics/actions/summarizeMetricsData?compartmentId=<compartment_OCID>
   Host: telemetry.us-phoenix-1.oraclecloud.com
   <authorization and other headers>
   {
     "namespace": "bucket_events",
     "query": "update[1m]{mytags = "buckets-from-connector", bucketName = "bucket-20210722-1148", coompartmentName = "development", availabilityDomain = "PHX-AD-3", publicAccessType = "ObjectRead", versioning = "Enabled"}.count()"
   }

For help with troubleshooting, see Troubleshooting Connectors and Troubleshooting Notifications.