Scenario: Alarming on Log Data
Set up alarms for log data using Connector Hub, Logging, and Monitoring services.
This scenario involves creating a connector and an alarm. The connector (Connector Hub) processes and moves log data from Logging to Monitoring while the alarm fires when triggered by received log data.
Required IAM Policy
If you're a member of the Administrators group, you already have the required access to execute this scenario. Otherwise, you need access to Monitoring and Notifications.
The workflow for creating the connector includes a default policy when needed to provide permission for writing to the target service. If you're new to policies, see Getting Started with Policies and Common Policies.
Setting Up This Scenario
Setup is easy in the Console. Alternatively, you can use the Oracle Cloud Infrastructure CLI or API, which lets you execute the individual operations yourself.
Using the Console
This section walks through creating a connector and an alarm using the Console and then updating the topic created with the alarm.
Another workflow for this scenario involves creating your topic and subscriptions first, then selecting this topic when you create your alarm.
For help with troubleshooting, see Troubleshooting Connectors and Troubleshooting Notifications.
This example walks through using the Console to create a connector that filters VCN flow log from Logging for rejected traffic and then moves this log data to a new metric in Monitoring.
- Open the navigation menu and click Analytics & AI. Under Messaging, click Connector Hub.
- Choose the Compartment where you want to create the service connector.
- Select Create connector.
-
On the Create connector page, filter VCN flow log to a new metric:
- Type a Connector name such as "VCN Flow Log Errors." Avoid entering confidential information.
- Select the Resource compartment where you want to store the new connector.
- Under Configure connector, select your source and target services to move log data to a metric:
- Source: Logging
- Target: Monitoring
- Under Configure source connection, select your VCN flow log:
- Compartment name: The compartment containing the VCN flow log data.
- Log group: The log group containing the VCN flow log data.
- Logs: The log object name for your VCN flow logs.
-
Under Log filter task, filter the log data to rejected traffic:
- Property: data.action
- Operator: =
- Value: REJECT
If you are interested in rejected traffic for a particular port or address, add another filter. For example, select the property data.destinationPort or data.destinationAddress.
- Under Configure target connection, enter the metric
namespace and metric name that you want to use for the filtered log data:
- Select the Compartment where you want to store the metric data points corresponding to the VCN flow log data for rejected traffic.
- Enter a new Metric Namespace: vcnlogs. Note
When typing a new metric namespace, press ENTER to submit it. - Enter a new Metric: rejectedtraffic
- If prompted to create a policy (required for access to create or update a service connector), select Create.
- Select Create.
This example walks through using the Console to create an alarm that sends a message when Monitoring receives metric data points corresponding to VCN flow log data for rejected traffic. During this process, you'll create a topic and an email subscription.
- Open the navigation menu and click Observability & Management. Under Monitoring, click Alarm Definitions.
-
Select Create Alarm.
-
On the Create Alarm page, under Define alarm, set up your threshold:
- Metric description:
- Compartment: (select the compartment specified in the previous task, where you are storing the metric data points corresponding to the VCN flow log data)
- Metric namespace: (enter the metric namespace that you specified in the connector)
- Metric name: (enter the metric name that you specified in the connector)
- Interval: 1m
- Statistic: Count
-
Trigger rule:
- Operator: greater than
- Value: 0
- Trigger delay minutes: 1
- Metric description:
- Under Notifications, Destinations, set
up an email notification:
- Destination Service: Notifications Service
- Compartment: (select the compartment where you want to create the topic and associated subscriptions)
-
Topic: Select Create a topic
Note
If you already created your topic and email subscription, you can select that topic here instead of creating a new one.- Topic name: Rejected Traffic Topic
- Subscription protocol: Email
- Subscription Email: (type your email address here)
-
Select Save alarm.
Using the CLI
This section walks through creating the connector, topic, subscriptions, and alarm using the CLI.
For information about using the API and signing requests, see REST API documentation and Security Credentials. For information about SDKs, see SDKs and the CLI.
-
Create a connector: Open a command prompt and run the
oci sch service-connector create
command:oci sch service-connector create --display-name "<display_name>" --compartment-id <compartment_OCID> --source [<source_in_JSON>] --tasks [<tasks_in_JSON>] --target [<targets_in_JSON>]
-
Create a topic: Open a command prompt and run the
oci ons topic create
command:oci ons topic create --name "Alarm Topic" --compartment-id "<compartment-ocid>"
-
To this topic, add a subscription referencing your email address. Open a command prompt and run the
oci ons subscription create
command:oci ons subscription create --compartment-id "<compartment-ocid>" --topic-id "<topic-ocid>" --protocol "EMAIL" --subscription-endpoint "john.smith@example.com"
-
Create an alarm that defines the error threshold and references this topic as the destination: Open a command prompt and run the
oci monitoring alarm create
command:oci monitoring alarm create --display-name "My Alarm" --compartment-id "<compartment-ocid>" --metric-compartment-id "<compartment-ocid>" --namespace "oci_computeagent" --query-text "<yourmetric>[1m].count() > 0" --severity "INFO" --destinations "<topic-ocid>" --is-enabled true
For help with troubleshooting, see Troubleshooting Connectors and Troubleshooting Notifications.
Using the API
This section walks through creating the connector, topic, subscription, and alarm using the API.
For information about using the API and signing requests, see REST API documentation and Security Credentials. For information about SDKs, see SDKs and the CLI.
Use the following operations:
-
CreateServiceConnector: Create a connector.
Example CreateServiceConnector requestPOST /20200909/serviceConnectors Host: service-connector-hub.us-phoenix-1.oraclecloud.com <authorization and other headers> { "compartmentId": "<compartment_OCID>", "description": "My connector description", "displayName": "My Connector", "source": { "kind": "logging", "logSources": [ { "compartmentId": "<compartment_OCID>", "logGroupId": "<log_group_OCID>", "logId": "<log_OCID>" } ] }, "target": { "compartmentId": "<compartment_OCID>", "kind": "metrics", "metric": "<yourmetric>", "metricNamespace": "<yournamespace>" }, "tasks": [ { "condition": "data.action='REJECT'", "kind": "logRule" } ] } }
-
CreateTopic: Create a topic.
Example CreateTopic requestPOST /20181201/topics Host: notification.us-phoenix-1.oraclecloud.com <authorization and other headers> { "name": "Alarm Topic", "compartmentId": "<compartment_OCID>" }
-
CreateSubscription: To this topic, add a subscription referencing your email address.
Example CreateSubscription request: EmailPOST /20181201/subscriptions Host: notification.us-phoenix-1.oraclecloud.com <authorization and other headers> { "topicId": "<topic_OCID>", "compartmentId": "<compartment_OCID>", "protocol": "EMAIL", "endpoint": "john.smith@example.com" }
-
CreateAlarm: Create an alarm that defines the memory threshold and references this topic.
Example CreateAlarm requestPOST /20180401/alarms Host: telemetry.us-phoenix-1.oraclecloud.com <authorization and other headers> { "displayName": "My Alarm", "compartmentId": "<compartment_OCID>", "metricCompartmentId": "<compartment_OCID>", "namespace": "<yournamespace>", "query": "<yourmetric>[1m].count() > 0", "severity": "INFO", "destinations": [ "<topic_OCID>" ], "isEnabled": true }
For help with troubleshooting, see Troubleshooting Connectors and Troubleshooting Notifications.