Oracle Cloud Infrastructure Documentation

Scenario: Split Messages by Metric Stream

Walk through setting up an alarm to send a message for each metric stream. In this example, you want to be notified whenever a server exceeds a threshold. With this setup, you receive server-specific messages.

Caution

With messages split by metric stream, consider the number of resources monitored by the alarm. If hundreds of resources simultaneously trigger the alarm to fire, then several messages are sent at the same time. Lots of messages can flood the phone (SMS), inbox (email), or other messaging endpoint, and some messages might be delayed because of service limits. For more information on limits and best practices, see Alarm Message Limits.

Required IAM Policy

This topic describes access requirements for the scenario.

To use Oracle Cloud Infrastructure, you must be granted security access in a policy  by an administrator. This access is required whether you're using the Console or the REST API with an SDK, CLI, or other tool. If you get a message that you don’t have permission or are unauthorized, verify with your administrator what type of access you have and which compartment  to work in.

If you're a member of the Administrators group, you already have the required access to complete this scenario.

Administrators: For common policies allowing users to manage alarms and create topics, see Alarm Access for Groups.

Goal

The goal of this hypothetical scenario is to receive separate alarm messages per server. Let's say that you are monitoring 50 servers that emit a custom CPU utilization metric and you want to know if any exceed 80 percent CPU utilization. You want to receive a message whenever an individual server's metrics trigger the alarm.

Setting Up This Scenario

Setup involves creating a threshold alarm enabled for metric stream-specific messages. In this hypothetical scenario, you select the custom metric MyCustomCPUMetric and the resource group MyServerResourceGroup.

You can complete these tasks in the Oracle Cloud Infrastructure Console, CLI, or API.

    1. Open the navigation menu and click Observability & Management. Under Monitoring, click Alarm Definitions.
    2. Click Create alarm.
    3. On the Create alarm page, under Define alarm, enter an Alarm name.

      Example: Server-Specific Messages

      Optionally change the Alarm severity and enter message text for Alarm body.

    4. Under Metric description, select the custom metric and define the query.
      • Compartment: Select the compartment .
      • Metric namespace: Select the namespace for the custom metric.
      • Resource group: MyServerResourceGroup

      • Metric name: MyCustomCPUMetric

        Note

        Any OCI metric or custom metric can be selected.
      • Interval: 1m
      • Statistic: Count
    5. Skip Metric dimensions.
    6. Under Trigger rule, set the threshold to 80 and delay messages by 10 minutes:
      • Operator: greater than
      • Value: 80
      • Trigger delay minutes: 10
    7. Under Set alarm notifications, Destination, provide the following values:
      • Destination service: Notifications
      • Compartment: Select the compartment containing the topic.
      • Topic: Select the topic that you want to send notifications to. In this scenario, the topic already exists, and contains the subscriptions you want (SMS, email, and others).
      This scenario uses a topic for alarm notifications. You can opt to use a stream instead, which is helpful when you expect a high volume of alarm notifications. For more information, see Alarm Message Limits.
    8. Under Message grouping, select Split notifications per metric stream.
      This option is required to receive a message for each metric stream.
    9. Select Repeat notification? and leave the default setting at 60 minutes.
      Messages are sent every hour as long as the alarm is in Firing state for one or more metric streams.
    10. Click Save alarm.
    Monitoring begins evaluating metrics for servers in the selected compartment, sending a single alarm message (per subscription) for each metric stream.

  • Create the alarm using the oci monitoring alarm create command. To configure the alarm for split messages by metric stream, set --is-notifications-per-metric-dimension-enabled to true.

    Example command
    oci monitoring alarm create 
--compartment-id "<compartment_OCID>"
--destinations ["<topic_OCID>"]
--display-name "Server-Specific Messages"
--is-enabled true
--is-notifications-per-metric-dimension-enabled true
--message-format "ONS_OPTIMIZED"
--metric-compartment-id "<metric_compartment_OCID>"
--namespace "<custom_metric_namespace>"
--query-text "MyCustomCPUMetric[1m].count() > 80"
--repeat-notification-duration "PT1H"
--resource-group "MyServerResourceGroup"
--severity "CRITICAL"

    For a complete list of flags and variable options for CLI commands, see the Command Line Reference.

  • Use the CreateAlarm operation to create the alarm. To configure the alarm for split messages by metric stream, set isNotificationsPerMetricDimensionEnabled to true.

    Example request
    POST /20180401/alarms
Host: telemetry.us-phoenix-1.oraclecloud.com
<authorization and other headers>
{
  "body": "Server exceeded CPU utilization threshold",
  "compartmentId": "<compartment_OCID>",
  "destinations": ["<topic_OCID>"],
  "displayName": "Server-Specific Messages",
  "isEnabled": true
  "isNotificationsPerMetricDimensionEnabled": true
  "messageFormat": "ONS_OPTIMIZED",
  "metricCompartmentId": "<metric_compartment_OCID>",
  "namespace": "<custom_metric_namespace>",
  "query": "MyCustomCPUMetric[1m].count() > 80",
  "repeatNotificationDuration": "PT1H",
  "resourceGroup": "MyServerResourceGroup",
  "severity": "CRITICAL"
}