To create Scheduled Tasks, first set up right permissions by creating the following IAM policies:

1. Create a dynamic group to allow Scheduled Tasks to post metrics to the monitoring service from specific compartment:

   ALL {resource.type='loganalyticsscheduledtask', resource.compartment.id='<compartment ocid>'}

   Alternatively, to allow metrics to be posted from all compartments:

   ALL {resource.type='loganalyticsscheduledtask'}

2. Create policies to allow the dynamic group to perform Scheduled Task operations in tenancy:

   allow group <group_name> to use loganalytics-scheduled-task in tenancy
   allow dynamic-group <dynamic_group_name> to use metrics in tenancy
   allow dynamic-group <dynamic_group_name> to read management-saved-search in tenancy
   allow dynamic-group <dynamic_group_name> to {LOG_ANALYTICS_QUERY_VIEW} in tenancy
   allow dynamic-group <dynamic_group_name> to {LOG_ANALYTICS_QUERYJOB_WORK_REQUEST_READ} in tenancy
   allow dynamic-group <dynamic_group_name> to READ loganalytics-log-group in tenancy
   allow dynamic-group <dynamic_group_name> to read compartments in tenancy

• Some of the above policy statements are included in the readily available Oracle-defined policy templates. You may want to consider using the template for your use case. See Oracle-defined Policy Templates for Common Use Cases.

• For information about dynamic groups and IAM policies, see OCI Documentation: Managing Dynamic Groups and OCI Documentation: Managing Policies.

• For the policy details, see Building Metric Queries - Prerequisites in Oracle Cloud Infrastructure Documentation.

• For the Scheduled Tasks API reference, see ScheduledTask Reference in Oracle Cloud Infrastructure API Documentation.

To view the scheduled tasks for a specific saved search, you can visit the saved search details page. However, if you want to list all the scheduled tasks in a specific compartment without reference to the saved searches for which the scheduled tasks were created for, then use the API to query for listing the scheduled tasks. See ListScheduledTasks.

Specify the following parameters in your GET command:

• taskType=SAVED_SEARCH
• compartmentId=<compartment_OCID>
• limit=1000
• sortOrder=DESC
• sortBy=timeUpdated

To run the command, you will need:

• Namespace: The Logging Analytics namespace that you specified while creating the scheduled tasks.
• Compartment OCID: The OCID of the compartment that you want to query for the list of scheduled tasks created in it.

You can monitor the health of your saved search scheduled tasks through the metrics Scheduled Task Execution Status. In case of failed or skipped execution of a task due to infrastructure anomaly or if a dependent resource or configuration is modified, the metric provides details of failure to help you to rectify it.

For steps to access the Scheduled Task Execution Status metric, see Monitor Logging Analytics Using Service Metrics.

Each saved search scheduled task has its own interval as specified in its task schedule. A metric is emitted to your tenancy for every scheduled task execution. Hover the cursor on the data points on the chart to view more details about the task. Follow these steps when you want to filter the metric data based on one of the dimensions Status, DisplayName, or ResourceId:

1. Click the Options menu on the top right corner of the Scheduled Task Execution Status metric, and select View Query in Metric Explorer.

   The metric is now displayed in the Metrics Explorer. Here, you can view the chart in finer detail.

2. Click Edit Queries and select Dimension Name and Dimension Value for the metric. You can filter the metric data based on taskResult the result of running the scheduled task, Status of task execution, the DisplayName of the task, the queryExecTimeRange, or its ResourceId.

   Note
   
   

   To view charts and tabular data from Metrics Explorer by specifying a dimension name and a dimension value, avoid using fields that have parentheses or other special characters in the name. If the field selected for the Dimension Name has special characters, then create a virtual field by using eval command or rename the existing field by using the rename command such that the parentheses or special characters are removed. For example, if the field used for Dimension Name is Host Name (Server), then you can create a virtual field hostname with | eval hostname=“Host Name (Server)”.

   The dimension queryExecTimeRange is useful in determining the time taken to run your scheduled task query. The values available are < 5s, >= 5s and < 10s, >= 10s and < 30s, and > 30s. Typically, the queries that take more than 30 seconds to run are considered expensive in terms of execution time. See How to Make Your Queries Performant.

   The dimension taskResult can have values Succeeded, Failed, and Paused. The dimension Status provides further details of taskResult. For example, if the value of taskResult is Paused, then Status value may be Paused by User.

   Click Update Chart to refresh the chart visualization. The chart will now display only the data points for which you applied the filter.

   You can switch to the Data Table view for a tabular representation of the collected data points.

3. Change the dimension name to view different perspectives in the chart.

You can set up alerts to notify you about the status through email, SMS, Slack, PagerDuty, HTTPS endpoint URL, or Function. See Create Alerts for Detected Events.

The following are the various values of status dimension reported through this metric for specific taskResult values:

Make note of the following factors for creating scheduled tasks:

• Requirements for Composing Queries:

  When you compose queries to create Scheduled Tasks, ensure to comply with the following requirements:

  • Note these limitations for detection rule queries:

    • Refrain from performing wild card search on the field Original Log Content in your scheduled task query. For more information on wild card searches, see Use Keywords, Phrases, and Wildcards.

    • timestats command cannot be followed by eval, extract, jsonextract, xmlextract and lookup.

    • The command regex must not be used on large fields like Message to avoid making the queries expensive for processing.

      like comparison and extract, jsonextract, xmlextract commands are not supported on large fields like Message.

      Link fields or fields used in BY clause cannot be used on large fields like Message.

    • The commands which are not supported in the queries for scheduled tasks are addfields, cluster, clustercompare, clusterdetails, clustersplit, compare, createview, delta, eventstats, fieldsummary, highlightgroups, geostats, linkdetails, map, nlp and timecompare.

  • Maximum limits:

    Maximum number of fields supported for the by clause is 3.

    Maximum number of fields supported for timestats command is 3.

    Maximum number of supported aggregate functions in a scheduled task query is 1.

  • Use the values of link fields as dimensions for posting metrics:

    Select up to three dimension fields and one numeric metric to post to monitoring service. To indicate which fields must be posted to monitoring, the queries must end with:

    ... | link ... | fields -*, dim1, dim2, dim3, metric1

    link command has several columns in the output such as Start Time, End Time, Count etc, by default. Use -* in the fields command to remove these fields and specify optionally up to three dimension fields and one mandatory metric field.

    You can have multiple eval statements after stats command and multiple stats functions for computing intermediate results. However, the query must end with fields -*, dim1, dim2, dim3, metric1 indicating what dimensions and metric must be posted. Use the following guidelines for detection rule queries:

    • Use up to 2 addfields command.
    • Use up to 3 stats functions.
    • eval statements are required for computing intermediate and final results.

    Examples queries:

    'Log Source' = 'OCI Email Delivery'
     | link 'Entity'
     | addfields [ * | where deliveryEventType = r and bounceType = hard | stats count as 'hard bounces' ],
                 [ * | where deliveryEventType = e and length(ipPoolName) > 0 | stats count as 'total sent messages' ]
     | eval 'Total Rate' = ('hard bounces' / 'total sent messages') * 100
     | fields -*, 'Entity', 'Total Rate'

    'Log Source' = 'My Network Logs'
     | stats sum(Success) as TotalSuccess, sum(Failure) as TotalFailure
     | eval SuccessRate = (TotalSuccess / (TotalSuccess + TotalFailure)) * 100 | fields -*, SuccessRate

• Late Arrival of Logs:

  If the scheduled tasks run before the arrival of the logs, then the scheduled tasks may not return the results as expected. To avoid missing such logs in the scheduled tasks due to their late arrival, the query must account for it by using an adjustment to the time range.

  For example, if the scheduled task runs every 5 minutes to check for the number of authentication errors and if there is a 3 minute delay between the time the logs are generated and the time they reach Oracle Logging Analytics, then the scheduled task will not detect the logs. Consider that the scheduled task runs every 5 minutes at say, 01:00, 01:05, 01:10, and so on. If the log record L1 that gets generated at 01:04 reaches Oracle Logging Analytics at 01:07. L1 is not detected in the scheduled task that executed at 1:05 because the log did not arrive at Oracle Logging Analytics at this time. During the next execution at 01:10, the query looks for logs with timestamps between 01:05 and 01:10. In this cycle too, L1 is not detected because it has a timestamp of 01:04. The following query may not see all the logs records if the logs arrive late:

  Label = 'Authentication Error' | stats count as logrecords by 'Log Source'

  To determine the delay in the logs arrival into Oracle Logging Analytics, calculate the difference between the timestamp mentioned in the log record and the Log Processor Posting Time. The following example query can be used to check if there is a delay:

  Label = 'Authentication Error' and 'Log Processor Posting Time (OMC INT)' != null | fields 'Agent Collection Time (OMC INT)', 'Data Services Load Time', 'Process Time', 'Log Processor Posting Time (OMC INT)'

  The following query uses the dateRelative function to adjust for the 3 minute delay in a task that runs at 5 minute interval:

  Label = 'Authentication Error' and Time between dateRelative(8minute, minute) and dateRelative(3minute, minute) | stats count as logrecords by 'Log Source'

• Other Factors:

  • To understand how queries are built in Monitoring service, see Building Metric Queries in Oracle Cloud Infrastructure Documentation.

  • Note the limits information for publishing the metrics data to the Monitoring service. The limits correspond to the metrics for one scheduled task. See PostMetricData API in Oracle Cloud Infrastructure Documentation.

    When your saved search can generate more than 50 unique by field values, partial results are posted due to the limits imposed by the Monitoring service. In such cases, to view the top or bottom 50 results, use the sort command.