Data Integration Metrics

Monitor the health, capacity, and overall performance and management of Data Integration instances by using metrics, alarms, and notifications.

This topic describes the metrics emitted by the metric namespace oci_dataintegration (the Data Integration service).

Resources: workspaces

Overview

The Data Integration metrics help you measure the number of objects stored in a Data Integration instance, the number of bytes read in GB from the sources, and the number of task runs.

Metrics terminology

Namespace: A container for Data Integration metrics. The namespace identifies the service sending the metrics. The namespace for Data Integration is oci_dataintegration.
Metrics: The fundamental concept in telemetry and monitoring. Metrics define a time-series set of datapoints. Each metric is uniquely defined by namespace, metric name, compartment identifier, and a set of one or more dimensions, and a unit of measure. Each datapoint has a timestamp, a value, and a count associated with it.
Dimensions: A key-value pair that defines the characteristics associated with the metric. For example, resourceId is the Data Integration workspace OCID.
Statistics: Metric data aggregations over specified periods of time. Aggregations are done using the namespace, metric name, dimensions, and the data point unit of measure within the time period specified.
Alarms: Used to automate operations monitoring and performance. An alarm tracks changes that occur over a specific period of time and performs one or more defined actions, based on the rules defined for the metric.

Prerequisites

OCI IAM with Identity Domains policies: To monitor resources, you must be given the required type of access in a policy written by an administrator, whether you're using the Console or the REST API with an SDK, CLI, or other tool. The policy must give you access to the monitoring services, and the resources being monitored. If you try to perform an action and get a message that you don't have permission or are unauthorized, confirm with the administrator the type of access you have been granted and which compartment to work in. For more information on user authorizations for monitoring, see the Authentication and Authorization section for the related service: Monitoring or Notifications.
The metrics listed on this page are automatically available for any Data Integration instance you create. You don't need to enable monitoring on the resource to get these metrics.

Available Metrics

Data Integration metrics can include the following dimensions:

APPLICATIONID: ID of the application where the task is published.
COMPARTMENTID: OCID of the compartment.
RESOURCEID: For BytesReadInGB, BytesWrittenInGB, PipelineRunDurationInHours , and WorkspaceUsageInHour, the RESOURCEID is the OCID of the workspace. For TaskRuns, the RESOURCEID is the Task ID or Task Key.
TASKID: The ID of the Task.
TASKNAME: The name of the Task.
TASKTYPE: The type of Task.
TASKSTATUS: The status of the Task Run.
WORKSPACEID: OCID of the workspace where the Task Run was executed.


Metric Name	Metric Display Name	Unit	Description	Dimensions
`BytesReadInGB`	BytesReadInGB	GigaBytes	Total number of bytes read in GB.	`resourceId` `workspaceId` `compartmentId`
`BytesWrittenInGB`	BytesWrittenInGB	GigaBytes	Total number of bytes written in GB.	`resourceId` `workspaceId` `compartmentId`
`TaskRuns`	TaskRuns	Count	Total number of Task Runs.	`resourceId` `workspaceId` `compartmentId` `applicationId` `taskId` `taskName` `taskType` `taskStatus`
`PipelineRunDurationInHours`	PipelineRunDurationInHours	Hours	Scheduled Task Runs and Scheduled Pipeline Task Runs duration in hours.	`resourceId` `workspaceId` `compartmentId`
`WorkspaceUsageInHours`	WorkspaceUsageInHours	Hours	Workspace Usage in hours.	`resourceId` `workspaceId` `compartmentId`

Note

For Oracle Database, the BytesWrittenInGB does not display any data.

Using the Console

Here's how you view metric charts:

In the Console navigation menu, select Observability & Management.
Under Monitoring, select Service Metrics.
For Compartment, select the compartment that contains the Data Integration workspaces that you're interested in.
For Metric namespace, select oci_dataintegration.

The Service Metrics page dynamically updates the page to show charts for each metric that's emitted by the selected metric namespace.

Using the API

For information about using the API and signing requests, see REST API documentation and Security Credentials. For information about SDKs, see SDKs and the CLI (Software Development Kits and Command Line Interface).

Use the following APIs for monitoring:

Monitoring API for metrics and alarms
Notifications API for notifications (used with alarms)