Oracle Cloud Infrastructure Documentation

Model Deployment Metrics

Learn about how to use metrics for model deployments.

Metrics are automatically available for any Data Science model deployments that you create in the oci_datascience_modeldeploy namespace. You don't need to enable monitoring on OCI resources to get these metrics.

Data Science model deployments metrics include these dimensions:

resourceId

The OCID of the model deployment.

statusCode

The HTTP response status code.

result

Result of response:

  • Success

  • Failure

statusFamily

Status family of result:

  • Success: 2XX

  • Failure: 4XX and 5XX

instanceId

The ID of instance.

networkType

Types of network:

  • BytesIn: Network receive throughput

  • BytesOut: Network transmission throughput

Metric Names Display Name Unit Description Dimensions
PredictRequestCount Predict Request Count Count Number of predict requests.

resourceId
PredictResponse Predict Response Success Rate Percentage Predict response success rate. It's calculated based on the number of successful predict requests out of total number of predict requests.

resourceId

statusCode

result

statusFamily
Predict Response Status Count Predict response result and status code.
PredictLatency Predict Latency Milliseconds Latency of predict calls.

resourceId

result
PredictBandwidth Predict Bandwidth Utilization Percentage

Predict provisioned and consumed bandwidth. Provisioned bandwidth is the customer expected bandwidth set during the model deployment creation. Consumed bandwidth is the active bandwidth consumed by all ongoing predict requests for a specific model deployment OCID. This bandwidth is computed as current consumed bandwidth relative to the total provisioned load balancer bandwidth on every predict request.

resourceId
CpuUtilization CPU Utilization Percentage Activity level from the CPU.

resourceId

instanceId
MemoryUtilization Memory Utilization Percentage Memory in use.

resourceId

instanceId
NetworkBytes Network Receive and Transmit Bytes Bytes/sec Network receive and transmission throughput.

resourceId

instanceId

networkType
GPUCoreUtilization GPU Utilization Percentage Shows how busy the GPU devices are in a model deployment cluster.

resourceId

instanceId
GPUMemoryUtilization GPU Memory Utilization Percentage Shows the mean memory consumption of all GPU devices in a model deployment cluster.

resourceId

instanceId

Viewing Model Deployment Metrics

You can view the default metric charts using a model deployment details page in the Data Science service.

  1. Open the navigation menu and select Analytics & AI. Under Machine Learning, select Data Science.
  2. Select the region you are using with Data Science.
  3. Select the compartment that contains the project of the model deployment that you want to view the metrics for.
  4. Select the name of the project associated with the model deployment that you want to view the metrics for.
  5. Select the name of the model deployment that you want to view metrics for.
  6. Under Resources, select Metrics.

    The Metrics area displays a chart for each metric that's emitted to the metric namespace for Data Science.

For more information about the emitted metrics, see model deployment metrics.

If you don't see the metrics data for the model deployment session that you expect, see missing metrics data.