Oracle Cloud Infrastructure Documentation

ML Monitoring

Learn about ML Monitoring in Data Science.

Data Science ML Monitoring lets you:
  • Read data from Object Storage using the built-in data readers.
  • Extend the library to create a data reader.
  • Transform data using Conditional Features to add more depth to the data.
ML Monitoring supports distributed processing and CSV, JSON, and JSON-L data types. It's compute engine-agnostic, it supports pandas, dask, and spark.

If you don't want to add a code extension use the service managed offering of the ML Monitoring Application container running in an ML Job. Otherwise extend the code of ML Insights Library SDK to add a custom reader, metrics, or a post-processor.

ML Insights

Use ML Insights to quickly evaluate the data to decide on the ML Monitoring use cases. You can set up long running monitoring process to continuously evaluate models and data.

Configurable metrics for monitoring include:
  • Data Integrity
  • Data Quality or Summary
  • Feature and Prediction Drift Detection
  • Model Performance for both classification and regression models
ML Insights is extendable with:
  • Custom metrics
  • Conditional features and transformers
  • Data readers
  • Post processing
  • Tests and Test Suites
For more information, see the ML Insights SDK documentation. Sample files are available from the Data Science repository on GitHub.

ML Monitoring Application

The ML Monitoring Application is a service-managed container running the ML Insights library inside an ML Job. Provide the monitoring configuration as a one-time set up and run it many times using ML Job Run. Runs can also be scheduled.
  • It's integrated with ML Jobs.
  • It uses an ML Insights configuration file as its input.
  • It can be run as a baseline (using golden or training data), as a prediction (using deployed or inference data), or as a validation.
  • (Optional) It can operate on data for a specified date range.
  • It outputs profiles containing metrics from the data.
For more information, see the ML Monitoring Application documentation.