Writing Monitoring results to OCI Autonomous Data Warehouse(ADW)¶

Overview
ADW Authentication
Database Schema
SaveMetricToOracleADWPostProcessor Post Processor

Overview¶

ML Monitoring Application provides a post processor SaveMetricToOracleADWPostProcessor that writes the monitoring results(monitor details, metrics and test results) to a set of tables in an OCI ADW database.

ADW Authentication¶

SaveMetricToOracleADWPostProcessor post processor connects to an OCI ADW instance using mTLS connection. This requires the use of wallet files and database credentials which contain the required authentication information. SaveMetricToOracleADWPostProcessor post processor reads wallet files and database credentials from OCI Vault. The steps are as below:

Upload each wallet file to OCI Vault. This creates a secret for each file.

Write the appropriate policy to allow the principal (user/resource) to read secrets from OCI Vault.

Allow dynamic-group <ml-monitoring-application-dynamic-group> to read secret-bundles in compartment <COMPARTMENT NAME>

Add the following details in ML Monitoring Application Configuration file:

    "post_processors": [
    {
        "type": "SaveMetricToOracleADWPostProcessor",
        "params": {
         "auth_config": {
           "vault_ocid": "<vault_ocid>",
           "vault_tenancy_ocid": "<tanancy_ocid>"
         }
    }
]

vault_ocid: The OCID of the vault that contains the secret.
vault_tenancy_ocid: The OCID of the tenancy where vault resides

By default, SaveMetricToOracleADWPostProcessor post processor expects the following wallet files to be present in OCI Vault in Base64 encoded format:

[
    "cwallet.sso",
    "ewallet.pem",
    "ewallet.p12",
    "keystore.jks",
    "ojdbc.properties",
    "tnsnames.ora",
    "truststore.jks",
    "sqlnet.ora",
    "db_username",
    "db_password"
]

5. If the secret names in OCI Vault are not the same as the default file names, provide the names explicitly in the configuration. As shown below, the user has uploaded the cwallet.sso file as custom-cwallet.sso in OCI Vault. In this case, SaveMetricToOracleADWPostProcessor post processor is configured to read the custom wallet file names from OCI Vault.

      "post_processors": [
    {
        "type": "SaveMetricToOracleADWPostProcessor",
        "params": {
         "auth_config": {
           "vault_ocid": "<vault_ocid>",
            "vault_tenancy_ocid": "<tenancy_ocid>",
            "vault_secret_names": {
                "db_username": "custom_db_username",
                "db_password": "custom_db_password",
                "cwallet.sso": "custom-cwallet.sso",
                "ewallet.pem": "custom-ewallet.pem",
                "ewallet.p12": "custom-ewallet.p12",
                "keystore.jks": "custom-keystore.jks",
                "ojdbc.properties": "custom-ojdbc.properties",
                "tnsnames.ora": "custom-tnsnames.ora",
                "truststore.jks": "custom-truststore.jks",
                "sqlnet.ora": "custom-sqlnet.ora"
            }
         }
        }
    }
]

Database Schema¶

Before SaveMetricToOracleADWPostProcessor post processor can write monitoring results to ADW, create a set of tables in a new or existing database. A database user with appropriate permissions must run the following Data Definition Language (DDL) queries to create the required tables:

Monitor Table¶

CREATE TABLE "MONITOR_TABLE" (
    id VARCHAR2(64 CHAR) NOT NULL,
    monitor_id VARCHAR2(1024 CHAR) NOT NULL,
    monitor_run_id VARCHAR2(1024 CHAR) NOT NULL,
    monitor_display_id VARCHAR2(2048 CHAR) NOT NULL,
    action_type VARCHAR2(64 CHAR) NOT NULL,
    monitor_name VARCHAR2(64 CHAR) NOT NULL,
    baseline_monitor_id VARCHAR2(1024 CHAR),
    start_date DATE,
    end_date DATE,
    monitor_tags CLOB,
    time_created TIMESTAMP NOT NULL,
    time_updated TIMESTAMP NOT NULL,
    user_created VARCHAR2(1024 CHAR) NOT NULL,
    user_updated VARCHAR2(1024 CHAR) NOT NULL,
    PRIMARY KEY (id)
)

Metric Table¶

CREATE TABLE "METRIC_TABLE" (
    id VARCHAR2(64 CHAR) NOT NULL,
    monitor_id VARCHAR2(128 CHAR) NOT NULL,
    feature_name VARCHAR2(128 CHAR),
    metric_name VARCHAR2(128 CHAR) NOT NULL,
    metric_description CLOB,
    feature_params CLOB,
    metric_path VARCHAR2(1024 CHAR) NOT NULL,
    variable_name VARCHAR2(1024 CHAR) NOT NULL,
    variable_dtype VARCHAR2(64 CHAR) NOT NULL,
    variable_dimension INTEGER NOT NULL,
    metric_data CLOB NOT NULL,
    metric_metadata CLOB NOT NULL,
    error CLOB,
    PRIMARY KEY (id)
)

Test Results Table¶

CREATE TABLE "TEST_RESULT_TABLE" (
    id VARCHAR2(64 CHAR) NOT NULL,
    monitor_id VARCHAR2(64 CHAR) NOT NULL,
    test_name VARCHAR2(128 CHAR),
    feature_name VARCHAR2(128 CHAR),
    metric_key VARCHAR2(128 CHAR),
    selector_type VARCHAR2(128 CHAR),
    test_type VARCHAR2(128 CHAR) NOT NULL,
    test_operator VARCHAR2(128 CHAR) NOT NULL,
    test_config CLOB,
    test_assertion_expected VARCHAR2(128 CHAR),
    test_assertion_actual VARCHAR2(128 CHAR),
    test_status VARCHAR2(128 CHAR) NOT NULL,
    test_error CLOB,
    PRIMARY KEY (id)
)

SaveMetricToOracleADWPostProcessor Post Processor¶

Below is an example configuration to configure SaveMetricToOracleADWPostProcessor Post Processor in ML Monitoring Application configuration file

      "post_processors": [
    {
        "type": "SaveMetricToOracleADWPostProcessor",
        "params": {
         "auth_config": {
           "vault_ocid": "<vault_ocid>",
            "vault_tenancy_ocid": "<tenancy_ocid>",
            "vault_secret_names": {
                "db_username": "custom_db_username",
                "db_password": "custom_db_password",
                "cwallet.sso": "custom-cwallet.sso",
                "ewallet.pem": "custom-ewallet.pem",
                "ewallet.p12": "custom-ewallet.p12",
                "keystore.jks": "custom-keystore.jks",
                "ojdbc.properties": "custom-ojdbc.properties",
                "tnsnames.ora": "custom-tnsnames.ora",
                "truststore.jks": "custom-truststore.jks",
                "sqlnet.ora": "custom-sqlnet.ora"
            }
         },
         "monitor_table_name": "mlm-int-mlm_monitor",
        "metric_table_name": "mlm-int-mlm_metric",
        "test_result_table_name": "mlm-int-mlm_test_result"
        "monitor_tags":{"model_name":"bert"
                        "model_version":"1.0"
                        }
    }
]

monitor_table_name :Name of the the monitor table provided by the user which contains monitor run details such as monitor id, baseline/prediction run.
metric_table_name :Name of the the metric table provided by the user which contains metric results specific to a monitor run. This table has a parent child relationship with the monitor table using monitor id.
test_result_table_name :Name of the the metric table provided by the user which contains test results specific to a monitor run, This table has a parent child relationship with the monitor table using monitor id.
monitor_tags: User can provide the tags to a monitor run specified in the json format for a monitor run

Note

If the monitor table name monitor_table_name is missing, the post processor gives an exception during initialization.
If the metric table name metric_table_name and test result table name test_result_table_name are both missing, the post processor gives an exception during initialization.