Configuring a Data Science Connector

Prerequisites 🔗

• To use a Data Science connector with OCI Search with OpenSearch, you need a cluster configured to use OpenSearch version 2.11 or newer. By default, new clusters are configured to use version 2.11. To create a cluster, see Creating an OpenSearch Cluster.

  For existing clusters configured for version 2.3, you can perform an inline upgrade to version 2.11. For more information, see OpenSearch Cluster Software Upgrades.

  To upgrade existing clusters configured for version 1.2.3 to 2.11, you need to use the upgrade process described in OpenSearch Cluster Software Upgrades.

• If the OpenSearch cluster is in a different tenancy than the Data Science endpoint tenancy, you need to create policies in both tenancies to grant access to Data Science resources.

  The following policy examples includes the required permissions. To use these examples, replace <caller_tenancy_name> with the name of the tenancy for the OpenSearch cluster, and replace <resource_host_tenancy_name> with the name of the tenancy for the Data Science endpoint.

  • Policy for Data Science tenancy:
    Copy

    define tenancy <caller_tenancy_name> as <caller_tenancy_ocid>
    admit any-user of tenancy <caller_tenancy_name> to {DATA_SCIENCE_MODEL_DEPLOYMENT_PREDICT} in tenancy

  • Policy for OpenSearch cluster tenancy:
    Copy

    define tenancy <resource_host_tenancy_name> as <resource_host_tenancy_ocid>
    endorse any-user to {DATA_SCIENCE_MODEL_DEPLOYMENT_PREDICT} in tenancy <resource_host_tenancy_name>

  If you're new to policies, see Getting Started with Policies and Common Policies.

1: Configure Cluster Settings 🔗

Use the settings operation of the Cluster APIs to configure the recommended cluster settings, as follows:

PUT _cluster/settings
{
  "persistent": {
    "plugins": {
      "ml_commons": {
        "only_run_on_ml_node": "false",
        "model_access_control_enabled": "true",
        "native_memory_threshold": "99",
        "rag_pipeline_feature_enabled": "true",
        "memory_feature_enabled": "true",
        "allow_registering_model_via_local_file": "true",
        "allow_registering_model_via_url": "true",
        "model_auto_redeploy.enable":"true",
        "model_auto_redeploy.lifetime_retry_times": 10
      }
    }
  }
}

2: Create the Connector 🔗

Create the Data Science connector using the following payload:

POST _plugins/_ml/connectors/_create
{
   "name": "DS Chat Connector",
   "description": "Check errors in logs",
   "version": 2,
   "protocol": "oci_sigv1",
   "parameters": {
       "endpoint": "<model_deployment_endpoint>",
        "modelId": "<model_deployment_OCID>",
       "auth_type": "resource_principal"
   },
   "credential": {
   },
   "actions": [
       {
           "action_type": "predict",
           "method": "POST",
           "url": "https://${parameters.endpoint}/${parameters.modelId}/predict",
           "request_body": "{\"model\": \"/opt/ds/model/deployed_model\", \"prompt\" : \"${parameters.prompt}\", \"max_tokens\":250,  \"temperature\": 0.7, \"top_p\":0.8, \"stream\" : false}",
           "post_process_function": "def text = params['choices'][0]['text'].replace('\n', '\\\\n');\n return '{\"name\":\"response\",\"dataAsMap\":{\"inferenceResponse\":{\"generatedTexts\":[{\"text\":\"' + text + '\"}]}}}'"
       }
   ]
}

Replace <model_deployment_endpoint> and <model_deployment_OCID> with the actual values for your Data Science model deployment, for example:

...
"endpoint": "modeldeployment.us-ashburn-1.oci.customer-oci.com"
"modelId": "ocid1.datasciencemodeldeployment.oc1.IAD.examplesmtpsuqmoy4m5cvblu..."
...

Make note of the <connector_ID> returned in the response:

{
  "connector_id": "<connector_ID>",
}

3: Register the Model Group 🔗

Register a model group for the connector using the register operation in the Model Group APIs, as shown in the following example:

POST /_plugins/_ml/model_groups/_register
{
   "name": "<model_group_name>",
   "description": "<model_group_description>"
}

Make note of the <model_group_ID returned in the response:

{
  "model_group_id": "<model_group_ID>",
  "status": "CREATED"
}

4: Register the Model 🔗

Register the model using the <model_group_ID> and <connector_ID> from the preview step, as shown in the following example:

POST /_plugins/_ml/models/_register
{
   "name": "oci-genai-test",
   "function_name": "remote",
   "model_group_id": "<model_group_ID>",
   "description": "test model",
   "connector_id": "<connector_ID>"
 }

5: Deploy the Model 🔗

Deploy the model, as shown in the following example:

POST /_plugins/_ml/models/<model_ID>/_deploy

6: Create a RAG Pipeline 🔗

Create a RAG pipeline using the <model_ID> from the previous step, as shown in the following example:

PUT /_search/pipeline/<pipeline_name>
{
  "response_processors": [
    {
      "retrieval_augmented_generation": {
        "tag": "genai_pipeline_demo",
        "description": "Demo pipeline Using Genai Connector",
        "model_id": "<model_ID>",
        "context_field_list": ["<text_field_name>"],
        "system_prompt": "You are a helpful assistant",
        "user_instructions": "Generate a concise and informative answer for the given question"
      }
    }
  ]
}

You can specify one or more text field names for "context_field_list", separate the values with a comma, for example:

"context_field_list" : ["value1", "value2",…,"valueN"],

Values specified for "context_field_list" must match a field or fields of interest in the target document index.

Next Steps 🔗