Creating a Model Deployment with Autoscaling

• Console
• CLI
• API

• 1. From the model deployments page, select Create model deployment. If you need help finding the list of model deployments, see Listing Model Deployments.
  2. Follow the steps in Creating a Model Deployment to configure the model deployment.
  3. Under Autoscaling configuration, select Enable autoscaling.
     Several lists and fields are displayed to let you configure the autoscaling.
  4. (Optional) Update the values in each list or field as appropriate for the configuration.
  5. (Optional) To autoscale the load balancer, select Show advanced options.
     Set the value of maximum bandwidth to be more than the minimum bandwidth value, and no more than twice the minimum bandwidth value.
  6. Select Create.

• Use the oci data-science model-deployment create command and required parameters to create a model deployment:

  Command

  CopyTry It

  oci data-science model-deployment create --required-param-name variable-name ... [OPTIONS]

  For example, create a deployment with:

  Copy

  oci data-science model-deployment create \
  --compartment-id <MODEL_DEPLOYMENT_COMPARTMENT_OCID> \
  --model-deployment-configuration-details file://<MODEL_DEPLOYMENT_CONFIGURATION_FILE> \
  --project-id <PROJECT_OCID> \
  --display-name <MODEL_DEPLOYMENT_NAME>

  Use this model deployment JSON configuration file with this example:

  Copy

  {
    "deploymentType": "SINGLE_MODEL",
    "modelConfigurationDetails": {
      "modelId": "ocid1.datasciencemodel....",
      "instanceConfiguration": {
        "instanceShapeName": "VM.Standard.E4.Flex",
        "modelDeploymentInstanceShapeConfigDetails": {
          "ocpus": 1,
          "memoryInGBs": 16
        }
      },
      "scalingPolicy": {
        "policyType": "AUTOSCALING",
        "coolDownInSeconds": 600,
        "isEnabled": true,
        "autoScalingPolicies": [
          {
            "autoScalingPolicyType": "THRESHOLD",
            "initialInstanceCount": 1,
            "maximumInstanceCount": 2,
            "minimumInstanceCount": 1,
            "rules": [
              {
                "metricExpressionRuleType": "PREDEFINED_EXPRESSION",
                "metricType": "CPU_UTILIZATION",
                "scaleInConfiguration": {
                  "scalingConfigurationType": "THRESHOLD",
                  "threshold": "10"
                },
                "scaleOutConfiguration": {
                  "scalingConfigurationType": "THRESHOLD",
                  "threshold": "65"
                }
              }
            ]
          }
        ]
      },
      "bandwidthMbps": 10,
      "maximumBandwidthMbps": 20
    }
  }

  For a complete list of parameters and values for CLI commands, see the CLI Command Reference.

• Use the CreateModelDeployment operation to create a model deployment.