Use Custom Networking
Create a model deployment with the custom networking option.
The workload is attached by using a secondary VNIC to a customer-managed VCN and subnet. The subnet can be configured for egress to the public internet through a NAT/Internet gateway.
allow service datascience to use virtual-network-family in compartment <subnet_compartment>
For custom egress, the subnet must have at least 255 IP addresses available.
You can create and run custom networking model deployments using the Console, the OCI Python SDK, the OCI CLI, or the Data Science API.
You can use the OCI CLI to create a model deployment as in this example.
Use the CreateModelDeployment operation to create a model deployment with custom networking.
Using the OCI Python SDK
We've developed an OCI Python SDK model deployment example that includes authentication.
Artifacts that exeed 6 GB aren't supported for deployment. Select a smaller model artifact for deployment.
You must upgrade the OCI SDK to version 2.33.0 or later before creating a deployment with the Python SDK. Use the following command:
pip install --upgrade oci
Use this example to create a model deployment that uses a custom container:
# create a model configuration details object
model_config_details = ModelConfigurationDetails(
model_id=<model-id>,
bandwidth_mbps=<bandwidth-mbps>,
instance_configuration=<instance-configuration>,
scaling_policy=<scaling-policy>
)
# create the container environment configiguration
environment_config_details = OcirModelDeploymentEnvironmentConfigurationDetails(
environment_configuration_type="OCIR_CONTAINER",
environment_variables={'key1': 'value1', 'key2': 'value2'},
image="iad.ocir.io/testtenancy/ml_flask_app_demo:1.0.0",
image_digest="sha256:243590ea099af4019b6afc104b8a70b9552f0b001b37d0442f8b5a399244681c",
entrypoint=[
"python",
"/opt/ds/model/deployed_model/api.py"
],
server_port=5000,
health_check_port=5000
)
# create a model type deployment
single_model_deployment_config_details = data_science.models.SingleModelDeploymentConfigurationDetails(
deployment_type="SINGLE_MODEL",
model_configuration_details=model_config_details,
environment_configuration_details=environment_config_details
)
# set up parameters required to create a new model deployment.
create_model_deployment_details = CreateModelDeploymentDetails(
display_name=<deployment_name>,
model_deployment_configuration_details=single_model_deployment_config_details,
compartment_id=<compartment-id>,
project_id=<project-id>
)
Notebook Examples
We have provided various notebook examples that show you how to train, prepare, save, deploy, and invoke model deployments.