Model Deployment Environment Variables
When you create a model deployment resource, you can provide custom environment variables as part of the default configuration. These environment variables are available for use inside the container at runtime.
Format constraint
- Key must be under 32 characters.
- Key must contain only letters, digits, and underscore (_).
- Key must start with a letter.
- Key must have at least 2 characters.
- Key must not end with underscore. For example, TEST_.
- Key, if added, can't be empty. Value can be empty.
- Value could be any character.
- Key can't be service-provided environment variables.
- If ports are passed as an environment variable, then limit them between 1024 and 65535 and also exclude ports 24224, 8446, 8447.
Size constraint
Individual values don't have specific size limits. The overall environment variables including key value pairs is limited to 2048 bytes. If the size is more than 2048 bytes, specify the environment variables by using the model artifact or use OCI Object Storage to pull the data.
Service Provided Environment Variables
The service has reserved environment variables inside the container at runtime for logging or other functional use. These environment variables can't be overridden.
| Name | Custom Container | Default Container |
|---|---|---|
MODEL_DEPLOYMENT_OCID |
Model deployment resource OCID. | Model deployment resource OCID. |
MODEL_SERVING_PYTHON_VERSION |
None. | The Python version from runtime.yaml used for model serving. |
TENANCY_OCID |
Tenancy OCID. | Tenancy OCID. |
CONTAINER_CUSTOM_IMAGE |
Container image path (with tag or digest) that's provided through the API environment configuration object. | None. |
Application Environment Variables
These environment variables functionally control how the model deployment is created and can be overridden or configured by the customers by passing in the default configuration.
| Name | Custom Container | Managed Container |
|---|---|---|
WEB_CONCURRENCY |
Based on available memory and size of model artifact, a heuristic method calculates the number of workers needed to run inside the custom container. Customers can also override the variable or pass a custom concurrency variable as required. | Based on available memory and size of model artifact, a heuristic method calculates the number of workers needed to run inside the custom container. Customers can also override the variable or pass a custom concurrency variable as required. |
CONTAINER_TYPE |
Type of container viz TRITON. Only TRITON is supported. | None. |