Bring Your Own Models

If you have models you want to use instead of the service cached models provided by Data Science, you can bring them into AI Quick Actions from Object Storage or from Hugging Face by registering the model.

Hugging Face is an open source model repository. You can bring in models from here to use in AI Quick Actions. Hugging Face offers certain gated models that require the acceptance of user agreement. To bring a gated model from Hugging Face into AI Quick Actions, sign in to Hugging Face using the Hugging Face CLI and your Hugging Face token from a terminal inside the Notebook. This is to verify your access to the model. See the Hugging Face guides to see how to sign in with the Hugging Face CLI. If you don't have a Hugging Face token, see this Hugging Face article on security tokens to generate one. If you try to register a gated model which you haven't been granted access to in Hugging Face or fail to sign in with the Hugging Face CLI, the registration process fails.

For more information, see a video on importing a Hugging Face model to AI Quick Actions.

There are two ways to register a model:

A service verified model is one the Data Science service has tested the configurations for deployment and fine tuning.

Note

The difference between a service cached model and a verified model is that, for a verified model, you must register the model in AI Quick Actions before using it.

Service Managed Inference Containers

Four inference containers are available to use with Bring Your Own Model.

For cached and verified models, Data Science has tested which inference container works best with each model and so the inference container can't be chosen. For unverified models, you must decide which inference container is most suitable for each model. Four service managed inference containers are available:

for models compatible with inference engine vLLM 0.7.1
for models compatible with TGI 2.0.1
for models compatible with inference framework llama.cpp 0.3.2 (for models in GGUF format)
for models compatible with inference framework Test Embeddings Inference (TEI)

for models in the GGUF format.

Download Model Artifacts

Before you can bring a model from Object Storage and register it in AI quick Actions, you need to download the model artifacts, such as scoring logic, ML model, and dependent files, from the model source and upload them to a versioned OCI Object Storage bucket. If you don't have a versioned Object Storage bucket, follow these steps to create one. Then you can upload the model artifacts to Object Storage.

Register Service Verified Models

Data Science has models you can select to use that have been tested.

Follow the steps in Prerequisites.

Steps 3 and 4 show different ways of registering a model. Model registration is a necessary process for a model to be brought into AI Quick Actions.
1. Select Models if it's not already shown.
2. Opt to import a new model from Object Storage or Hugging Face to register, or to use a service verified model:
  
  To import a new model, select My models.
  
  To use a service verified model, select a model card with the Ready-to-register tag.
3. If in step 2 you selected My models:
  
  Select Import new model.
  
  Under Model artifact, select one of:
  
  Download from Hugging Face to download a model from Hugging Face.
  
  I have artifacts in Object storage to download a model from Object Storage.
  
  Select Register service verified model to select a model that's been tested by Oracle Data Science for deployment and fine-tuning.
  
  From the Select model list, select the model name.
  
  From the Select compartment list, select the compartment of the Object Storage bucket.
  
  From the Object storage location list, select a bucket.
  
  If downloading from Object Storage, it's the bucket where the model artifact is stored.
  
  If downloading from Hugging Face, it's the bucket where the model artifact is downloaded.
  
  Provide a directory path for the object storage.
  
  Select Register to register the model. When the model registration finishes, the Model Information screen is displayed. The model is included in the list of models under My models.
  
  Select Fine-tune to fine tune the model.
  
  Select Deploy to deploy the model.
4. If in step 2 you selected a model card with the Ready-to-register tag:
  
  Select register in the upper right corner of model information page.
  
  Under Model artifact, select one of:
  
  Download from Hugging Face to download a model from Hugging Face.
  
  I have artifacts in Object storage to download a model from Object Storage.
  
  From the Select compartment list, select the compartment of the Object Storage bucket..
  
  From the Object storage location list, select a bucket.
  
  If downloading from Object Storage, it's the bucket where the model artifact is stored.
  
  If downloading from Hugging Face, it's the bucket where the model artifact is downloaded.
  
  Provide a directory path for the object storage.
  
  Select Register to register the model. When the model registration finishes, the Model Information screen is displayed. The model is included in the list of models under My models.
  
  Select Fine-tune to fine tune the model.
  
  (Optional) Under Advanced options, enter the file or file format to exclude in the Exclusions list.
  
  Select Deploy to deploy the model.
For a complete list of parameters and values for AI Quick Actions CLI commands, see AI Quick Actions CLI.
This task can't be performed using the API.

Register Any Model

Follow these steps to use models that haven't been tested by Data Science.

Follow the steps in Prerequisites.

Model registration is a necessary process for a model to be brought into AI Quick Actions.
1. Select Models if it's not already shown.
2. Select My models.
3. Select Import new model.
4. Under Model artifact, select one of:
  
  Download from Hugging Face to download a model from Hugging Face.
  
  I have artifacts in Object storage to download a model from Object Storage.
5. Select Register any model to register a model.
6. In Model name, give the model a name with which to register it.
7. Under Inference Container, select a container from the list to use for inferencing.
  
  To select TEI as the inference container:
  
  Push the container image to OCIR.
  
  Enter the URI of the OCIR image in the model registration page.
  
  For details on how to prepare the container and push the image to OCIR, see Register an Embedding Model.
8. From the Select compartment list, select the compartment of the Object Storage bucket.
9. From the Object storage location list, select the bucket where:
  
  If downloading from Object Storage, the model artifact is stored.
  
  If downloading from Hugging Face, the model is downloaded to.
10. Provide a directory path for the object storage.
11. (Optional) To fine-tune the model, check Enable Fine tuning.
12. Select Register to register the model.
  When the model registration finishes, the Model Information screen is displayed. The model is included in the list of models under My models.
13. Select Fine-tune to fine tune the model.
14. (Optional) Under Advanced options, enter the file or file format to exclude in the Exclusions list.
15. Select Deploy to deploy the model.
For a complete list of parameters and values for AI Quick Actions CLI commands, see AI Quick Actions CLI.
This task can't be performed using the API.

Oracle Cloud Infrastructure Documentation Try Free Tier

Bring Your Own Models

Service Managed Inference Containers 🔗

Download Model Artifacts 🔗

Register Service Verified Models 🔗

Register Any Model 🔗

Oracle Cloud Infrastructure Documentation
Try Free Tier

Service Managed Inference Containers

Download Model Artifacts

Register Service Verified Models

Register Any Model