Best Practices for Model Artifacts

Use these extra considerations for model artifact creation and packaging.

Writing a score.py File

Always ensure that the score.py and runtime.yaml files are in the top-level directory of a model artifact.

Any other files that need to be part of a artifact must be at the same level as those two files or in directories under them:
```
.
|-- runtime.yaml
|-- score.py
|-- <your-serialized-models>
```
Model deployment uses the score.py functions to load a model into memory and to make predictions.
The function definitions, load_model() and predict(), aren't editable. Only the body of these functions is customizable.
The allowed path to write data to disk when using model deployment service is /home/datascience.
You can access Object Storage using resource principals when the OCI identity permissions are defined correctly to enable it.

Packaging Custom Modules

Any custom module on which score.py or the serialized model depends on should be written as separate Python scripts in the same top level directory as score.py or under. For example, model.joblib depends on a custom DataFrameLabelEncoder class, which is defined in the dataframelabelencoder.py script as in this example:

from category_encoders.ordinal import OrdinalEncoder
from collections import defaultdict
 
from sklearn.base import TransformerMixin
from sklearn.ensemble import RandomForestClassifier
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import LabelEncoder
 
class DataFrameLabelEncoder(TransformerMixin):
    def __init__(self):
        self.label_encoders = defaultdict(LabelEncoder)
         
    def fit(self, X):
        for column in X.columns:
            if X[column].dtype.name  in ["object", "category"]:
                self.label_encoders[column] = OrdinalEncoder()
                self.label_encoders[column].fit(X[column])
        return self
     
    def transform(self, X):
        for column, label_encoder in self.label_encoders.items():
            X[column] = label_encoder.transform(X[column])
        return X

The module is then imported by the score.py file:

"""
   Inference script. This script is used for prediction by scoring server when schema is known.
"""
 
import json
import os
from joblib import load
import io
import pandas as pd
 
from dataframelabelencoder import DataFrameLabelEncoder
 
def load_model():
    """
    Loads model from the serialized format
 
    Returns
    -------
    model:  a model instance on which predict API can be invoked
    """
    model_dir = os.path.dirname(os.path.realpath(__file__))
    contents = os.listdir(model_dir)
    model_file_name = "model.joblib"
    # TODO: Load the model from the model_dir using the appropriate loader
    # Below is a sample code to load a model file using `cloudpickle` which was serialized using `cloudpickle`
    # from cloudpickle import cloudpickle
    if model_file_name in contents:
        with open(os.path.join(os.path.dirname(os.path.realpath(__file__)), model_file_name), "rb") as file:
            model = load(file) # Use the loader corresponding to your model file.
    else:
        raise Exception('{0} is not found in model directory {1}'.format(model_file_name, model_dir))
     
    return model
 
 
def predict(data, model=load_model()) -> dict:
    """
    Returns prediction given the model and data to predict
 
    Parameters
    ----------
    model: Model instance returned by load_model API
    data: Data format as expected by the predict API of the core estimator. For eg. in case of sckit models it could be numpy array/List of list/Panda DataFrame
 
    Returns
    -------
    predictions: Output from scoring server
        Format: { 'prediction': output from `model.predict` method }
 
    """
    assert model is not None, "Model is not loaded"
    X = pd.read_json(io.StringIO(data)) if isinstance(data, str) else pd.DataFrame.from_dict(data)
    preds = model.predict(X).tolist()  
    return { 'prediction': preds }

In the preceding example, the artifact structure should be:


.
|-- score.py 
|-- dataframelabelencoder.py 
|-- model.joblib 
|-- runtime.yaml

Changing load_model() and predict() signatures

The predict(data, model=load_model()) function expects the payload data and a model object and is returned by load_model() by default. Your use case might require that an extra parameter is passed to predict(). An example is a scaler or a lookup table. You can add parameters to those function if the parameter you add has a default value assigned.

In the following example, the predictions rely on a PCA model and a scaler object. It shows you how predict() can take an extra parameter called scaler. By default, the load_scaler() function returns a value to the scaler parameter. We recommend that you follow that pattern. If predict() or load_model() require extra parameters, they must be set to default values returned by functions that are defined in score.py. This example adds parameters to predict():

import json
import os
from cloudpickle import cloudpickle
model_pickle_name = 'pca.pkl'
scaler_pickle_name = 'scaler.pkl'
"""
   Inference script. This script is used for prediction by scoring server when schema is known.
"""
def load_model(model_file_name=model_pickle_name):
    """
    Loads model from the serialized format
    Returns
    -------
    model:  a model instance on which predict API can be invoked
    """
    model_dir = os.path.dirname(os.path.realpath(__file__))
    contents = os.listdir(model_dir)
    if model_file_name in contents:
        with open(os.path.join(os.path.dirname(os.path.realpath(__file__)), model_file_name), "rb") as file:
            return cloudpickle.load(file)
    else:
        raise Exception('{0} is not found in model directory {1}'.format(model_file_name, model_dir))
 
def load_scaler(model_file_name=scaler_pickle_name):
    """
    Loads model from the serialized format
    Returns
    -------
    model:  a model instance on which predict API can be invoked
    """
    model_dir = os.path.dirname(os.path.realpath(__file__))
    contents = os.listdir(model_dir)
    if model_file_name in contents:
        with open(os.path.join(os.path.dirname(os.path.realpath(__file__)), model_file_name), "rb") as file:
            return cloudpickle.load(file)
    else:
        raise Exception('{0} is not found in model directory {1}'.format(model_file_name, model_dir))
 
def predict(data, model=load_model(), scaler=load_scaler()):
    """
    Returns prediction given the model and data to predict
    Parameters
    ----------
    model: Model instance returned by load_model API
    data: Data format as expected by the predict API of the core estimator. For eg. in case of sckit models it could be numpy array/List of list/Panda DataFrame
    Returns
    -------
    predictions: Output from scoring server
        Format: {'prediction':output from model.predict method}
    """
    from pandas import read_json, DataFrame
    from io import StringIO
    X = read_json(StringIO(data)) if isinstance(data, str) else DataFrame.from_dict(data)
    X_s = scaler.transform(X)
    return {'prediction':model.transform(X_s).tolist()[0]}

Testing a Model Artifact Before Saving

Before saving a model to the catalog, we recommend that you test the artifact thoroughly. This code snippet to tests a model artifact before saving it to the catalog:

Changes the Python path by inserting the path to the model artifact.
Loads the model into memory using load_model().
Lastly, calls predict().

Before you run the code snippet, create a new notebook file in a notebook session, change the kernel, and then select the same conda environment that you want to use for model deployment (inference conda environment). Copy and paste the code snippet and run the code.

Model Artifact Test Code Example

import sys
from json import dumps
 
 
# The local path to your model artifact directory is added to the Python path.
# replace <your-model-artifact-path>
sys.path.insert(0, f"<your-model-artifact-path>")
 
# importing load_model() and predict() that are defined in score.py
from score import load_model, predict
 
# Loading the model to memory
_ = load_model()
# Making predictions on a JSON string object (dumps(data)). Here we assume
# that predict() is taking data in JSON format
predictions_test = predict(dumps(data), _)
predictions_test

The predictions_test method contains the predictions made by the model on the sample data JSON string payload. Compare predictions_test against a known model outcome for a particular dataset. For example, data could be a sample of the training dataset.

Image Predictions Example

The score.py predict() method handles the processing from the received request as in this example score.py file:

"""
   Inference script. This script is used for prediction by scoring server when schema is known.
"""
import torch
import torchvision
import io
import numpy as np
from PIL import Image
import os
 
 
# COCO Labels
COCO_INSTANCE_CATEGORY_NAMES = [
    '__background__', 'person', 'bicycle', 'car', 'motorcycle']
 
 
model_name = 'PyTorch_Retinanet.pth'
def load_model(model_file_name = model_name):
    """
    Loads model from the serialized format
 
    Returns
    -------
    model:  Pytorch model instance
    """
    model = torchvision.models.detection.retinanet_resnet50_fpn(pretrained=False, pretrained_backbone=False)
    cur_dir = os.path.dirname(os.path.abspath(__file__))
    model.load_state_dict(torch.load(os.path.join(cur_dir, model_file_name)))
    model.eval()
    return model
 
 
def predict(data, model=load_model()):
    """
    Returns prediction given the model and data to predict
 
    Parameters
    ----------
    model: Model instance returned by load_model API
    data: Data format in json
 
    Returns
    -------
    predictions: Output from scoring server
        Format: {'prediction':output from model.predict method}
 
    """
    img_bytes = io.BytesIO(data)
    image = Image.open(img_bytes)
    image_np = np.asarray(image)
    image_th = torch.from_numpy(image_np)
    image_th = image_th.permute(2, 0, 1)
    image_th = image_th.unsqueeze(0) / 255
    with torch.no_grad():
        pred = model(image_th)
    object_index_list = np.argwhere(pred[0].get("scores") > 0.5)
    label_index_list = pred[0].get("labels")
    labels = [COCO_INSTANCE_CATEGORY_NAMES[label_index_list[i]] for i in object_index_list]
    box_list = pred[0].get("boxes")
    boxes = [box_list[i].numpy().tolist() for i in object_index_list][0]
    return {'prediction': {
        'labels': labels,
        'boxes': boxes,
    }}