Creating a Job

Create and run a job in Data Science.

Ensure that you have created the necessary policies, authentication, and authorization for your jobs.

Before you begin:

Create a job artifact file.
If you want to store and manage job logs, learn about logging.

When creating jobs, you can use the fast launch enabled Compute shapes when they're available in the region. This launches the job in the fastest way possible.

Use the Console to sign in to a tenancy with the necessary policies.
Open the navigation menu and click Analytics & AI. Under Machine Learning, click Data Science.
Select the compartment that contains the project with the model deployments.

All projects in the compartment are listed.
Click the name of the project.

The project details page opens and lists the notebook sessions.
Under Resources, click Jobs.

A tabular list of jobs in the project is displayed.
Click Create job.
(Optional) Select a different compartment for the job.
(Optional) Enter a unique name and description for the job (limit of 255 characters). If you don't provide a name, a name is automatically generated.

For example, job20210808222435.

Use one of the following options:

Option	Description
Bring your own container	Select to use your container, and then specify the image as a container with the custom environment variable keys.
Upload job artifact	Drag the job artifact file into the box.

A container image or a job artifact is required for a job.

(Optional) Create a default job configuration that's used when the job is run using these options.

Enter or select any of the following values:

Custom environment variable key

The environment variables that control the job. When you're using a container image, you must specify CONTAINER_CUSTOM_IMAGE, and other options are optional.

Note

If you uploaded zip file or compressed tar file, add the JOB_RUN_ENTRYPOINT as a custom environment variable to point to the file.

Value

The value for the custom environment variable key.

You can click Additional custom environment key to specify more variables.

Command line arguments

The command line arguments that you want to use for running the job.

Maximum runtime (in minutes)

The maximum number of minutes that the job can run. The service cancels the job run if its runtime exceeds the specified value. The maximum runtime is 30 days (43,200 minutes). We recommend that you configure a maximum runtime on all job runs to prevent runaway job runs.
Select a Compute shape, either Fast launch or Custom configuration.

A shape is automatically selected.

We recommend that you use the fast launch option that automatically selects a Compute shape from a predefined pool of shapes so that the job can start as fast as possible. There must be a shape available in the pool for this option to be selected.
(Optional) Change the Compute shape by clicking Change shape. Then, follow these steps in the Select compute panel.
1. Select an instance type.
2. Select an shape series.
3. Select one of the supported Compute shapes in the series.
4. Select the shape that best suits how you want to use the resource. For the AMD shape, you can use the default or set the number of OCPUs and memory.
  
  For each OCPU, select up to 64 GB of memory and a maximum total of 512 GB. The minimum amount of memory allowed is either 1 GB or a value matching the number of OCPUs, whichever is greater.
5. Click Select shape.
(Optional) To use logging, click Select, and then ensure that Enable logging is selected.
1. Select a log group from the list. You can change to a different compartment to specify a log group in a different compartment from the job.
2. Select one of the following to store all stdout and stderr messages:
  
  Enable automatic log creation
  
  Data Science automatically creates a log when the job starts.
  
  Select a log
  
  Select a log to use.
3. Click Select to return to the job run creation page.
For Storage. enter the amount of block storage to use between 50 GB and 10, 240 GB (10 TB). You can change the value by 1 GB increments. The default value is 100 GB.
Select one of the following options to configure the network:
- Default networking—The workload is attached by using a secondary VNIC to a preconfigured, service-managed VCN, and subnet. This provided subnet allows egress to the public internet through a NAT gateway, and access to other Oracle Cloud services through a service gateway.
  
  If you need access only to the public internet and OCI services, we recommend using this option. It doesn't require you to create networking resources or write policies for networking permissions.
- Custom networking—Select the VCN and subnet that you want to use for the resource (notebook session or job).
  
  For egress access to the public internet, use a private subnet with a route to a NAT gateway.
  
  If you don't see the VCN or subnet that you want to use, click Change Compartment, and then select the compartment that contains the VCN or subnet.
Note

The network configuration of a notebook session can't be changed when the notebook session is reactivated. If a notebook session was configured to use a default network, you can't deactivate the notebook session then select custom networking when you reactivate it.
(Optional) Click Show advanced options to add tags to the job.
(Optional) Enter the tag namespace (for a defined tag), key, and value to assign tags to the resource.

To add more than one tag, click Add tag.

Tagging describes the various tags that you can use organize and find resources including cost-tracking tags.
Click Create.

After the job is in an active state, you can use job runs to repeatedly run the job.

These environment variables control the job.

Tip

You can use the ListFastLaunchJobConfigs API to retrieve the configurations that are fast launch capable.

Use the Data Science CLI to create a job as in this example:

Create a job with:

oci data-science job create \
--display-name <job_name>\
--compartment-id <compartment_ocid>\
--project-id <project_ocid> \
--configuration-details file://<jobs_configuration_json_file> \
--infrastructure-configuration-details file://<jobs_infrastructure_configuration_json_file> \
--log-configuration-details file://<optional_jobs_infrastructure_configuration_json_file>

Use this jobs configuration JSON file:

{
  "jobType": "DEFAULT",
  "maximumRuntimeInMinutes": 240,
  "commandLineArguments" : "test-arg",
  "environmentVariables": {
    "SOME_ENV_KEY": "some_env_value" 
  }
}

Use this jobs infrastructure configuration JSON file:

{
  "jobInfrastructureType": "STANDALONE",
  "shapeName": "VM.Standard2.1",
  "blockStorageSizeInGBs": "50",
  "subnetId": "<subnet_ocid>"
}

(Optional) Use this jobs logging configuration JSON file:

{
  "enableLogging": true,
  "enableAutoLogCreation": true,
  "logGroupId": "<log_group_ocid>"
}

Upload a job artifact file for the job you created with:

oci data-science job create-job-artifact \
--job-id <job_ocid> \
--job-artifact-file <job_artifact_file_path> \
--content-disposition "attachment; filename=<job_artifact_file_name>"

The ADS SDK is also a publicly available Python library that you can install with this command:
```
pip install oracle-ads
```
It provides the wrapper that makes the creation and running jobs from notebooks or on your client machine easy.

Use the ADS SDK to create and run jobs.

Tip

You can use the ListFastLaunchJobConfigs API to retrieve the configurations that are fast launch capable.

Oracle Cloud Infrastructure Documentation

Creating a Job