Oracle Cloud Infrastructure Documentation

Copying a Conda Environment to Another Region

In this tutorial, you install a conda environment in a notebook session. Then you perform the necessary steps to install the same conda environment in a second notebook session hosted in a different region.

Key tasks include how to:

  • Create two buckets in two different regions in your tenancy.
  • Register each bucket with a notebook session in their region.
  • Install a conda environment in the first notebook session.
  • Publish the conda environment. (Make the conda environment ready to be installed from any notebook session that has access to the environment.)
  • From the first notebook session, add the published conda environment to a bucket of the same region.
  • Copy the published conda environment from one bucket to another.
  • In a notebook session in the second region, install the conda environment, by using the second bucket.

For additional information, see:

Before You Begin

To successfully perform this tutorial, you must have the following:

Requirements

  • A paid Oracle Cloud Infrastructure (OCI) account, or a new account with Oracle Cloud promotions. See Request and Manage Free Oracle Cloud Promotions.

  • A MacOS, Linux, or Windows computer.
  • Data Science tutorial:
    • Go through all the steps in Manually Configuring a Data Science Tenancy and create two notebook sessions with the following properties:
      • Name the compartment for your work data-science-work.
      • Create notebook sessions in two regions:
        • To change your region, in the Console's top navigation bar, click your region and then click the name of a new region. For example, switch from US West (Phoenix) to US East (Ashburn).
        • Name your projects and notebook sessions:
          • In <region-1>: Initial Project and my-first-notebook-session
          • In <region-2>: Second Project and my-second-notebook-session

1. Create Buckets

Create two buckets to store conda environments of my-first-notebook-session and my-second-notebook-session.

Create Two Buckets

Create a bucket in <region-1> and another one in <region-2>.

  1. Open the navigation menu and click Storage. Under Object Storage & Archive Storage, click Buckets.
  2. In the left navigation, under List Scope, select the data-science-work compartment.
  3. In the top navigation bar, select <region-1>. For example, US West (Phoenix).
  4. Click Create Bucket.
  5. Fill in the following information:
    • Name: bucket-1
    • Leave the default value for other fields:
      • Default Storage Tier: Standard
      • Encryption: Encrypt using Oracle managed keys
  6. Click Create.
  7. In the list of buckets, ensure that bucket-1 has a Private Visibility:
    Buckets are private, unless you change their visibility after you create them.
  8. In the top navigation bar, click <region-1>, and from the list of regions, click <region-2>. For example, switch to US East (Ashburn).
  9. Repeat the previous steps to create bucket-2 in <region-2>.
You have successfully created two private buckets in two different regions.
Get the Tenancy Namespace

Every tenancy has a unique system-generated immutable Object Storage namespace name. This namespace name is the same in all regions. In this step, you copy this namespace name and later use it to register your buckets with your notebook sessions.

  1. Open the navigation menu and click Storage. Under Object Storage & Archive Storage, click Buckets.
  2. In the left navigation, under List Scope, select the data-science-work compartment.
  3. In the top navigation bar, select <region-2>. For example, US East (Ashburn).
  4. In the list of buckets, click bucket-2.
  5. Copy the Namespace in a notepad: <tenancy-namespace>.
    Note

    Namespace names are unique within a tenancy. If bucket-1 and bucket-2 are in different tenancies, then copy the namespace names for both buckets.

2. Publish a Conda Environment in Region 1

Install a conda environment in your notebook session. Then from the notebook session, publish the conda environment to a bucket.

Register bucket-1

Register bucket-1 with my-first-notebook-session.

  1. In the Console's top navigation bar, select <region-1>.
  2. In the left navigation, under List Scope, select the data-science-work compartment.
  3. Open the navigation menu and click Analytics & AI. Under Machine Learning, click Data Science.
  4. Under the list of projects, click Initial Project.
  5. Click my-first-notebook-session and then click Open.
  6. Enter your credentials to access the JupyterLab interactive notebook.
  7. In the UI, if you don't have a tab called Launcher, click File, and then New Launcher.
  8. In the Launcher, under Other, click the Terminal icon to start a new terminal session.
  9. In the terminal, enter the following command.

    Replace <tenancy-namespace> with the information that you gathered in the Create Buckets section.

    odsc conda init -b bucket-1 -n <tenancy-namespace>
  10. Go to File, and then click Open from Path.... In the Open Path field, enter /conda.
  11. To confirm that the preceding command registered your bucket with your notebook, under the conda folder, click config.yaml and find bucket-1 and your namespace in the following block of code: 
    auth_mode: resource_principal
bucket_info:
  name: bucket-1
  namespace: <tenancy-namespace>
pack_prefix: conda_environments
Install a Conda Environment

Install a prebuilt Data Science conda environment in your notebook session.

  1. In my-first-notebook-session, click File and then New Launcher.
  2. In the Launcher, under Extensions, click the Environment Explorer icon.
  3. In the list of Conda Environments, expand ONNX 1.10 for CPU on Python 3.9.
  4. Copy the command to install ONNX 1.10 for CPU on Python 3.9, and paste it into a terminal: 
    odsc conda install -s onnx110_p39_cpu_v1
    • For version number, keep the default displayed value, such as 1.0.
    • Example output:
      Version number [1.0]? 
Environment slug: onnx110_p39_cpu_v1
INFO:ODSC:Verifying that /home/datascience/conda/onnx110_p39_cpu_v1 exists.
INFO:ODSC:Downloading conda pack onnx110_p39_cpu_v1...
INFO:ODSC:Writing to /home/datascience/.onnx110_p39_cpu_v1.tar.gz
Downloading pack onnx110_p39_cpu_v1: 100% ...INFO:ODSC:download complete
INFO:ODSC:Extracting conda pack /home/datascience/.onnx110_p39_cpu_v1.tar.gz...
INFO:ODSC:Downloading Notebooks for the pack: ONNX 1.10 for CPU on Python 3.9
INFO:ODSC:Checking for notebooks with prefix notebooks/onnx110_p39_cpu_v1/
Saving Notebooks:   0%|                                                                                       | 0/1 [00:00<?, ?it/s]INFO:ODSC:Notebook location /home/datascience/conda/notebooks/onnx110_p39_cpu_v1/getting-started.ipynb...
Saving Notebooks: 100%| 1/1 [00:00<00:00,  8.28it/s]
INFO:ODSC:Start to update the kernel name...
INFO:ODSC:Conda environment setup complete.
The environment setup is complete. 
To activate it for use in the terminal run `conda activate /home/datascience/conda/onnx110_p39_cpu_v1`. 
It may take a few seconds for the kernel to appear in the JupyterLab launcher. 
To change the description of the environment, update /home/datascience/conda/onnx110_p39_cpu_v1/*_manifest.yaml.
  5. Verify that your package is installed: 
    odsc conda list -l

    Example output:

    packages:
  local_ONNX_1.10_for_CPU_on_Python_3.9_1.0:
  - arch_type: CPU
    conda_name: onnx110_p39_cpu_v1
    description: 'This environment is designed to provided to test and execute your
      ONNX model artifacts. ONNX is an open source, open model format which allows
      you to save a model from different machine learning (ML) libraries into a single,
      portable format that is independent of the training library. ONNX models can
      be deployed through Oracle Cloud Infrastruture Data Science Model Deployment
      service. Use this conda environment to convert models from most ML libraries
      into ONNX format. Then use the ONNX runtime to perform inferencing. Review the
      processing steps that your model makes by having ONNX generate a graph of the
      model workflow.

      To get started with the ONNX environment, review the getting-started notebook.

      '
    libraries:
    - onnx (v1.10.2)
    - onnxconverter-common (v1.9.0)
    - onnxmltools (v1.10.0)
    - onnxruntime (v1.10.0)
    - oracle-ads (v2.6.1)
    logo: onnx
    manifest_version: '1.0'
    name: ONNX 1.10 for CPU on Python 3.9
    notebooks:
    - notebooks/onnx110_p39_cpu_v1/
    pack_path: oci://service-conda-packs@id19sfcrra6z/service_pack/cpu/ONNX_1.10_for_CPU_on_Python_3.9/1.0/onnx110_p39_cpu_v1
    python: '3.9'
    slug: onnx110_p39_cpu_v1
    type: local
    version: '1.0'
  6. In the Environment Explorer, click the refresh icon.
  7. Click the Installed Conda Environments link.
  8. Under the list of Conda Environments, confirm that ONNX 1.10 for CPU on Python 3.9 is listed.
  9. Now, click File and then New Launcher. In the Kernels section, find your published conda environment, ONNX 1.10 for CPU on Python 3.9 listed as an icon.
You have successfully installed a conda environment in your notebook session in <region-1>.
Publish the Conda Environment

Publish the conda environment in a bucket registered with the notebook session.

  1. In my-first-notebook-session, in the Environment Explorer, click Installed Conda Environments.
  2. Expand ONNX 1.10 for CPU on Python 3.9.
  3. Copy the command to publish ONNX 1.10 for CPU on Python 3.9:
  4. Copy the command to publish ONNX 1.10 for CPU on Python 3.9 and paste the publish command in a terminal in your notebook session and click Enter. 
    odsc conda publish -s onnx110_p39_cpu_v1

    Example output:

    INFO:ODSC:Saving onnx110_p39_cpu_v1
INFO:ODSC:Loading environment information from /home/datascience/conda/onnx110_p39_cpu_v1/ONNX_1.10_for_CPU_on_Python_3.9_manifest.yaml.
INFO:ODSC:Overwriting manifest file at /home/datascience/conda/onnx110_p39_cpu_v1/ONNX_1.10_for_CPU_on_Python_3.9_manifest.yaml with latest dependency information
/opt/conda/lib/python3.8/site-packages/conda_pack/core.py:56:
...
INFO:ODSC:Saving conda environment to object storage: {'name': 'bucket-1', 'namespace': '<tenancy-namespace>'}
INFO:ODSC:The upload id is <some-id>.
...
INFO:ODSC:/home/datascience/conda/tmp/onnx110_p39_cpu_v1.tar.gz uploaded successfuly..
INFO:ODSC:Conda env saved...
  5. In the Environment Explorer, click refresh.
  6. Click the Published Conda Environments link.
  7. Under the list of Conda Environments, confirm that ONNX 1.10 for CPU on Python 3.9 is listed.
You have successfully published a conda environment to a bucket in <region-1>.

3. Copy the Bucket to Region 2

Copy your published conda environment from bucket-1 to bucket-2.

Copy Bucket Content

Your published conda environment is stored as an object in bucket-1. Copy this object to bucket-2.

  1. Open the navigation menu and click Storage. Under Object Storage & Archive Storage, click Buckets.
  2. In the left navigation, under List Scope, select the data-science-work compartment.
  3. In the top navigation bar, select <region-1>. For example, US West (Phoenix).
  4. Click bucket-1.
  5. Under Objects, expand conda_environments, cpu, ONNX 1.10 for CPU on Python 3.9, and 1.0 to get to onnx110_p39_cpu_v1.
  6. Click the Actions menu (three dots) for onnx110_p39_cpu_v1 and then click Copy.
  7. Fill in the following information:
    • Destination Namespace: <tenancy-namespace>
    • Destination Region: <region-2>, for example, US East (Ashburn).
    • Destination Bucket: bucket-2
    • Destination Object Name: To preserve the directory structure, keep the default string.
    • Leave the default value for other fields:
      • Destination Storage Tier: (leave blank)
      • Overwrite Rule: Overwrite destination object
  8. Click Copy Object.
  9. If you get a warning section for not having the correct IAM policy statements, click Try to add statements. 
    Allow service objectstorage-<region-1> to manage object-family in compartment data-science-work

    If you don't get this warning message, then skip the next three steps.

  10. In the Add IAM Policy panel, keep the defaults.
    • Name: console-iam-policy-<timestamp>
    • Description: IAM policy statements created by the Console on <timestamp>
    • Add policy statement to the root compartment: not selected
    • Policy Statements:
      Allow service objectstorage-<region-1> to manage object-family in compartment data-science-work
  11. (Optional) To allow objects in other regions to be copied to your compartment, you can add the following policy to the Policy Statements section in the Add IAM Policy panel
    Note

    To allow objects in other regions to be copied to your compartment, for each region, in a new line, add the following policy to the Policy Statements section in the Add IAM Policy panel:
    allow service objectstorage-<region_name> to manage object-family in compartment data-science-work
    For the <region_name> variable, some example values are:
    • objectstorage-us-phoenix-1
    • objectstorage-us-ashburn-1
    • objectstorage-eu-frankfurt-1
    • objectstorage-uk-london-1
    • objectstorage-ap-tokyo-1

    where object-storage-<region_name> contains the source bucket and performs the copy operation.

    For a complete list of region names, see the table in the Regions and Availability Domains page.

    For more details on copying objects to other regions, see Service Permissions.

  12. Click Create.
  13. Wait until you get the message: Policies added successfully.
  14. Click Copy Object.
  15. In the Work Request Details dialog, confirm that the Status shows Completed, with a Percent Complete of 100%. Then Close the dialog.
Verify Content in Second Bucket

Verify that your published conda environment is stored as an object in bucket-2.

  1. In the top navigation bar, select <region-2>. For example, US East (Ashburn).
  2. Click bucket-2.
  3. Under Objects, expand conda_environments, cpu, ONNX 1.10 for CPU on Python 3.9, and 1.0 to get to onnx110_p39_cpu_v1.
You have successfully copied a published conda environment from bucket-1 to bucket-2.

4. Install the Conda Environment in Region 2

Register bucket-2 to a notebook session in the second region. Then install and view the conda environment in the notebook session.

Register bucket-2

Register bucket-2 with my-second-notebook-session in <region-2>.

  1. In the Console's top navigation bar, select region-2.
  2. Open the navigation menu and click Analytics & AI. Under Machine Learning, click Data Science.
  3. In the left navigation, under List Scope, select the data-science-work compartment.
  4. Under the list of projects, click Second Project.
  5. Click my-second-notebook-session, and then click Open.
  6. Enter your credentials to access the JupyterLab UI.
  7. If you don't have a tab called Launcher, then click File, New Launcher.
  8. In the Launcher, click Terminal.
  9. In the terminal, enter the following command.

    Replace <tenancy-namespace> with the information you gathered in the Create Buckets section.

    odsc conda init -b bucket-2 -n <tenancy-namespace>
  10. Confirm that your bucket is registered with your notebook:

    In the file explorer section, under the conda folder, click config.yaml and review the file for the following code:

    auth_mode: resource_principal
bucket_info:
  name: bucket-2
  namespace: <tenancy-namespace>
pack_prefix: conda_environments
Install the Published Conda Environment

View the copied conda environment in your second notebook session.

  1. In my-second-notebook-session, click File and then New Launcher.
  2. In the Launcher, under Kernels, click the Environment Explorer icon.
  3. Click the Published Conda Environmentslink.
  4. In the list of Conda Environments, expand ONNX 1.10 for CPU on Python 3.9.
    Note

    Having a published conda environment in your notebook session means that:
    • The conda environment is available to install in your notebook session.
    • The published conda environment might not be installed yet.
    • If you want to use the conda environment, you can install it from here.
    • The published conda environment remains in the list of published conda environments after you install it.
  5. Copy the command to install ONNX 1.10 for CPU on Python 3.9, and paste it into a terminal: 
    odsc conda install -s onnx110_p39_cpu_v1
    • For version number, keep the default displayed value, such as 1.0.
    • Example output:
      Version number [1.0]? 
Environment slug: onnx110_p39_cpu_v1
INFO:ODSC:Verifying that /home/datascience/conda/onnx110_p39_cpu_v1 exists.
INFO:ODSC:Downloading conda pack onnx110_p39_cpu_v1...
INFO:ODSC:Writing to /home/datascience/.onnx110_p39_cpu_v1.tar.gz
Downloading pack onnx110_p39_cpu_v1: 100% ...INFO:ODSC:download complete
INFO:ODSC:Extracting conda pack /home/datascience/.onnx110_p39_cpu_v1.tar.gz...
INFO:ODSC:Downloading Notebooks for the pack: ONNX 1.10 for CPU on Python 3.9
INFO:ODSC:Checking for notebooks with prefix notebooks/onnx110_p39_cpu_v1/
Saving Notebooks:   0%|                                                                                       | 0/1 [00:00<?, ?it/s]INFO:ODSC:Notebook location /home/datascience/conda/notebooks/onnx110_p39_cpu_v1/getting-started.ipynb...
Saving Notebooks: 100%| 1/1 [00:00<00:00,  8.28it/s]
INFO:ODSC:Start to update the kernel name...
INFO:ODSC:Conda environment setup complete.
The environment setup is complete. 
To activate it for use in the terminal run `conda activate /home/datascience/conda/onnx110_p39_cpu_v1`. 
It may take a few seconds for the kernel to appear in the JupyterLab launcher. 
To change the description of the environment, update /home/datascience/conda/onnx110_p39_cpu_v1/*_manifest.yaml.
  6. In a terminal, verify that your package is installed: 
    odsc conda list -l

    Example output:

    packages:
  local_ONNX_1.10_for_CPU_on_Python_3.9_1.0:
  - arch_type: CPU
    conda_name: onnx110_p39_cpu_v1
    description: 'This environment is designed to provided to test and execute your
      ONNX model artifacts. ONNX is an open source, open model format which allows
      you to save a model from different machine learning (ML) libraries into a single,
      portable format that is independent of the training library. ONNX models can
      be deployed through Oracle Cloud Infrastruture Data Science Model Deployment
      service. Use this conda environment to convert models from most ML libraries
      into ONNX format. Then use the ONNX runtime to perform inferencing. Review the
      processing steps that your model makes by having ONNX generate a graph of the
      model workflow.

      To get started with the ONNX environment, review the getting-started notebook.

      '
    libraries:
    - onnx (v1.10.2)
    - onnxconverter-common (v1.9.0)
    - onnxmltools (v1.10.0)
    - onnxruntime (v1.10.0)
    - oracle-ads (v2.6.1)
    logo: onnx
    manifest_version: '1.0'
    name: ONNX 1.10 for CPU on Python 3.9
    notebooks:
    - notebooks/onnx110_p39_cpu_v1/
    pack_path: oci://bucket-2@<tenancy-namespace>/conda_environments/cpu/ONNX 1.10 for CPU on
      Python 3.9/1.0/onnx110_p39_cpu_v1
    python: '3.9'
    slug: onnx110_p39_cpu_v1
    type: local
    version: '1.0'
  7. In the Environment Explorer, click refresh.
  8. Click the Installed Conda Environments link.
  9. Under the list of Conda Environments, confirm that ONNX 1.10 for CPU on Python 3.9 is listed.
You have successfully installed a conda environment from a published copy in bucket-2 in your notebook session.

What's Next

You have successfully copied a conda environment from one region to another, and used it in a notebook session in the second region.

To learn more about conda environments, in your notebook session:

  • In the Launcher, under Extenstions, click the Notebook Explorer icon. Select ONNX Integration with the Accelerated Data Science (ADS) SDK, click Open, and explore the examples.
  • Go to Environment Explorer and install other prebuilt Data Science conda environments, not mentioned in this tutorial.