Oracle Cloud Infrastructure Documentation

Paying for Dedicated AI Clusters

You get the following benefits for using dedicated AI clusters in OCI Generative AI:

  • Predictable pricing that doesn’t fluctuate with demand.
  • Great for fine-tuning or hosting models.
  • Minimum hosting commitment: 744 unit-hours per hosting cluster.
  • Minimum fine-tuning commitment: 1 unit-hour per fine-tuning job. (Fine-tuning requires 2 units to run).
Note

To find out which models are available for fine-tuning, see Matching Clusters to Base Models.

The following examples calculate dedicated AI cluster cost in OCI Generative AI. For calculating on-demand inferencing cost, see Paying for On-Demand Inferencing.

Hosting a Foundational Model Example 1

John wants to host an instance of the Cohere command (52 billion parameters) (cohere.command) model on dedicated infrastructure. John deletes the cluster after 40 days and wants to know cost of the cluster. To host a cohere.command model, John first needs to identify the unit size that can host the cohere.command model. The unit size for cohere.command model is a Large Cohere unit. See matching clusters to base models.

John needs a minimum of one Large Cohere unit to host the cohere.command model. Here are the steps to calculate the cost of a hosting cluster with one Large Cohere unit.

  1. Calculate the unit hours for 40 days. 
    40 days x 24 hours/day x 1 unit = 960 unit hours.
  2. Ensure that the unit hours exceed the minimum commitment for hosting the models. 
    960 unit hours > 744 minimum unit hours
  3. Go to AI Pricing and under OCI Generative AI, for Oracle Cloud Infrastructure Generative AI- Large Cohere - Dedicated, find the <Large-Cohere-dedicated-unit-per-hour-price>.
  4. Calculate the price for 40 days. 
    price = (960 unit hours) x $<Large-Cohere-dedicated-unit-per-hour-price>

Hosting a Foundational Model Example 2

Alice wants to host an instance of the Cohere command light (6 billion parameters) (cohere.command-light) model on dedicated infrastructure. To host a cohere.command-light model, Alice first needs to identify the unit size that can host the command light model. The unit size for command light is a Small Cohere unit. See matching clusters to base models.

Alice decides to buy three units of Small Cohere to handle a higher call volume to the model than a single unit would provide. Alice plans to delete the cluster after five days. Here are the steps to calculate the cost of a hosting cluster with three Small Cohere units for five days.

  1. Calculate the unit hours. 
    5 days x 24 hours/day x 3 units = 360 unit hours.
  2. Compare the unit hours to the minimum commitment for hosting the models. 
    360 unit hours < 744 minimum unit hours
Alice is charged for 744 unit hours.
  3. On the AI Pricing, under OCI Generative AI, for Oracle Cloud Infrastructure Generative AI- Small Cohere - Dedicated, find the <Small-Cohere-dedicated-unit-per-hour-price>.
  4. Calculate the cost for five days. 
    price = (744 unit hours) x $<Small-Cohere-dedicated-unit-per-hour-price>

Fine-Tuning and Hosting a Model Example

Bob wants to fine-tune a Cohere command (52 billion parameters) (cohere.command) model. Bob creates a fine-tuning dedicated AI cluster with the preset value of two Large Cohere units. Bob creates a custom model on the fine-tuning dedicated and fine-tunes a foundational model with training data. The fine-tuning job takes 5 hours to complete. Bob creates a fine-tuning cluster every week.

To host a cohere.command model, Bob needs to identify the unit size that can host the cohere.command model. The unit size for cohere.command model is a Large Cohere unit. See matching clusters to base models. Bob can host up to 50 fine-tuned models on a single hosting cluster. Here are the steps to calculate the monthly cost for fine-tuning and hosting the models.

  1. Calculate the unit hours for each fine-tuning. 
    Each fine-tuning cluster requires 2 units and each cluster is active for 5 hours
fine-tuning per cluster= 10 unit-hours
  2. Compare the unit hours to the minimum commitment for fine-tuning the models. 
    10 unit hours > 1 unit hour
  3. Calculate the unit hours for hosting. 
    31 days x 24 hours/day x 1 unit = 744 unit hours
  4. Compare the unit hours to the minimum commitment for hosting the models. 
    744 unit hours = 744 minimum unit hours
  5. Go to AI Pricing and under OCI Generative AI, for Oracle Cloud Infrastructure Generative AI- Large Cohere - Dedicated, find the <Large-Cohere-dedicated-unit-per-hour-price>.
  6. Find the total monthly price. 
    fine-tuning price = 
(10 unit hours)/week x (4 weeks) x $<Large-Cohere-dedicated-unit-per-hour-price>

fine-tuning price = 
40 x <Large-Cohere-dedicated-unit-per-hour-price>
    hosting price = 
(744 unit hours) x $<Large-Cohere-dedicated-unit-per-hour-price>
    total monthly price = 
(40 + 744 unit hours) x $<Large-Cohere-dedicated-unit-per-hour-price>