
Paying for Dedicated AI Clusters

Paying for Dedicated AI Clusters
You get the following benefits for using dedicated AI clusters in OCI
Generative AI:
Predictable pricing that doesn’t fluctuate with demand.
Great for fine-tuning or hosting models.
Minimum hosting commitment: 744 unit-hours per hosting cluster.
Minimum fine-tuning commitment: 1 unit-hour per fine-tuning job. (Fine-tuning requires 2 units to run). 
 Note To find out which models are available for fine-tuning, see Matching Clusters to Base Models.
The following examples calculate dedicated AI cluster cost in OCI
Generative AI. For calculating on-demand inferencing
 cost, see Paying for On-Demand Inferencing.
Hosting a Foundational Model Example 1
John wants to host an instance of the Cohere
 command (52 billion parameters)
 (cohere.command) model on dedicated infrastructure. John
 deletes the cluster after 40 days and wants to know cost of the cluster. To host a
 cohere.command model, John first needs to identify the unit
 size that can host the cohere.command model. The unit size for
 cohere.command model is a Large
 Cohere unit. See matching
 clusters to base models.
 John needs a minimum of one Large Cohere unit to host the
 cohere.command model. Here are the steps to calculate the
 cost of a hosting cluster with one Large Cohere unit.
Calculate the unit hours for 40 days.
40 days x 24 hours/day x 1 unit = 960 unit hours.
Ensure that the unit hours exceed the minimum commitment for hosting the models.
960 unit hours > 744 minimum unit hours
Go to AI Pricing and under OCI Generative AI, for Oracle Cloud Infrastructure Generative AI- Large Cohere - Dedicated, find the <Large-Cohere-dedicated-unit-per-hour-price>.
Calculate the price for 40 days.
price = (960 unit hours) x $<Large-Cohere-dedicated-unit-per-hour-price>
Related conceptsModel Limitations in Generative AIRelated informationAI PricingCost estimator
Hosting a Foundational Model Example 2
Alice wants to host an instance of the Cohere
 command light (6 billion parameters)
 (cohere.command-light) model on dedicated infrastructure. To
 host a cohere.command-light model, Alice first needs to identify
 the unit size that can host the command light model. The unit size for command light
 is a Small Cohere unit. See matching clusters to base
 models.
Alice decides to buy three units of Small Cohere to handle a higher call volume to the model than a single unit would provide. Alice plans to delete the cluster after five days. Here are the steps to calculate the cost of a hosting cluster with three Small Cohere units for five days.
Calculate the unit hours.
5 days x 24 hours/day x 3 units = 360 unit hours. 
Compare the unit hours to the minimum commitment for hosting the models.
360 unit hours < 744 minimum unit hours
Alice is charged for 744 unit hours.
On the AI Pricing, under OCI Generative AI, for Oracle Cloud Infrastructure Generative AI- Small Cohere - Dedicated, find the <Small-Cohere-dedicated-unit-per-hour-price>.
Calculate the cost for five days.
price = (744 unit hours) x $<Small-Cohere-dedicated-unit-per-hour-price>
Related conceptsModel Limitations in Generative AIRelated informationAI PricingCost estimator
Fine-Tuning and Hosting a Model Example
Bob wants to fine-tune a Cohere command (52
 billion parameters) (cohere.command) model. Bob creates a
 fine-tuning dedicated AI cluster with the preset value of two Large Cohere units. Bob creates a custom model on the
 fine-tuning dedicated and fine-tunes a foundational model with training
 data. The fine-tuning job takes 5 hours to complete. Bob creates a fine-tuning
 cluster every week.
To host a cohere.command model, Bob needs to identify the unit size
 that can host the cohere.command model. The unit size for
 cohere.command model is a Large Cohere
 unit. See matching clusters to
 base models. Bob can host up to 50 fine-tuned models on a single hosting
 cluster. Here are the steps to calculate the monthly cost for fine-tuning and
 hosting the models.
Calculate the unit hours for each fine-tuning.
Each fine-tuning cluster requires 2 units and each cluster is active for 5 hours
fine-tuning per cluster= 10 unit-hours
Compare the unit hours to the minimum commitment for fine-tuning the models.
10 unit hours > 1 unit hour
Calculate the unit hours for hosting.
31 days x 24 hours/day x 1 unit = 744 unit hours
Compare the unit hours to the minimum commitment for hosting the models.
744 unit hours = 744 minimum unit hours
Go to AI Pricing and under OCI Generative AI, for Oracle Cloud Infrastructure Generative AI- Large Cohere - Dedicated, find the <Large-Cohere-dedicated-unit-per-hour-price>.
Find the total monthly price.
fine-tuning price = 
(10 unit hours)/week x (4 weeks) x $<Large-Cohere-dedicated-unit-per-hour-price>
fine-tuning price = 
40 x <Large-Cohere-dedicated-unit-per-hour-price>
hosting price = 
(744 unit hours) x $<Large-Cohere-dedicated-unit-per-hour-price>
total monthly price = 
(40 + 744 unit hours) x $<Large-Cohere-dedicated-unit-per-hour-price>
Related conceptsModel Limitations in Generative AIRelated informationAI PricingCost estimator
