Class: OCI::GenerativeAiInference::GenerativeAiInferenceClient

Inherits:
Object
  • Object
show all
Defined in:
lib/oci/generative_ai_inference/generative_ai_inference_client.rb

Overview

OCI Generative AI is a fully managed service that provides a set of state-of-the-art, customizable large language models (LLMs) that cover a wide range of use cases for text generation, summarization, and text embeddings.

Use the Generative AI service inference API to access your custom model endpoints, or to try the out-of-the-box models to generate text, summarize, and create text embeddings.

To use a Generative AI custom model for inference, you must first create an endpoint for that model. Use the Generative AI service management API to create a custom model by fine-tuning an out-of-the-box model, or a previous version of a custom model, using your own data. Fine-tune the custom model on a fine-tuning dedicated AI cluster. Then, create a hosting dedicated AI cluster with an endpoint to host your custom model. For resource management in the Generative AI service, use the Generative AI service management API.

To learn more about the service, see the Generative AI documentation.

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(config: nil, region: nil, endpoint: nil, signer: nil, proxy_settings: nil, retry_config: nil) ⇒ GenerativeAiInferenceClient

Creates a new GenerativeAiInferenceClient. Notes: If a config is not specified, then the global OCI.config will be used.

This client is not thread-safe

Either a region or an endpoint must be specified. If an endpoint is specified, it will be used instead of the region. A region may be specified in the config or via or the region parameter. If specified in both, then the region parameter will be used.

Parameters:

  • config (Config) (defaults to: nil) —

    A Config object.

  • region (String) (defaults to: nil) —

    A region used to determine the service endpoint. This will usually correspond to a value in Regions::REGION_ENUM, but may be an arbitrary string.

  • endpoint (String) (defaults to: nil) —

    The fully qualified endpoint URL

  • signer (OCI::BaseSigner) (defaults to: nil) —

    A signer implementation which can be used by this client. If this is not provided then a signer will be constructed via the provided config. One use case of this parameter is instance principals authentication, so that the instance principals signer can be provided to the client

  • proxy_settings (OCI::ApiClientProxySettings) (defaults to: nil) —

    If your environment requires you to use a proxy server for outgoing HTTP requests the details for the proxy can be provided in this parameter

  • retry_config (OCI::Retry::RetryConfig) (defaults to: nil) —

    The retry configuration for this service client. This represents the default retry configuration to apply across all operations. This can be overridden on a per-operation basis. The default retry configuration value is nil, which means that an operation will not perform any retries



61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
 
# File 'lib/oci/generative_ai_inference/generative_ai_inference_client.rb', line 61

def initialize(config: nil, region: nil, endpoint: nil, signer: nil, proxy_settings: nil, retry_config: nil)
  # If the signer is an InstancePrincipalsSecurityTokenSigner or SecurityTokenSigner and no config was supplied (they are self-sufficient signers)
  # then create a dummy config to pass to the ApiClient constructor. If customers wish to create a client which uses instance principals
  # and has config (either populated programmatically or loaded from a file), they must construct that config themselves and then
  # pass it to this constructor.
  #
  # If there is no signer (or the signer is not an instance principals signer) and no config was supplied, this is not valid
  # so try and load the config from the default file.
  config = OCI::Config.validate_and_build_config_with_signer(config, signer)

  signer = OCI::Signer.config_file_auth_builder(config) if signer.nil?

  @api_client = OCI::ApiClient.new(config, signer, proxy_settings: proxy_settings)
  @retry_config = retry_config

  if endpoint
    @endpoint = endpoint + '/20231130'
  else
    region ||= config.region
    region ||= signer.region if signer.respond_to?(:region)
    self.region = region
  end
  logger.info "GenerativeAiInferenceClient endpoint set to '#{@endpoint}'." if logger
end

Instance Attribute Details

#api_client ⇒ OCI::ApiClient (readonly)

Client used to make HTTP requests.

Returns:



21
22
23
 
# File 'lib/oci/generative_ai_inference/generative_ai_inference_client.rb', line 21

def api_client
  @api_client
end

#endpoint ⇒ String (readonly)

Fully qualified endpoint URL

Returns:

  • (String) 


25
26
27
 
# File 'lib/oci/generative_ai_inference/generative_ai_inference_client.rb', line 25

def endpoint
  @endpoint
end

#region ⇒ String

The region, which will usually correspond to a value in Regions::REGION_ENUM.

Returns:

  • (String) 


35
36
37
 
# File 'lib/oci/generative_ai_inference/generative_ai_inference_client.rb', line 35

def region
  @region
end

#retry_config ⇒ OCI::Retry::RetryConfig (readonly)

The default retry configuration to apply to all operations in this service client. This can be overridden on a per-operation basis. The default retry configuration value is nil, which means that an operation will not perform any retries

Returns:



31
32
33
 
# File 'lib/oci/generative_ai_inference/generative_ai_inference_client.rb', line 31

def retry_config
  @retry_config
end

Instance Method Details

#chat(chat_details, opts = {}, &block) ⇒ Response

Note:

Click here to see an example of how to use chat API.

Creates a response for the given conversation.

Parameters:

Options Hash (opts):

  • :retry_config (OCI::Retry::RetryConfig) —

    The retry configuration to apply to this operation. If no key is provided then the service-level retry configuration defined by #retry_config will be used. If an explicit nil value is provided then the operation will not retry

  • :opc_retry_token (String) —

    A token that uniquely identifies a request so it can be retried in case of a timeout or server error without risk of executing that same action again. Retry tokens expire after 24 hours, but can be invalidated before then due to conflicting operations. For example, if a resource has been deleted and purged from the system, then a retry of the original creation request might be rejected.

  • :opc_request_id (String) —

    The client request ID for tracing.

Returns:



124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
 
# File 'lib/oci/generative_ai_inference/generative_ai_inference_client.rb', line 124

def chat(chat_details, opts = {}, &block)
  logger.debug 'Calling operation GenerativeAiInferenceClient#chat.' if logger

  raise "Missing the required parameter 'chat_details' when calling chat." if chat_details.nil?

  path = '/actions/chat'
  operation_signing_strategy = :standard

  # rubocop:disable Style/NegatedIf
  # Query Params
  query_params = {}

  # Header Params
  header_params = {}
  header_params[:accept] = 'application/json, text/event-stream'
  header_params[:'content-type'] = 'application/json'
  header_params[:'opc-retry-token'] = opts[:opc_retry_token] if opts[:opc_retry_token]
  header_params[:'opc-request-id'] = opts[:opc_request_id] if opts[:opc_request_id]
  # rubocop:enable Style/NegatedIf
  header_params[:'opc-retry-token'] ||= OCI::Retry.generate_opc_retry_token

  post_body = @api_client.object_to_http_body(chat_details)

  # rubocop:disable Metrics/BlockLength
  OCI::Retry.make_retrying_call(applicable_retry_config(opts), call_name: 'GenerativeAiInferenceClient#chat') do
    @api_client.call_api(
      :POST,
      path,
      endpoint,
      header_params: header_params,
      query_params: query_params,
      operation_signing_strategy: operation_signing_strategy,
      body: post_body,
      return_type: 'OCI::GenerativeAiInference::Models::ChatResult',
      &block
    )
  end
  # rubocop:enable Metrics/BlockLength
end

#embed_text(embed_text_details, opts = {}) ⇒ Response

Note:

Click here to see an example of how to use embed_text API.

Produces embeddings for the inputs.

An embedding is numeric representation of a piece of text. This text can be a phrase, a sentence, or one or more paragraphs. The Generative AI embedding model transforms each phrase, sentence, or paragraph that you input, into an array with 1024 numbers. You can use these embeddings for finding similarity in your input text such as finding phrases that are similar in context or category. Embeddings are mostly used for semantic searches where the search function focuses on the meaning of the text that it's searching through rather than finding results based on keywords.

Parameters:

Options Hash (opts):

  • :retry_config (OCI::Retry::RetryConfig) —

    The retry configuration to apply to this operation. If no key is provided then the service-level retry configuration defined by #retry_config will be used. If an explicit nil value is provided then the operation will not retry

  • :opc_retry_token (String) —

    A token that uniquely identifies a request so it can be retried in case of a timeout or server error without risk of executing that same action again. Retry tokens expire after 24 hours, but can be invalidated before then due to conflicting operations. For example, if a resource has been deleted and purged from the system, then a retry of the original creation request might be rejected.

  • :opc_request_id (String) —

    The client request ID for tracing.

Returns:



189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
 
# File 'lib/oci/generative_ai_inference/generative_ai_inference_client.rb', line 189

def embed_text(embed_text_details, opts = {})
  logger.debug 'Calling operation GenerativeAiInferenceClient#embed_text.' if logger

  raise "Missing the required parameter 'embed_text_details' when calling embed_text." if embed_text_details.nil?

  path = '/actions/embedText'
  operation_signing_strategy = :standard

  # rubocop:disable Style/NegatedIf
  # Query Params
  query_params = {}

  # Header Params
  header_params = {}
  header_params[:accept] = 'application/json'
  header_params[:'content-type'] = 'application/json'
  header_params[:'opc-retry-token'] = opts[:opc_retry_token] if opts[:opc_retry_token]
  header_params[:'opc-request-id'] = opts[:opc_request_id] if opts[:opc_request_id]
  # rubocop:enable Style/NegatedIf
  header_params[:'opc-retry-token'] ||= OCI::Retry.generate_opc_retry_token

  post_body = @api_client.object_to_http_body(embed_text_details)

  # rubocop:disable Metrics/BlockLength
  OCI::Retry.make_retrying_call(applicable_retry_config(opts), call_name: 'GenerativeAiInferenceClient#embed_text') do
    @api_client.call_api(
      :POST,
      path,
      endpoint,
      header_params: header_params,
      query_params: query_params,
      operation_signing_strategy: operation_signing_strategy,
      body: post_body,
      return_type: 'OCI::GenerativeAiInference::Models::EmbedTextResult'
    )
  end
  # rubocop:enable Metrics/BlockLength
end

#generate_text(generate_text_details, opts = {}, &block) ⇒ Response

Note:

Click here to see an example of how to use generate_text API.

Generates a text response based on the user prompt.

Parameters:

Options Hash (opts):

  • :retry_config (OCI::Retry::RetryConfig) —

    The retry configuration to apply to this operation. If no key is provided then the service-level retry configuration defined by #retry_config will be used. If an explicit nil value is provided then the operation will not retry

  • :opc_retry_token (String) —

    A token that uniquely identifies a request so it can be retried in case of a timeout or server error without risk of executing that same action again. Retry tokens expire after 24 hours, but can be invalidated before then due to conflicting operations. For example, if a resource has been deleted and purged from the system, then a retry of the original creation request might be rejected.

  • :opc_request_id (String) —

    The client request ID for tracing.

Returns:



251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
 
# File 'lib/oci/generative_ai_inference/generative_ai_inference_client.rb', line 251

def generate_text(generate_text_details, opts = {}, &block)
  logger.debug 'Calling operation GenerativeAiInferenceClient#generate_text.' if logger

  raise "Missing the required parameter 'generate_text_details' when calling generate_text." if generate_text_details.nil?

  path = '/actions/generateText'
  operation_signing_strategy = :standard

  # rubocop:disable Style/NegatedIf
  # Query Params
  query_params = {}

  # Header Params
  header_params = {}
  header_params[:accept] = 'application/json, text/event-stream'
  header_params[:'content-type'] = 'application/json'
  header_params[:'opc-retry-token'] = opts[:opc_retry_token] if opts[:opc_retry_token]
  header_params[:'opc-request-id'] = opts[:opc_request_id] if opts[:opc_request_id]
  # rubocop:enable Style/NegatedIf
  header_params[:'opc-retry-token'] ||= OCI::Retry.generate_opc_retry_token

  post_body = @api_client.object_to_http_body(generate_text_details)

  # rubocop:disable Metrics/BlockLength
  OCI::Retry.make_retrying_call(applicable_retry_config(opts), call_name: 'GenerativeAiInferenceClient#generate_text') do
    @api_client.call_api(
      :POST,
      path,
      endpoint,
      header_params: header_params,
      query_params: query_params,
      operation_signing_strategy: operation_signing_strategy,
      body: post_body,
      return_type: 'OCI::GenerativeAiInference::Models::GenerateTextResult',
      &block
    )
  end
  # rubocop:enable Metrics/BlockLength
end

#logger ⇒ Logger

Returns The logger for this client. May be nil.

Returns:

  • (Logger) —

    The logger for this client. May be nil.



100
101
102
 
# File 'lib/oci/generative_ai_inference/generative_ai_inference_client.rb', line 100

def logger
  @api_client.config.logger
end

#summarize_text(summarize_text_details, opts = {}) ⇒ Response

Note:

Click here to see an example of how to use summarize_text API.

Summarizes the input text.

Parameters:

Options Hash (opts):

  • :retry_config (OCI::Retry::RetryConfig) —

    The retry configuration to apply to this operation. If no key is provided then the service-level retry configuration defined by #retry_config will be used. If an explicit nil value is provided then the operation will not retry

  • :opc_retry_token (String) —

    A token that uniquely identifies a request so it can be retried in case of a timeout or server error without risk of executing that same action again. Retry tokens expire after 24 hours, but can be invalidated before then due to conflicting operations. For example, if a resource has been deleted and purged from the system, then a retry of the original creation request might be rejected.

  • :opc_request_id (String) —

    The client request ID for tracing.

Returns:



314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
 
# File 'lib/oci/generative_ai_inference/generative_ai_inference_client.rb', line 314

def summarize_text(summarize_text_details, opts = {})
  logger.debug 'Calling operation GenerativeAiInferenceClient#summarize_text.' if logger

  raise "Missing the required parameter 'summarize_text_details' when calling summarize_text." if summarize_text_details.nil?

  path = '/actions/summarizeText'
  operation_signing_strategy = :standard

  # rubocop:disable Style/NegatedIf
  # Query Params
  query_params = {}

  # Header Params
  header_params = {}
  header_params[:accept] = 'application/json'
  header_params[:'content-type'] = 'application/json'
  header_params[:'opc-retry-token'] = opts[:opc_retry_token] if opts[:opc_retry_token]
  header_params[:'opc-request-id'] = opts[:opc_request_id] if opts[:opc_request_id]
  # rubocop:enable Style/NegatedIf
  header_params[:'opc-retry-token'] ||= OCI::Retry.generate_opc_retry_token

  post_body = @api_client.object_to_http_body(summarize_text_details)

  # rubocop:disable Metrics/BlockLength
  OCI::Retry.make_retrying_call(applicable_retry_config(opts), call_name: 'GenerativeAiInferenceClient#summarize_text') do
    @api_client.call_api(
      :POST,
      path,
      endpoint,
      header_params: header_params,
      query_params: query_params,
      operation_signing_strategy: operation_signing_strategy,
      body: post_body,
      return_type: 'OCI::GenerativeAiInference::Models::SummarizeTextResult'
    )
  end
  # rubocop:enable Metrics/BlockLength
end