CohereChatRequest¶

class oci.generative_ai_inference.models.CohereChatRequest(**kwargs)¶

Bases: oci.generative_ai_inference.models.base_chat_request.BaseChatRequest

Details for the chat request for Cohere models.

Attributes

`API_FORMAT_COHERE`	str(object=’’) -> str
`API_FORMAT_GENERIC`	str(object=’’) -> str
`CITATION_QUALITY_ACCURATE`	A constant which can be used with the citation_quality property of a CohereChatRequest.
`CITATION_QUALITY_FAST`	A constant which can be used with the citation_quality property of a CohereChatRequest.
`PROMPT_TRUNCATION_AUTO_PRESERVE_ORDER`	A constant which can be used with the prompt_truncation property of a CohereChatRequest.
`PROMPT_TRUNCATION_OFF`	A constant which can be used with the prompt_truncation property of a CohereChatRequest.
`api_format`	[Required] Gets the api_format of this BaseChatRequest.
`chat_history`	Gets the chat_history of this CohereChatRequest.
`citation_quality`	Gets the citation_quality of this CohereChatRequest.
`documents`	Gets the documents of this CohereChatRequest.
`frequency_penalty`	Gets the frequency_penalty of this CohereChatRequest.
`is_echo`	Gets the is_echo of this CohereChatRequest.
`is_force_single_step`	Gets the is_force_single_step of this CohereChatRequest.
`is_raw_prompting`	Gets the is_raw_prompting of this CohereChatRequest.
`is_search_queries_only`	Gets the is_search_queries_only of this CohereChatRequest.
`is_stream`	Gets the is_stream of this CohereChatRequest.
`max_input_tokens`	Gets the max_input_tokens of this CohereChatRequest.
`max_tokens`	Gets the max_tokens of this CohereChatRequest.
`message`	[Required] Gets the message of this CohereChatRequest.
`preamble_override`	Gets the preamble_override of this CohereChatRequest.
`presence_penalty`	Gets the presence_penalty of this CohereChatRequest.
`prompt_truncation`	Gets the prompt_truncation of this CohereChatRequest.
`response_format`	Gets the response_format of this CohereChatRequest.
`seed`	Gets the seed of this CohereChatRequest.
`stop_sequences`	Gets the stop_sequences of this CohereChatRequest.
`temperature`	Gets the temperature of this CohereChatRequest.
`tool_results`	Gets the tool_results of this CohereChatRequest.
`tools`	Gets the tools of this CohereChatRequest.
`top_k`	Gets the top_k of this CohereChatRequest.
`top_p`	Gets the top_p of this CohereChatRequest.

Methods

`__init__`(**kwargs)	Initializes a new CohereChatRequest object with values from keyword arguments.
`get_subtype`(object_dictionary)	Given the hash representation of a subtype of this class, use the info in the hash to return the class of the subtype.

API_FORMAT_COHERE = 'COHERE'¶

API_FORMAT_GENERIC = 'GENERIC'¶

CITATION_QUALITY_ACCURATE = 'ACCURATE'¶: A constant which can be used with the citation_quality property of a CohereChatRequest. This constant has a value of “ACCURATE”

CITATION_QUALITY_FAST = 'FAST'¶: A constant which can be used with the citation_quality property of a CohereChatRequest. This constant has a value of “FAST”

PROMPT_TRUNCATION_AUTO_PRESERVE_ORDER = 'AUTO_PRESERVE_ORDER'¶: A constant which can be used with the prompt_truncation property of a CohereChatRequest. This constant has a value of “AUTO_PRESERVE_ORDER”

PROMPT_TRUNCATION_OFF = 'OFF'¶: A constant which can be used with the prompt_truncation property of a CohereChatRequest. This constant has a value of “OFF”

__init__(**kwargs)¶

Initializes a new CohereChatRequest object with values from keyword arguments. The default value of the api_format attribute of this class is COHERE and it should not be changed. The following keyword arguments are supported (corresponding to the getters/setters of this class):

Parameters:

api_format (str) – The value to assign to the api_format property of this CohereChatRequest. Allowed values for this property are: “COHERE”, “GENERIC”
message (str) – The value to assign to the message property of this CohereChatRequest.
chat_history (list[oci.generative_ai_inference.models.CohereMessage]) – The value to assign to the chat_history property of this CohereChatRequest.
documents (list[object]) – The value to assign to the documents property of this CohereChatRequest.
response_format (oci.generative_ai_inference.models.CohereResponseFormat) – The value to assign to the response_format property of this CohereChatRequest.
is_search_queries_only (bool) – The value to assign to the is_search_queries_only property of this CohereChatRequest.
preamble_override (str) – The value to assign to the preamble_override property of this CohereChatRequest.
is_stream (bool) – The value to assign to the is_stream property of this CohereChatRequest.
max_tokens (int) – The value to assign to the max_tokens property of this CohereChatRequest.
max_input_tokens (int) – The value to assign to the max_input_tokens property of this CohereChatRequest.
temperature (float) – The value to assign to the temperature property of this CohereChatRequest.
top_k (int) – The value to assign to the top_k property of this CohereChatRequest.
top_p (float) – The value to assign to the top_p property of this CohereChatRequest.
prompt_truncation (str) – The value to assign to the prompt_truncation property of this CohereChatRequest. Allowed values for this property are: “OFF”, “AUTO_PRESERVE_ORDER”
frequency_penalty (float) – The value to assign to the frequency_penalty property of this CohereChatRequest.
presence_penalty (float) – The value to assign to the presence_penalty property of this CohereChatRequest.
seed (int) – The value to assign to the seed property of this CohereChatRequest.
is_echo (bool) – The value to assign to the is_echo property of this CohereChatRequest.
tools (list[oci.generative_ai_inference.models.CohereTool]) – The value to assign to the tools property of this CohereChatRequest.
tool_results (list[oci.generative_ai_inference.models.CohereToolResult]) – The value to assign to the tool_results property of this CohereChatRequest.
is_force_single_step (bool) – The value to assign to the is_force_single_step property of this CohereChatRequest.
stop_sequences (list[str]) – The value to assign to the stop_sequences property of this CohereChatRequest.
is_raw_prompting (bool) – The value to assign to the is_raw_prompting property of this CohereChatRequest.
citation_quality (str) – The value to assign to the citation_quality property of this CohereChatRequest. Allowed values for this property are: “ACCURATE”, “FAST”

api_format¶

[Required] Gets the api_format of this BaseChatRequest. The API format for the model’s family group. COHERE is for the Cohere family models such as the cohere.command-r-16k and cohere.command-r-plus models. GENERIC is for other model families such as the meta.llama-3-70b-instruct model.

Allowed values for this property are: “COHERE”, “GENERIC”

Returns:	The api_format of this BaseChatRequest.
Return type:	str

chat_history¶

Gets the chat_history of this CohereChatRequest. The list of previous messages between the user and the model. The chat history gives the model context for responding to the user’s inputs.

Returns:	The chat_history of this CohereChatRequest.
Return type:	list[oci.generative_ai_inference.models.CohereMessage]

citation_quality¶

Gets the citation_quality of this CohereChatRequest. When FAST is selected, citations are generated at the same time as the text output and the request will be completed sooner. May result in less accurate citations.

Allowed values for this property are: “ACCURATE”, “FAST”

Returns:	The citation_quality of this CohereChatRequest.
Return type:	str

documents¶

Gets the documents of this CohereChatRequest. A list of relevant documents that the model can refer to for generating grounded responses to the user’s requests. Some example keys that you can add to the dictionary are “text”, “author”, and “date”. Keep the total word count of the strings in the dictionary to 300 words or less.

Example: `[

{ “title”: “Tall penguins”, “snippet”: “Emperor penguins are the tallest.” }, { “title”: “Penguin habitats”, “snippet”: “Emperor penguins only live in Antarctica.” }

]`

Returns:	The documents of this CohereChatRequest.
Return type:	list[object]

frequency_penalty¶

Gets the frequency_penalty of this CohereChatRequest. To reduce repetitiveness of generated tokens, this number penalizes new tokens based on their frequency in the generated text so far. Greater numbers encourage the model to use new tokens, while lower numbers encourage the model to repeat the tokens. Set to 0 to disable.

Returns:	The frequency_penalty of this CohereChatRequest.
Return type:	float

static get_subtype(object_dictionary)¶: Given the hash representation of a subtype of this class, use the info in the hash to return the class of the subtype.

is_echo¶

Gets the is_echo of this CohereChatRequest. Returns the full prompt that was sent to the model when True.

Returns:	The is_echo of this CohereChatRequest.
Return type:	bool

is_force_single_step¶

Gets the is_force_single_step of this CohereChatRequest. When enabled, the model will issue (potentially multiple) tool calls in a single step, before it receives the tool responses and directly answers the user’s original message.

Returns:	The is_force_single_step of this CohereChatRequest.
Return type:	bool

is_raw_prompting¶

Gets the is_raw_prompting of this CohereChatRequest. When enabled, the user’s message will be sent to the model without any preprocessing.

Returns:	The is_raw_prompting of this CohereChatRequest.
Return type:	bool

is_search_queries_only¶

Gets the is_search_queries_only of this CohereChatRequest. When set to true, the response contains only a list of generated search queries without the search results and the model will not respond to the user’s message.

Returns:	The is_search_queries_only of this CohereChatRequest.
Return type:	bool

is_stream¶

Gets the is_stream of this CohereChatRequest. Whether to stream the partial progress of the model’s response. When set to true, as tokens become available, they are sent as data-only server-sent events.

Returns:	The is_stream of this CohereChatRequest.
Return type:	bool

max_input_tokens¶

Gets the max_input_tokens of this CohereChatRequest. The maximum number of input tokens to send to the model. If not specified, max_input_tokens is the model’s context length limit minus a small buffer.

Returns:	The max_input_tokens of this CohereChatRequest.
Return type:	int

max_tokens¶

Gets the max_tokens of this CohereChatRequest. The maximum number of output tokens that the model will generate for the response.

Returns:	The max_tokens of this CohereChatRequest.
Return type:	int

message¶

[Required] Gets the message of this CohereChatRequest. The text that the user inputs for the model to respond to.

Returns:	The message of this CohereChatRequest.
Return type:	str

preamble_override¶

Gets the preamble_override of this CohereChatRequest. If specified, the default Cohere preamble is replaced with the provided preamble. A preamble is an initial guideline message that can change the model’s overall chat behavior and conversation style. Default preambles vary for different models.

Example: You are a travel advisor. Answer with a pirate tone.

Returns:	The preamble_override of this CohereChatRequest.
Return type:	str

presence_penalty¶

Gets the presence_penalty of this CohereChatRequest. To reduce repetitiveness of generated tokens, this number penalizes new tokens based on whether they’ve appeared in the generated text so far. Greater numbers encourage the model to use new tokens, while lower numbers encourage the model to repeat the tokens.

Similar to frequency penalty, a penalty is applied to previously present tokens, except that this penalty is applied equally to all tokens that have already appeared, regardless of how many times they’ve appeared. Set to 0 to disable.

Returns:	The presence_penalty of this CohereChatRequest.
Return type:	float

prompt_truncation¶

Gets the prompt_truncation of this CohereChatRequest. Defaults to OFF. Dictates how the prompt will be constructed. With promptTruncation set to AUTO_PRESERVE_ORDER, some elements from chatHistory and documents will be dropped to construct a prompt that fits within the model’s context length limit. During this process the order of the documents and chat history will be preserved. With prompt_truncation set to OFF, no elements will be dropped.

Allowed values for this property are: “OFF”, “AUTO_PRESERVE_ORDER”

Returns:	The prompt_truncation of this CohereChatRequest.
Return type:	str

response_format¶

Gets the response_format of this CohereChatRequest.

Returns:	The response_format of this CohereChatRequest.
Return type:	oci.generative_ai_inference.models.CohereResponseFormat

seed¶

Gets the seed of this CohereChatRequest. If specified, the backend will make a best effort to sample tokens deterministically, such that repeated requests with the same seed and parameters should return the same result. However, determinism cannot be totally guaranteed.

Returns:	The seed of this CohereChatRequest.
Return type:	int

stop_sequences¶

Gets the stop_sequences of this CohereChatRequest. Stop the model generation when it reaches a stop sequence defined in this parameter.

Returns:	The stop_sequences of this CohereChatRequest.
Return type:	list[str]

temperature¶

Gets the temperature of this CohereChatRequest. A number that sets the randomness of the generated output. A lower temperature means less random generations. Use lower numbers for tasks such as question answering or summarizing. High temperatures can generate hallucinations or factually incorrect information. Start with temperatures lower than 1.0 and increase the temperature for more creative outputs, as you regenerate the prompts to refine the outputs.

Returns:	The temperature of this CohereChatRequest.
Return type:	float

tool_results¶

Gets the tool_results of this CohereChatRequest. A list of results from invoking tools recommended by the model in the previous chat turn.

Returns:	The tool_results of this CohereChatRequest.
Return type:	list[oci.generative_ai_inference.models.CohereToolResult]

tools¶

Gets the tools of this CohereChatRequest. A list of available tools (functions) that the model may suggest invoking before producing a text response.

Returns:	The tools of this CohereChatRequest.
Return type:	list[oci.generative_ai_inference.models.CohereTool]

top_k¶

Gets the top_k of this CohereChatRequest. A sampling method in which the model chooses the next token randomly from the top k most likely tokens. A higher value for k generates more random output, which makes the output text sound more natural. The default value for k is 0 which disables this method and considers all tokens. To set a number for the likely tokens, choose an integer between 1 and 500.

If also using top p, then the model considers only the top tokens whose probabilities add up to p percent and ignores the rest of the k tokens. For example, if k is 20 but only the probabilities of the top 10 add up to the value of p, then only the top 10 tokens are chosen.

Returns:	The top_k of this CohereChatRequest.
Return type:	int

top_p¶

Gets the top_p of this CohereChatRequest. If set to a probability 0.0 < p < 1.0, it ensures that only the most likely tokens, with total probability mass of p, are considered for generation at each step.

To eliminate tokens with low likelihood, assign p a minimum percentage for the next token’s likelihood. For example, when p is set to 0.75, the model eliminates the bottom 25 percent for the next token. Set to 1.0 to consider all tokens and set to 0 to disable. If both k and p are enabled, p acts after k.

Returns:	The top_p of this CohereChatRequest.
Return type:	float