chat-cohere-chat-request
¶
Description¶
Creates a response for the given conversation.
Required Parameters¶
-
--chat-request-message
[text]
¶
The text that the user inputs for the model to respond to.
-
--compartment-id
,
-c
[text]
¶
The OCID of compartment in which to call the Generative AI service to chat.
-
--serving-mode
[complex type]
¶
This is a complex type whose value must be valid JSON. The value can be provided as a string on the command line or passed in as a file using the file://path/to/file syntax.
The --generate-param-json-input
option can be used to generate an example of the JSON which must be provided. We recommend storing this example
in a file, modifying it as needed and then passing it back in via the file:// syntax.
Optional Parameters¶
-
--chat-request-chat-history
[complex type]
¶
The list of previous messages between the user and the model. The chat history gives the model context for responding to the user’s inputs.
This option is a JSON list with items of type CohereMessage. For documentation on CohereMessage please see our API reference: https://docs.cloud.oracle.com/api/#/en/generativeaiinference/20231130/datatypes/CohereMessage. This is a complex type whose value must be valid JSON. The value can be provided as a string on the command line or passed in as a file using the file://path/to/file syntax.
The --generate-param-json-input
option can be used to generate an example of the JSON which must be provided. We recommend storing this example
in a file, modifying it as needed and then passing it back in via the file:// syntax.
-
--chat-request-citation-quality
[text]
¶
When FAST is selected, citations are generated at the same time as the text output and the request will be completed sooner. May result in less accurate citations.
Accepted values are:
ACCURATE, FAST
-
--chat-request-documents
[complex type]
¶
A list of relevant documents that the model can refer to for generating grounded responses to the user’s requests. Some example keys that you can add to the dictionary are “text”, “author”, and “date”. Keep the total word count of the strings in the dictionary to 300 words or less.
Example:
[ { "title": "Tall penguins", "snippet": "Emperor penguins are the tallest." }, { "title": "Penguin habitats", "snippet": "Emperor penguins only live in Antarctica." } ]
This is a complex type whose value must be valid JSON. The value can be provided as a string on the command line or passed in as a file using the file://path/to/file syntax.
The --generate-param-json-input
option can be used to generate an example of the JSON which must be provided. We recommend storing this example
in a file, modifying it as needed and then passing it back in via the file:// syntax.
-
--chat-request-frequency-penalty
[text]
¶
To reduce repetitiveness of generated tokens, this number penalizes new tokens based on their frequency in the generated text so far. Greater numbers encourage the model to use new tokens, while lower numbers encourage the model to repeat the tokens. Set to 0 to disable.
-
--chat-request-is-echo
[boolean]
¶
Returns the full prompt that was sent to the model when True.
-
--chat-request-is-force-single-step
[boolean]
¶
When enabled, the model will issue (potentially multiple) tool calls in a single step, before it receives the tool responses and directly answers the user’s original message.
-
--chat-request-is-raw-prompting
[boolean]
¶
When enabled, the user’s message will be sent to the model without any preprocessing.
-
--chat-request-is-search-queries-only
[boolean]
¶
When set to true, the response contains only a list of generated search queries without the search results and the model will not respond to the user’s message.
-
--chat-request-is-stream
[boolean]
¶
Whether to stream the partial progress of the model’s response. When set to true, as tokens become available, they are sent as data-only server-sent events.
-
--chat-request-max-tokens
[integer]
¶
The maximum number of output tokens that the model will generate for the response.
-
--chat-request-preamble-override
[text]
¶
If specified, the default Cohere preamble is replaced with the provided preamble. A preamble is an initial guideline message that can change the model’s overall chat behavior and conversation style. Default preambles vary for different models.
Example:
You are a travel advisor. Answer with a pirate tone.
-
--chat-request-presence-penalty
[text]
¶
To reduce repetitiveness of generated tokens, this number penalizes new tokens based on whether they’ve appeared in the generated text so far. Greater numbers encourage the model to use new tokens, while lower numbers encourage the model to repeat the tokens.
Similar to frequency penalty, a penalty is applied to previously present tokens, except that this penalty is applied equally to all tokens that have already appeared, regardless of how many times they’ve appeared. Set to 0 to disable.
-
--chat-request-prompt-truncation
[text]
¶
Defaults to OFF. Dictates how the prompt will be constructed. With prompt_truncation set to AUTO_PRESERVE_ORDER, some elements from chat_history and documents will be dropped to construct a prompt that fits within the model’s context length limit. During this process the order of the documents and chat history will be preserved. With prompt_truncation set to OFF, no elements will be dropped.
Accepted values are:
AUTO_PRESERVE_ORDER, OFF
-
--chat-request-seed
[integer]
¶
If specified, the backend will make a best effort to sample tokens deterministically, such that repeated requests with the same seed and parameters should return the same result. However, determinism cannot be totally guaranteed.
-
--chat-request-stop-sequences
[complex type]
¶
Stop the model generation when it reaches a stop sequence defined in this parameter. This is a complex type whose value must be valid JSON. The value can be provided as a string on the command line or passed in as a file using the file://path/to/file syntax.
The --generate-param-json-input
option can be used to generate an example of the JSON which must be provided. We recommend storing this example
in a file, modifying it as needed and then passing it back in via the file:// syntax.
-
--chat-request-temperature
[text]
¶
A number that sets the randomness of the generated output. A lower temperature means less random generations. Use lower numbers for tasks such as question answering or summarizing. High temperatures can generate hallucinations or factually incorrect information. Start with temperatures lower than 1.0 and increase the temperature for more creative outputs, as you regenerate the prompts to refine the outputs.
-
--chat-request-tool-results
[complex type]
¶
A list of results from invoking tools recommended by the model in the previous chat turn.
This option is a JSON list with items of type CohereToolResult. For documentation on CohereToolResult please see our API reference: https://docs.cloud.oracle.com/api/#/en/generativeaiinference/20231130/datatypes/CohereToolResult. This is a complex type whose value must be valid JSON. The value can be provided as a string on the command line or passed in as a file using the file://path/to/file syntax.
The --generate-param-json-input
option can be used to generate an example of the JSON which must be provided. We recommend storing this example
in a file, modifying it as needed and then passing it back in via the file:// syntax.
-
--chat-request-tools
[complex type]
¶
A list of available tools (functions) that the model may suggest invoking before producing a text response.
This option is a JSON list with items of type CohereTool. For documentation on CohereTool please see our API reference: https://docs.cloud.oracle.com/api/#/en/generativeaiinference/20231130/datatypes/CohereTool. This is a complex type whose value must be valid JSON. The value can be provided as a string on the command line or passed in as a file using the file://path/to/file syntax.
The --generate-param-json-input
option can be used to generate an example of the JSON which must be provided. We recommend storing this example
in a file, modifying it as needed and then passing it back in via the file:// syntax.
-
--chat-request-top-k
[integer]
¶
A sampling method in which the model chooses the next token randomly from the top k most likely tokens. A higher value for k generates more random output, which makes the output text sound more natural. The default value for k is 0 which disables this method and considers all tokens. To set a number for the likely tokens, choose an integer between 1 and 500.
If also using top p, then the model considers only the top tokens whose probabilities add up to p percent and ignores the rest of the k tokens. For example, if k is 20 but only the probabilities of the top 10 add up to the value of p, then only the top 10 tokens are chosen.
-
--chat-request-top-p
[text]
¶
If set to a probability 0.0 < p < 1.0, it ensures that only the most likely tokens, with total probability mass of p, are considered for generation at each step.
To eliminate tokens with low likelihood, assign p a minimum percentage for the next token’s likelihood. For example, when p is set to 0.75, the model eliminates the bottom 25 percent for the next token. Set to 1.0 to consider all tokens and set to 0 to disable. If both k and p are enabled, p acts after k.
-
--from-json
[text]
¶
Provide input to this command as a JSON document from a file using the file://path-to/file syntax.
The --generate-full-command-json-input
option can be used to generate a sample json file to be used with this command option. The key names are pre-populated and match the command option names (converted to camelCase format, e.g. compartment-id –> compartmentId), while the values of the keys need to be populated by the user before using the sample file as an input to this command. For any command option that accepts multiple values, the value of the key can be a JSON array.
Options can still be provided on the command line. If an option exists in both the JSON document and the command line then the command line specified value will be used.
For examples on usage of this option, please see our “using CLI with advanced JSON options” link: https://docs.cloud.oracle.com/iaas/Content/API/SDKDocs/cliusing.htm#AdvancedJSONOptions
Global Parameters¶
Use oci --help
for help on global parameters.
--auth-purpose
, --auth
, --cert-bundle
, --cli-auto-prompt
, --cli-rc-file
, --config-file
, --connection-timeout
, --debug
, --defaults-file
, --endpoint
, --generate-full-command-json-input
, --generate-param-json-input
, --help
, --latest-version
, --max-retries
, --no-retry
, --opc-client-request-id
, --opc-request-id
, --output
, --profile
, --proxy
, --query
, --raw-output
, --read-timeout
, --realm-specific-endpoint
, --region
, --release-info
, --request-id
, --version
, -?
, -d
, -h
, -i
, -v
Example using required parameter¶
Copy and paste the following example into a JSON file, replacing the example parameters with your own.
oci generative-ai-inference chat-result chat-cohere-chat-request --generate-param-json-input serving-mode > serving-mode.json
Copy the following CLI commands into a file named example.sh. Run the command by typing “bash example.sh” and replacing the example parameters with your own.
Please note this sample will only work in the POSIX-compliant bash-like shell. You need to set up the OCI configuration and appropriate security policies before trying the examples.
export chat_request_message=<substitute-value-of-chat_request_message> # https://docs.cloud.oracle.com/en-us/iaas/tools/oci-cli/latest/oci_cli_docs/cmdref/generative-ai-inference/chat-result/chat-cohere-chat-request.html#cmdoption-chat-request-message
export compartment_id=<substitute-value-of-compartment_id> # https://docs.cloud.oracle.com/en-us/iaas/tools/oci-cli/latest/oci_cli_docs/cmdref/generative-ai-inference/chat-result/chat-cohere-chat-request.html#cmdoption-compartment-id
oci generative-ai-inference chat-result chat-cohere-chat-request --chat-request-message $chat_request_message --compartment-id $compartment_id --serving-mode file://serving-mode.json