com.oracle.bmc.generativeaiinference.model

Class GenericChatRequest.Builder

java.lang.Object

com.oracle.bmc.generativeaiinference.model.GenericChatRequest.Builder

Enclosing class:

GenericChatRequest

public static class GenericChatRequest.Builder
extends Object

Constructor Summary

Constructors

Constructor and Description
Builder()

Method Summary

Modifier and Type	Method and Description
GenericChatRequest	build()
GenericChatRequest.Builder	copy(GenericChatRequest model)
GenericChatRequest.Builder	frequencyPenalty(Double frequencyPenalty) To reduce repetitiveness of generated tokens, this number penalizes new tokens based on their frequency in the generated text so far.
GenericChatRequest.Builder	isEcho(Boolean isEcho) Whether to include the user prompt in the response.
GenericChatRequest.Builder	isStream(Boolean isStream) Whether to stream back partial progress.
GenericChatRequest.Builder	logitBias(Object logitBias) Modifies the likelihood of specified tokens that appear in the completion.
GenericChatRequest.Builder	logProbs(Integer logProbs) Includes the logarithmic probabilities for the most likely output tokens and the chosen tokens.
GenericChatRequest.Builder	maxTokens(Integer maxTokens) The maximum number of tokens that can be generated per output sequence.
GenericChatRequest.Builder	messages(List<Message> messages) The series of messages in a chat request.
GenericChatRequest.Builder	numGenerations(Integer numGenerations) The number of of generated texts that will be returned.
GenericChatRequest.Builder	presencePenalty(Double presencePenalty) To reduce repetitiveness of generated tokens, this number penalizes new tokens based on whether they’ve appeared in the generated text so far.
GenericChatRequest.Builder	stop(List<String> stop) List of strings that stop the generation if they are generated for the response text.
GenericChatRequest.Builder	temperature(Double temperature) A number that sets the randomness of the generated output.
GenericChatRequest.Builder	topK(Integer topK) An integer that sets up the model to use only the top k most likely tokens in the generated output.
GenericChatRequest.Builder	topP(Double topP) If set to a probability 0.0 < p < 1.0, it ensures that only the most likely tokens, with total probability mass of p, are considered for generation at each step.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

Builder

public Builder()

Method Detail

messages

public GenericChatRequest.Builder messages(List<Message> messages)

The series of messages in a chat request. Includes the previous messages in a conversation. Each message includes a role (USER or the CHATBOT) and content.

Parameters:

messages - the value to set

Returns:

this builder

isStream

public GenericChatRequest.Builder isStream(Boolean isStream)

Whether to stream back partial progress. If set to true, as tokens become available, they are sent as data-only server-sent events.

Parameters:

isStream - the value to set

Returns:

this builder

numGenerations

public GenericChatRequest.Builder numGenerations(Integer numGenerations)

The number of of generated texts that will be returned.

Parameters:

numGenerations - the value to set

Returns:

this builder

isEcho

public GenericChatRequest.Builder isEcho(Boolean isEcho)

Whether to include the user prompt in the response. Applies only to non-stream results.

Parameters:

isEcho - the value to set

Returns:

this builder

topK

public GenericChatRequest.Builder topK(Integer topK)

An integer that sets up the model to use only the top k most likely tokens in the generated output. A higher k introduces more randomness into the output making the output text sound more natural. Default value is -1 which means to consider all tokens. Setting to 0 disables this method and considers all tokens.

If also using top p, then the model considers only the top tokens whose probabilities add up to p percent and ignores the rest of the k tokens. For example, if k is 20, but the probabilities of the top 10 add up to .75, then only the top 10 tokens are chosen.

Parameters:

topK - the value to set

Returns:

this builder

topP

public GenericChatRequest.Builder topP(Double topP)

If set to a probability 0.0 < p < 1.0, it ensures that only the most likely tokens, with total probability mass of p, are considered for generation at each step.

To eliminate tokens with low likelihood, assign p a minimum percentage for the next token's likelihood. For example, when p is set to 0.75, the model eliminates the bottom 25 percent for the next token. Set to 1 to consider all tokens and set to 0 to disable. If both k and p are enabled, p acts after k.

Parameters:

topP - the value to set

Returns:

this builder

temperature

public GenericChatRequest.Builder temperature(Double temperature)

A number that sets the randomness of the generated output. A lower temperature means a less random generations.

Use lower numbers for tasks with a correct answer such as question answering or summarizing. High temperatures can generate hallucinations or factually incorrect information. Start with temperatures lower than 1.0 and increase the temperature for more creative outputs, as you regenerate the prompts to refine the outputs.

Parameters:

temperature - the value to set

Returns:

this builder

frequencyPenalty

public GenericChatRequest.Builder frequencyPenalty(Double frequencyPenalty)

To reduce repetitiveness of generated tokens, this number penalizes new tokens based on their frequency in the generated text so far. Values > 0 encourage the model to use new tokens and values < 0 encourage the model to repeat tokens. Set to 0 to disable.

Parameters:

frequencyPenalty - the value to set

Returns:

this builder

presencePenalty

public GenericChatRequest.Builder presencePenalty(Double presencePenalty)

To reduce repetitiveness of generated tokens, this number penalizes new tokens based on whether they’ve appeared in the generated text so far. Values > 0 encourage the model to use new tokens and values < 0 encourage the model to repeat tokens.

Similar to frequency penalty, a penalty is applied to previously present tokens, except that this penalty is applied equally to all tokens that have already appeared, regardless of how many times they've appeared. Set to 0 to disable.

Parameters:

presencePenalty - the value to set

Returns:

this builder

stop

public GenericChatRequest.Builder stop(List<String> stop)

List of strings that stop the generation if they are generated for the response text. The returned output will not contain the stop strings.

Parameters:

stop - the value to set

Returns:

this builder

logProbs

public GenericChatRequest.Builder logProbs(Integer logProbs)

Includes the logarithmic probabilities for the most likely output tokens and the chosen tokens.

For example, if the log probability is 5, the API returns a list of the 5 most likely tokens. The API returns the log probability of the sampled token, so there might be up to logprobs+1 elements in the response.

Parameters:

logProbs - the value to set

Returns:

this builder

maxTokens

public GenericChatRequest.Builder maxTokens(Integer maxTokens)

The maximum number of tokens that can be generated per output sequence. The token count of your prompt plus maxTokens must not exceed the model’s context length. Not setting a value for maxTokens results in the possible use of model’s full context length.

Parameters:

maxTokens - the value to set

Returns:

this builder

logitBias

public GenericChatRequest.Builder logitBias(Object logitBias)

Modifies the likelihood of specified tokens that appear in the completion.

Example: '{"6395": 2, "8134": 1, "21943": 0.5, "5923": -100}'

Parameters:

logitBias - the value to set

Returns:

this builder

build

public GenericChatRequest build()

copy

public GenericChatRequest.Builder copy(GenericChatRequest model)