generation/logits_process

• generation/logits_process
  • .LogitsProcessor
    • ._call(input_ids, logits)
  • .LogitsWarper
    • ._call(input_ids, logits)
  • .LogitsProcessorList
    • new LogitsProcessorList()
    • .push(item)
    • .extend(items)
    • ._call(input_ids, logits)
  • .ForcedBOSTokenLogitsProcessor
    • new ForcedBOSTokenLogitsProcessor(bos_token_id)
    • ._call(input_ids, logits) ⇒ Object
  • .ForcedEOSTokenLogitsProcessor
    • new ForcedEOSTokenLogitsProcessor(max_length, eos_token_id)
    • ._call(input_ids, logits)
  • .SuppressTokensAtBeginLogitsProcessor
    • new SuppressTokensAtBeginLogitsProcessor(begin_suppress_tokens, begin_index)
    • ._call(input_ids, logits) ⇒ Object
  • .WhisperTimeStampLogitsProcessor
    • new WhisperTimeStampLogitsProcessor(generate_config, init_tokens)
    • ._call(input_ids, logits) ⇒ Tensor
  • .NoRepeatNGramLogitsProcessor
    • new NoRepeatNGramLogitsProcessor(no_repeat_ngram_size)
    • .getNgrams(prevInputIds) ⇒ Map.<string, Array<number>>
    • .getGeneratedNgrams(bannedNgrams, prevInputIds) ⇒ Array.<number>
    • .calcBannedNgramTokens(prevInputIds) ⇒ Array.<number>
    • ._call(input_ids, logits) ⇒ Object
  • .RepetitionPenaltyLogitsProcessor
    • new RepetitionPenaltyLogitsProcessor(penalty)
    • ._call(input_ids, logits) ⇒ Object
  • .MinLengthLogitsProcessor
    • new MinLengthLogitsProcessor(min_length, eos_token_id)
    • ._call(input_ids, logits) ⇒ Object
  • .MinNewTokensLengthLogitsProcessor
    • new MinNewTokensLengthLogitsProcessor(prompt_length_to_skip, min_new_tokens, eos_token_id)
    • ._call(input_ids, logits) ⇒ Object
  • .NoBadWordsLogitsProcessor
    • new NoBadWordsLogitsProcessor(bad_words_ids, eos_token_id)
    • ._call(input_ids, logits) ⇒ Object
  • .ClassifierFreeGuidanceLogitsProcessor
    • new ClassifierFreeGuidanceLogitsProcessor(guidance_scale)
    • ._call(input_ids, logits) ⇒ Object
  • .TemperatureLogitsWarper
    • new TemperatureLogitsWarper(temperature)
    • ._call(input_ids, logits) ⇒ Object
  • .TopPLogitsWarper
    • new TopPLogitsWarper(top_p, options)
  • .TopKLogitsWarper
    • new TopKLogitsWarper(top_k, options)

---

generation/logits_process.LogitsProcessor

Abstract base class for all logit processors that can be applied during generation.

Kind: static class of generation/logits_process

---

logitsProcessor._call(input_ids, logits)

Apply the processor to the input logits.

Kind: instance abstract method of LogitsProcessor
Throws:

• Error Throws an error if `_call` is not implemented in the subclass.

Param	Type	Description
input_ids	Array.<Array<bigint>>	The input ids.
logits	Tensor	The logits to process.

---

generation/logits_process.LogitsWarper

Abstract base class for all logit warpers that can be applied during generation with multinomial sampling.

Kind: static class of generation/logits_process

---

logitsWarper._call(input_ids, logits)

Apply the processor to the input logits.

Kind: instance abstract method of LogitsWarper
Throws:

• Error Throws an error if `_call` is not implemented in the subclass.

Param	Type	Description
input_ids	Array.<Array<bigint>>	The input ids.
logits	Tensor	The logits to process.

---

generation/logits_process.LogitsProcessorList

A class representing a list of logits processors. A logits processor is a function that modifies the logits output of a language model. This class provides methods for adding new processors and applying all processors to a batch of logits.

Kind: static class of generation/logits_process

• .LogitsProcessorList
  • new LogitsProcessorList()
  • .push(item)
  • .extend(items)
  • ._call(input_ids, logits)

---

new LogitsProcessorList()

Constructs a new instance of LogitsProcessorList.

---

logitsProcessorList.push(item)

Adds a new logits processor to the list.

Kind: instance method of LogitsProcessorList

Param	Type	Description
item	LogitsProcessor	The logits processor function to add.

---

logitsProcessorList.extend(items)

Adds multiple logits processors to the list.

Kind: instance method of LogitsProcessorList

Param	Type	Description
items	Array.<LogitsProcessor>	The logits processor functions to add.

---

logitsProcessorList._call(input_ids, logits)

Applies all logits processors in the list to a batch of logits, modifying them in-place.

Kind: instance method of LogitsProcessorList

Param	Type	Description
input_ids	Array.<Array<bigint>>	The input IDs for the language model.
logits	Tensor

---

generation/logits_process.ForcedBOSTokenLogitsProcessor

A LogitsProcessor that forces a BOS token at the beginning of the generated sequence.

Kind: static class of generation/logits_process

• .ForcedBOSTokenLogitsProcessor
  • new ForcedBOSTokenLogitsProcessor(bos_token_id)
  • ._call(input_ids, logits) ⇒ Object

---

new ForcedBOSTokenLogitsProcessor(bos_token_id)

Create a ForcedBOSTokenLogitsProcessor.

Param	Type	Description
bos_token_id	number	The ID of the beginning-of-sequence token to be forced.

---

forcedBOSTokenLogitsProcessor._call(input_ids, logits) ⇒ <code> Object </code>

Apply the BOS token forcing to the logits.

Kind: instance method of ForcedBOSTokenLogitsProcessor
Returns: Object - The logits with BOS token forcing.

Param	Type	Description
input_ids	Array.<Array<bigint>>	The input IDs.
logits	Tensor	The logits.

---

generation/logits_process.ForcedEOSTokenLogitsProcessor

A logits processor that enforces the specified token as the last generated token when max_length is reached.

Kind: static class of generation/logits_process

• .ForcedEOSTokenLogitsProcessor
  • new ForcedEOSTokenLogitsProcessor(max_length, eos_token_id)
  • ._call(input_ids, logits)

---

new ForcedEOSTokenLogitsProcessor(max_length, eos_token_id)

Create a ForcedEOSTokenLogitsProcessor.

Param	Type	Description
max_length	number	The maximum length of the sequence to be generated.
eos_token_id	number | Array<number>	The id(s) of the end-of-sequence token.

---

forcedEOSTokenLogitsProcessor._call(input_ids, logits)

Apply the processor to input_ids and logits.

Kind: instance method of ForcedEOSTokenLogitsProcessor

Param	Type	Description
input_ids	Array.<Array<bigint>>	The input ids.
logits	Tensor	The logits tensor.

---

generation/logits_process.SuppressTokensAtBeginLogitsProcessor

A LogitsProcessor that suppresses a list of tokens as soon as the generate function starts generating using begin_index tokens. This should ensure that the tokens defined by begin_suppress_tokens at not sampled at the begining of the generation.

Kind: static class of generation/logits_process

• .SuppressTokensAtBeginLogitsProcessor
  • new SuppressTokensAtBeginLogitsProcessor(begin_suppress_tokens, begin_index)
  • ._call(input_ids, logits) ⇒ Object

---

new SuppressTokensAtBeginLogitsProcessor(begin_suppress_tokens, begin_index)

Create a SuppressTokensAtBeginLogitsProcessor.

Param	Type	Description
begin_suppress_tokens	Array.<number>	The IDs of the tokens to suppress.
begin_index	number	The number of tokens to generate before suppressing tokens.

---

suppressTokensAtBeginLogitsProcessor._call(input_ids, logits) ⇒ <code> Object </code>

Apply the BOS token forcing to the logits.

Kind: instance method of SuppressTokensAtBeginLogitsProcessor
Returns: Object - The logits with BOS token forcing.

Param	Type	Description
input_ids	Array.<Array<bigint>>	The input IDs.
logits	Tensor	The logits.

---

generation/logits_process.WhisperTimeStampLogitsProcessor

A LogitsProcessor that handles adding timestamps to generated text.

Kind: static class of generation/logits_process

• .WhisperTimeStampLogitsProcessor
  • new WhisperTimeStampLogitsProcessor(generate_config, init_tokens)
  • ._call(input_ids, logits) ⇒ Tensor

---

new WhisperTimeStampLogitsProcessor(generate_config, init_tokens)

Constructs a new WhisperTimeStampLogitsProcessor.

Param	Type	Description
generate_config	*	The config object passed to the generate() method of a transformer model.
init_tokens	Array.<number>	The initial tokens of the input sequence.

---

whisperTimeStampLogitsProcessor._call(input_ids, logits) ⇒ <code> Tensor </code>

Modify the logits to handle timestamp tokens.

Kind: instance method of WhisperTimeStampLogitsProcessor
Returns: Tensor - The modified logits.

Param	Type	Description
input_ids	Array.<Array<bigint>>	The input sequence of tokens.
logits	Tensor	The logits output by the model.

---

generation/logits_process.NoRepeatNGramLogitsProcessor

A logits processor that disallows ngrams of a certain size to be repeated.

Kind: static class of generation/logits_process

• .NoRepeatNGramLogitsProcessor
  • new NoRepeatNGramLogitsProcessor(no_repeat_ngram_size)
  • .getNgrams(prevInputIds) ⇒ Map.<string, Array<number>>
  • .getGeneratedNgrams(bannedNgrams, prevInputIds) ⇒ Array.<number>
  • .calcBannedNgramTokens(prevInputIds) ⇒ Array.<number>
  • ._call(input_ids, logits) ⇒ Object

---

new NoRepeatNGramLogitsProcessor(no_repeat_ngram_size)

Create a NoRepeatNGramLogitsProcessor.

Param	Type	Description
no_repeat_ngram_size	number	The no-repeat-ngram size. All ngrams of this size can only occur once.

---

noRepeatNGramLogitsProcessor.getNgrams(prevInputIds) ⇒ <code> Map. < string, Array < number > > </code>

Generate n-grams from a sequence of token ids.

Kind: instance method of NoRepeatNGramLogitsProcessor
Returns: Map.<string, Array<number>> - Map of generated n-grams

Param	Type	Description
prevInputIds	Array.<bigint>	List of previous input ids

---

noRepeatNGramLogitsProcessor.getGeneratedNgrams(bannedNgrams, prevInputIds) ⇒ <code> Array. < number > </code>

Generate n-grams from a sequence of token ids.

Kind: instance method of NoRepeatNGramLogitsProcessor
Returns: Array.<number> - Map of generated n-grams

Param	Type	Description
bannedNgrams	Map.<string, Array<number>>	Map of banned n-grams
prevInputIds	Array.<bigint>	List of previous input ids

---

noRepeatNGramLogitsProcessor.calcBannedNgramTokens(prevInputIds) ⇒ <code> Array. < number > </code>

Calculate banned n-gram tokens

Kind: instance method of NoRepeatNGramLogitsProcessor
Returns: Array.<number> - Map of generated n-grams

Param	Type	Description
prevInputIds	Array.<bigint>	List of previous input ids

---

noRepeatNGramLogitsProcessor._call(input_ids, logits) ⇒ <code> Object </code>

Apply the no-repeat-ngram processor to the logits.

Kind: instance method of NoRepeatNGramLogitsProcessor
Returns: Object - The logits with no-repeat-ngram processing.

Param	Type	Description
input_ids	Array.<Array<bigint>>	The input IDs.
logits	Tensor	The logits.

---

generation/logits_process.RepetitionPenaltyLogitsProcessor

A logits processor that penalises repeated output tokens.

Kind: static class of generation/logits_process

• .RepetitionPenaltyLogitsProcessor
  • new RepetitionPenaltyLogitsProcessor(penalty)
  • ._call(input_ids, logits) ⇒ Object

---

new RepetitionPenaltyLogitsProcessor(penalty)

Create a RepetitionPenaltyLogitsProcessor.

Param	Type	Description
penalty	number	The penalty to apply for repeated tokens.

---

repetitionPenaltyLogitsProcessor._call(input_ids, logits) ⇒ <code> Object </code>

Apply the repetition penalty to the logits.

Kind: instance method of RepetitionPenaltyLogitsProcessor
Returns: Object - The logits with repetition penalty processing.

Param	Type	Description
input_ids	Array.<Array<bigint>>	The input IDs.
logits	Tensor	The logits.

---

generation/logits_process.MinLengthLogitsProcessor

A logits processor that enforces a minimum number of tokens.

Kind: static class of generation/logits_process

• .MinLengthLogitsProcessor
  • new MinLengthLogitsProcessor(min_length, eos_token_id)
  • ._call(input_ids, logits) ⇒ Object

---

new MinLengthLogitsProcessor(min_length, eos_token_id)

Create a MinLengthLogitsProcessor.

Param	Type	Description
min_length	number	The minimum length below which the score of eos_token_id is set to negative infinity.
eos_token_id	number | Array<number>	The ID/IDs of the end-of-sequence token.

---

minLengthLogitsProcessor._call(input_ids, logits) ⇒ <code> Object </code>

Apply logit processor.

Kind: instance method of MinLengthLogitsProcessor
Returns: Object - The processed logits.

Param	Type	Description
input_ids	Array.<Array<bigint>>	The input IDs.
logits	Tensor	The logits.

---

generation/logits_process.MinNewTokensLengthLogitsProcessor

A logits processor that enforces a minimum number of new tokens.

Kind: static class of generation/logits_process

• .MinNewTokensLengthLogitsProcessor
  • new MinNewTokensLengthLogitsProcessor(prompt_length_to_skip, min_new_tokens, eos_token_id)
  • ._call(input_ids, logits) ⇒ Object

---

new MinNewTokensLengthLogitsProcessor(prompt_length_to_skip, min_new_tokens, eos_token_id)

Create a MinNewTokensLengthLogitsProcessor.

Param	Type	Description
prompt_length_to_skip	number	The input tokens length.
min_new_tokens	number	The minimum new tokens length below which the score of eos_token_id is set to negative infinity.
eos_token_id	number | Array<number>	The ID/IDs of the end-of-sequence token.

---

minNewTokensLengthLogitsProcessor._call(input_ids, logits) ⇒ <code> Object </code>

Apply logit processor.

Kind: instance method of MinNewTokensLengthLogitsProcessor
Returns: Object - The processed logits.

Param	Type	Description
input_ids	Array.<Array<bigint>>	The input IDs.
logits	Tensor	The logits.

---

generation/logits_process.NoBadWordsLogitsProcessor

Kind: static class of generation/logits_process

• .NoBadWordsLogitsProcessor
  • new NoBadWordsLogitsProcessor(bad_words_ids, eos_token_id)
  • ._call(input_ids, logits) ⇒ Object

---

new NoBadWordsLogitsProcessor(bad_words_ids, eos_token_id)

Create a NoBadWordsLogitsProcessor.

Param	Type	Description
bad_words_ids	Array.<Array<number>>	List of list of token ids that are not allowed to be generated.
eos_token_id	number | Array<number>	The id of the end-of-sequence token. Optionally, use a list to set multiple end-of-sequence tokens.

---

noBadWordsLogitsProcessor._call(input_ids, logits) ⇒ <code> Object </code>

Apply logit processor.

Kind: instance method of NoBadWordsLogitsProcessor
Returns: Object - The processed logits.

Param	Type	Description
input_ids	Array.<Array<bigint>>	The input IDs.
logits	Tensor	The logits.

---

generation/logits_process.ClassifierFreeGuidanceLogitsProcessor

[LogitsProcessor] for classifier free guidance (CFG). The scores are split over the batch dimension, where the first half correspond to the conditional logits (predicted from the input prompt) and the second half correspond to the unconditional logits (predicted from an empty or ‘null’ prompt). The processor computes a weighted average across the conditional and unconditional logits, parameterised by the guidance_scale.

See the paper for more information.

Kind: static class of generation/logits_process

• .ClassifierFreeGuidanceLogitsProcessor
  • new ClassifierFreeGuidanceLogitsProcessor(guidance_scale)
  • ._call(input_ids, logits) ⇒ Object

---

new ClassifierFreeGuidanceLogitsProcessor(guidance_scale)

Create a ClassifierFreeGuidanceLogitsProcessor.

Param	Type	Description
guidance_scale	number	The guidance scale for classifier free guidance (CFG). CFG is enabled by setting guidance_scale > 1. Higher guidance scale encourages the model to generate samples that are more closely linked to the input prompt, usually at the expense of poorer quality.

---

classifierFreeGuidanceLogitsProcessor._call(input_ids, logits) ⇒ <code> Object </code>

Apply logit processor.

Kind: instance method of ClassifierFreeGuidanceLogitsProcessor
Returns: Object - The processed logits.

Param	Type	Description
input_ids	Array.<Array<bigint>>	The input IDs.
logits	Tensor	The logits.

---

generation/logits_process.TemperatureLogitsWarper

[LogitsWarper] for temperature (exponential scaling output probability distribution), which effectively means that it can control the randomness of the predicted tokens. Often used together with [TopPLogitsWarper] and [TopKLogitsWarper].

Kind: static class of generation/logits_process

• .TemperatureLogitsWarper
  • new TemperatureLogitsWarper(temperature)
  • ._call(input_ids, logits) ⇒ Object

---

new TemperatureLogitsWarper(temperature)

Create a TemperatureLogitsWarper.

Param	Type	Description
temperature	number	Strictly positive float value used to modulate the logits distribution. A value smaller than 1 decreases randomness (and vice versa), with 0 being equivalent to shifting all probability mass to the most likely token.

---

temperatureLogitsWarper._call(input_ids, logits) ⇒ <code> Object </code>

Apply logit warper.

Kind: instance method of TemperatureLogitsWarper
Returns: Object - The processed logits.

Param	Type	Description
input_ids	Array.<Array<bigint>>	The input IDs.
logits	Tensor	The logits.

---

generation/logits_process.TopPLogitsWarper

[LogitsWarper] that performs top-p, i.e. restricting to top tokens summing to prob_cut_off <= prob_cut_off. Often used together with [TemperatureLogitsWarper] and [TopKLogitsWarper].

Kind: static class of generation/logits_process

---

new TopPLogitsWarper(top_p, options)

Create a TopPLogitsWarper.

Param	Type	Default	Description
top_p	number		If set to < 1, only the smallest set of most probable tokens with probabilities that add up to top_p or higher are kept for generation.
options	Object		Additional options for the top-p sampling.
[options.filter_value]	number	-Infinity	All filtered values will be set to this float value.
[options.min_tokens_to_keep]	number	1	Minimum number of tokens that cannot be filtered.

---

generation/logits_process.TopKLogitsWarper

[LogitsWarper] that performs top-k, i.e. restricting to the k highest probability elements. Often used together with [TemperatureLogitsWarper] and [TopPLogitsWarper].

Kind: static class of generation/logits_process

---

new TopKLogitsWarper(top_k, options)

Create a TopKLogitsWarper.

Param	Type	Default	Description
top_k	number		If set to > 0, only the top top_k tokens are kept for generation.
options	Object		Additional options for the top-k sampling.
[options.filter_value]	number	-Infinity	All filtered values will be set to this float value.
[options.min_tokens_to_keep]	number	1	Minimum number of tokens that cannot be filtered.

---

< > Update on GitHub