参数

• token_ids_0 — （可能已格式化）序列的 ID 列表。
• token_ids_1 — 当 already_has_special_tokens=True 时未使用。在这种情况下必须为 None。
• already_has_special_tokens — 序列是否已用特殊标记格式化。

特殊令牌为 1，序列令牌为 0。

参数

• input_ids (torch.Tensor, 形状为 (batch_size, sequence_length), optional) — 词汇表中输入序列标记的索引。默认情况下将忽略填充。

  可以使用 AutoTokenizer 获取索引。有关详细信息，请参阅 PreTrainedTokenizer.encode() 和 PreTrainedTokenizer.call()。

  什么是输入 ID？

• attention_mask (torch.Tensor, 形状为 (batch_size, sequence_length), optional) — 避免对填充标记索引执行注意力的掩码。掩码值选择在 [0, 1] 中：
  • 1 表示未掩码的标记，
  • 0 表示已掩码的标记。

  什么是注意力掩码？

• position_ids (torch.Tensor, 形状为 (batch_size, sequence_length), optional) — 位置嵌入中每个输入序列标记的位置索引。选择范围为 [0, config.n_positions - 1]。

  什么是位置 ID？

• encoder_hidden_states (torch.FloatTensor, 形状为 (batch_size, encoder_sequence_length, hidden_size), optional) — 编码器最后一层的输出隐藏状态序列。用于解码器的交叉注意力。
• encoder_attention_mask (torch.LongTensor, 形状为 (batch_size, encoder_sequence_length), optional) — 避免对编码器 input_ids 的填充标记执行交叉注意力的掩码。掩码值选择在 [0, 1] 中：
  • 1 表示未掩码的标记，
  • 0 表示已掩码的标记。

  什么是注意力掩码？

• past_key_values (~cache_utils.Cache, optional) — 可用于加速顺序解码的预计算隐藏状态（自注意力块和交叉注意力块中的键和值）。这通常包括在解码的先前阶段由模型返回的 past_key_values，当 use_cache=True 或 config.use_cache=True 时。

  仅允许 Cache 实例作为输入，请参阅我们的 kv 缓存指南。如果未传入 past_key_values，则默认初始化 DynamicCache。

  模型将输出与输入相同的缓存格式。

  如果使用 past_key_values，则用户应仅传入未处理的 input_ids（其过去键值状态未提供给此模型的那些），形状为 (batch_size, unprocessed_length)，而不是所有 input_ids，形状为 (batch_size, sequence_length)。

• inputs_embeds (torch.Tensor, 形状为 (batch_size, sequence_length, hidden_size), optional) — 可选地，您可以直接传入嵌入表示，而不是传入 input_ids。如果您想比模型内部的嵌入查找矩阵更精确地控制如何将 input_ids 索引转换为关联向量，这将很有用。
• use_cache (bool, optional) — 如果设置为 True，则返回 past_key_values 键值状态，并可用于加速解码。
• output_attentions (bool, optional) — 是否返回所有注意力层的注意力张量。有关详细信息，请参阅返回张量下的 attentions。
• output_hidden_states (bool, optional) — 是否返回所有层的隐藏状态。有关详细信息，请参阅返回张量下的 hidden_states。
• return_dict (bool, optional) — 是否返回一个 ModelOutput 而不是一个普通的元组。
• cache_position (torch.Tensor of shape (sequence_length), optional) — Indices depicting the position of the input sequence tokens in the sequence. Contrarily to position_ids, this tensor is not affected by padding. It is used to update the cache in the correct position and to infer the complete sequence length.

A transformers.modeling_outputs.BaseModelOutputWithPastAndCrossAttentions or a tuple of torch.FloatTensor (if return_dict=False is passed or when config.return_dict=False) comprising various elements depending on the configuration (XGLMConfig) and inputs.

• last_hidden_state (torch.FloatTensor, 形状为 (batch_size, sequence_length, hidden_size)) — 模型最后一层输出的隐藏状态序列。

  如果使用了 past_key_values，则只输出形状为 (batch_size, 1, hidden_size) 的序列的最后一个隐藏状态。

• past_key_values (Cache, optional, 当传递 use_cache=True 或当 config.use_cache=True 时返回) — 它是 Cache 实例。更多详情，请参阅我们的 kv cache 指南。

  Contains pre-computed hidden-states (key and values in the self-attention blocks and optionally if config.is_encoder_decoder=True in the cross-attention blocks) that can be used (see past_key_values input) to speed up sequential decoding.

• hidden_states (tuple(torch.FloatTensor), optional, 当传递 output_hidden_states=True 或当 config.output_hidden_states=True 时返回) — torch.FloatTensor 的元组（一个用于嵌入层的输出，如果模型有嵌入层；+一个用于每个层的输出），形状为 (batch_size, sequence_length, hidden_size)。

  模型在每个层输出的隐藏状态以及可选的初始嵌入输出。

• attentions (tuple(torch.FloatTensor), optional, 当传递 output_attentions=True 或当 config.output_attentions=True 时返回) — torch.FloatTensor 的元组（每个层一个），形状为 (batch_size, num_heads, sequence_length, sequence_length)。

  注意力 softmax 后的注意力权重，用于计算自注意力头中的加权平均值。

• cross_attentions (tuple(torch.FloatTensor), optional, returned when output_attentions=True and config.add_cross_attention=True is passed or when config.output_attentions=True) — Tuple of torch.FloatTensor (one for each layer) of shape (batch_size, num_heads, sequence_length, sequence_length).

  解码器交叉注意力层的注意力权重，在注意力 softmax 之后，用于计算交叉注意力头中的加权平均。

参数

• input_ids (torch.Tensor of shape (batch_size, sequence_length), optional) — Indices of input sequence tokens in the vocabulary. Padding will be ignored by default.

  Indices can be obtained using AutoTokenizer. See PreTrainedTokenizer.encode() and PreTrainedTokenizer.call() for details.

  What are input IDs?

• attention_mask (torch.Tensor of shape (batch_size, sequence_length), optional) — Mask to avoid performing attention on padding token indices. Mask values selected in [0, 1]:
  • 1 for tokens that are not masked,
  • 0 for tokens that are masked.

  What are attention masks?

• position_ids (torch.Tensor of shape (batch_size, sequence_length), optional) — Indices of positions of each input sequence tokens in the position embeddings. Selected in the range [0, config.n_positions - 1].

  What are position IDs?

• encoder_hidden_states (torch.FloatTensor of shape (batch_size, encoder_sequence_length, hidden_size), optional) — Sequence of hidden-states at the output of the last layer of the encoder. Used in the cross-attention of the decoder.
• encoder_attention_mask (torch.LongTensor of shape (batch_size, encoder_sequence_length), optional) — Mask to avoid performing cross-attention on padding tokens indices of encoder input_ids. Mask values selected in [0, 1]:
  • 1 for tokens that are not masked,
  • 0 for tokens that are masked.

  What are attention masks?

• past_key_values (~cache_utils.Cache, optional) — Pre-computed hidden-states (key and values in the self-attention blocks and in the cross-attention blocks) that can be used to speed up sequential decoding. This typically consists in the past_key_values returned by the model at a previous stage of decoding, when use_cache=True or config.use_cache=True.

  Only Cache instance is allowed as input, see our kv cache guide. If no past_key_values are passed, DynamicCache will be initialized by default.

  The model will output the same cache format that is fed as input.

  If past_key_values are used, the user is expected to input only unprocessed input_ids (those that don’t have their past key value states given to this model) of shape (batch_size, unprocessed_length) instead of all input_ids of shape (batch_size, sequence_length).

• inputs_embeds (torch.Tensor of shape (batch_size, sequence_length, hidden_size), optional) — Optionally, instead of passing input_ids you can choose to directly pass an embedded representation. This is useful if you want more control over how to convert input_ids indices into associated vectors than the model’s internal embedding lookup matrix.
• labels (torch.LongTensor of shape (batch_size, sequence_length), optional) — Labels for computing the masked language modeling loss. Indices should either be in [0, ..., config.vocab_size] or -100 (see input_ids docstring). Tokens with indices set to -100 are ignored (masked), the loss is only computed for the tokens with labels in [0, ..., config.vocab_size].
• use_cache (bool, optional) — If set to True, past_key_values key value states are returned and can be used to speed up decoding (see past_key_values).
• output_attentions (bool, optional) — Whether or not to return the attentions tensors of all attention layers. See attentions under returned tensors for more detail.
• output_hidden_states (bool, optional) — Whether or not to return the hidden states of all layers. See hidden_states under returned tensors for more detail.
• return_dict (bool, optional) — Whether or not to return a ModelOutput instead of a plain tuple.
• cache_position (torch.Tensor of shape (sequence_length), optional) — Indices depicting the position of the input sequence tokens in the sequence. Contrarily to position_ids, this tensor is not affected by padding. It is used to update the cache in the correct position and to infer the complete sequence length.
• logits_to_keep (Union[int, torch.Tensor], optional, defaults to 0) — If an int, compute logits for the last logits_to_keep tokens. If 0, calculate logits for all input_ids (special case). Only last token logits are needed for generation, and calculating them only for that token can save memory, which becomes pretty significant for long sequences or large vocabulary size. If a torch.Tensor, must be 1D corresponding to the indices to keep in the sequence length dimension. This is useful when using packed tensor format (single dimension for batch and sequence length).

A transformers.modeling_outputs.CausalLMOutputWithCrossAttentions or a tuple of torch.FloatTensor (if return_dict=False is passed or when config.return_dict=False) comprising various elements depending on the configuration (XGLMConfig) and inputs.

• loss (torch.FloatTensor 形状为 (1,)，可选，当提供 labels 时返回) — 语言建模损失（用于下一个 token 预测）。

• logits (形状为 (batch_size, sequence_length, config.vocab_size) 的 torch.FloatTensor) — 语言建模头部的预测分数（SoftMax 之前的每个词汇标记的分数）。

• hidden_states (tuple(torch.FloatTensor), optional, 当传递 output_hidden_states=True 或当 config.output_hidden_states=True 时返回) — torch.FloatTensor 的元组（一个用于嵌入层的输出，如果模型有嵌入层；+一个用于每个层的输出），形状为 (batch_size, sequence_length, hidden_size)。

  模型在每个层输出的隐藏状态以及可选的初始嵌入输出。

• attentions (tuple(torch.FloatTensor), optional, 当传递 output_attentions=True 或当 config.output_attentions=True 时返回) — torch.FloatTensor 的元组（每个层一个），形状为 (batch_size, num_heads, sequence_length, sequence_length)。

  注意力 softmax 后的注意力权重，用于计算自注意力头中的加权平均值。

• cross_attentions (tuple(torch.FloatTensor), optional, returned when output_attentions=True is passed or when config.output_attentions=True) — Tuple of torch.FloatTensor (one for each layer) of shape (batch_size, num_heads, sequence_length, sequence_length).

  注意力 softmax 后的交叉注意力权重，用于计算交叉注意力头中的加权平均。

• past_key_values (Cache, optional, 当传递 use_cache=True 或当 config.use_cache=True 时返回) — 它是 Cache 实例。更多详情，请参阅我们的 kv cache 指南。

  包含预先计算的隐藏状态（注意力块中的键和值），可用于（参见 past_key_values 输入）加速顺序解码。