参数

• input_ids (torch.LongTensor, shape (batch_size, sequence_length), optional) — Indices of input sequence tokens in the vocabulary. 默认情况下会忽略填充。

  可以使用 AutoTokenizer 获取索引。有关详细信息，请参阅 PreTrainedTokenizer.encode() 和 PreTrainedTokenizer.call()。

  输入 ID 是什么？

• attention_mask (torch.Tensor, shape (batch_size, sequence_length), optional) — Mask to avoid performing attention on padding token indices. Mask values selected in [0, 1]:
  • 1 for tokens that are not masked,
  • 0 for tokens that are masked.

  注意掩码是什么？

• position_ids (torch.LongTensor, shape (batch_size, sequence_length), optional) — Indices of positions of each input sequence tokens in the position embeddings. Selected in the range [0, config.n_positions - 1].

  位置 ID 是什么？

• cross_attention_states (torch.FloatTensor, optional) — 视觉模型的输出，用于交叉注意力。此张量包含语言模型将关注的已处理图像特征。
• cross_attention_mask (torch.Tensor, shape (batch_size, seq_length, max_num_images, max_num_tiles), optional) — Cross-attention mask to control the interaction between text tokens and image tiles. This 4D tensor defines which image tiles each text token should attend to.

  For each text token (in seq_length):

  • 1 indicates the token should attend to the corresponding image tile
  • 0 indicates the token should not attend to the corresponding image tile
• full_text_row_masked_out_mask (tuple[torch.Tensor, torch.Tensor], optional) — A tuple containing two tensors that mask out rows in the cross-attention mechanism:
  • The first tensor has shape (batch_size, 1, seq_length, 1) and contains values of 0 or 1. A value of 0 indicates that the corresponding text token’s entire row in the cross-attention matrix should be masked out (all image tokens ignored).
  • The second tensor has the same shape and is used internally to apply the masking during the forward pass of cross-attention layers. This mask is derived from the cross_attention_mask and is used to handle cases where a text token should not attend to any image token.
• past_key_values (~cache_utils.Cache, optional) — Pre-computed hidden-states (key and values in the self-attention blocks and in the cross-attention blocks) that can be used to speed up sequential decoding. This typically consists in the past_key_values returned by the model at a previous stage of decoding, when use_cache=True or config.use_cache=True.

  Only Cache instance is allowed as input, see our kv cache guide. If no past_key_values are passed, DynamicCache will be initialized by default.

  The model will output the same cache format that is fed as input.

  If past_key_values are used, the user is expected to input only unprocessed input_ids (those that don’t have their past key value states given to this model) of shape (batch_size, unprocessed_length) instead of all input_ids of shape (batch_size, sequence_length).

• inputs_embeds (torch.FloatTensor, shape (batch_size, sequence_length, hidden_size), optional) — Optionally, instead of passing input_ids you can choose to directly pass an embedded representation. This is useful if you want more control over how to convert input_ids indices into associated vectors than the model’s internal embedding lookup matrix.
• labels (torch.LongTensor, shape (batch_size, sequence_length), optional) — Labels for computing the masked language modeling loss. Indices should either be in [0, ..., config.vocab_size] or -100 (see input_ids docstring). Tokens with indices set to -100 are ignored (masked), the loss is only computed for the tokens with labels in [0, ..., config.vocab_size].
• use_cache (bool, optional) — If set to True, past_key_values key value states are returned and can be used to speed up decoding (see past_key_values).
• cache_position (torch.LongTensor, shape (sequence_length), optional) — Indices depicting the position of the input sequence tokens in the sequence. Contrarily to position_ids, this tensor is not affected by padding. It is used to update the cache in the correct position and to infer the complete sequence length.
• logits_to_keep (Union[int, torch.Tensor], optional, defaults to 0) — If an int, compute logits for the last logits_to_keep tokens. If 0, calculate logits for all input_ids (special case). Only last token logits are needed for generation, and calculating them only for that token can save memory, which becomes pretty significant for long sequences or large vocabulary size. If a torch.Tensor, must be 1D corresponding to the indices to keep in the sequence length dimension. This is useful when using packed tensor format (single dimension for batch and sequence length).

一个 transformers.modeling_outputs.CausalLMOutputWithPast 或一个 torch.FloatTensor 的元组（如果传递了 return_dict=False 或当 config.return_dict=False 时），包含根据配置（BltConfig）和输入而变化的各种元素。

• loss (torch.FloatTensor 形状为 (1,)，可选，当提供 labels 时返回) — 语言建模损失（用于下一个 token 预测）。

• logits (形状为 (batch_size, sequence_length, config.vocab_size) 的 torch.FloatTensor) — 语言建模头部的预测分数（SoftMax 之前的每个词汇标记的分数）。

• past_key_values (Cache, optional, 当传递 use_cache=True 或当 config.use_cache=True 时返回) — 它是 Cache 实例。更多详情，请参阅我们的 kv cache 指南。

  包含预计算的隐藏状态（自注意力块中的键和值），可用于（参见 past_key_values 输入）加速顺序解码。

• hidden_states (tuple(torch.FloatTensor), optional, 当传递 output_hidden_states=True 或当 config.output_hidden_states=True 时返回) — torch.FloatTensor 的元组（一个用于嵌入层的输出，如果模型有嵌入层；+一个用于每个层的输出），形状为 (batch_size, sequence_length, hidden_size)。

  模型在每个层输出的隐藏状态以及可选的初始嵌入输出。

• attentions (tuple(torch.FloatTensor), optional, 当传递 output_attentions=True 或当 config.output_attentions=True 时返回) — torch.FloatTensor 的元组（每个层一个），形状为 (batch_size, num_heads, sequence_length, sequence_length)。

  注意力 softmax 后的注意力权重，用于计算自注意力头中的加权平均值。