Diffusers 文档
HunyuanDiT2DModel
并获得增强的文档体验
开始使用
HunyuanDiT2DModel
来自 Hunyuan-DiT 的 2D 数据扩散 Transformer 模型。
HunyuanDiT2DModel
class diffusers.HunyuanDiT2DModel
< source 查看源文件 >( num_attention_heads: int = 16 attention_head_dim: int = 88 in_channels: typing.Optional[int] = None patch_size: typing.Optional[int] = None activation_fn: str = 'gelu-approximate' sample_size = 32 hidden_size = 1152 num_layers: int = 28 mlp_ratio: float = 4.0 learn_sigma: bool = True cross_attention_dim: int = 1024 norm_type: str = 'layer_norm' cross_attention_dim_t5: int = 2048 pooled_projection_dim: int = 1024 text_len: int = 77 text_len_t5: int = 256 use_style_cond_and_image_meta_size: bool = True )
参数
- num_attention_heads (
int, 可选, 默认值为 16) — 用于多头注意力机制的头数。 - attention_head_dim (
int, 可选, 默认值为 88) — 每个注意力头的通道数。 - in_channels (
int, 可选) — 输入和输出的通道数（如果输入是连续的，请指定）。 - patch_size (
int, 可选) — 用于输入的块大小。 - activation_fn (
str, 可选, 默认值为"geglu") — 前馈网络中使用的激活函数。 - sample_size (
int, 可选) — 潜在图像的宽度。在训练过程中是固定的，因为它用于学习多个位置嵌入。 - dropout (
float, 可选, 默认值为 0.0) — 使用的 dropout 概率。 - cross_attention_dim (
int, 可选) — Clip 文本嵌入的维度数。 - hidden_size (
int, 可选) — 条件嵌入层中的隐藏层大小。 - num_layers (
int, 可选, 默认值为 1) — 要使用的 Transformer 块层数。 - mlp_ratio (
float, 可选, 默认值为 4.0) — 隐藏层大小与输入大小之比。 - learn_sigma (
bool, 可选, 默认值为True) — 是否预测方差。 - cross_attention_dim_t5 (
int, 可选) — T5 文本嵌入的维度数。 - pooled_projection_dim (
int, 可选) — 池化投影的尺寸。 - text_len (
int, 可选) — Clip 文本嵌入的长度。 - text_len_t5 (
int, 可选) — T5 文本嵌入的长度。 - use_style_cond_and_image_meta_size (
bool, 可选) — 是否使用样式条件和图像元大小。版本 <=1.1 为 True，版本 >= 1.2 为 False。
HunYuanDiT：具有 Transformer 主干的扩散模型。
继承 ModelMixin 和 ConfigMixin 以兼容 diffusers 的 StableDiffusionPipeline 采样器。
启用前向分块
< source 查看源文件 >( chunk_size: typing.Optional[int] = None dim: int = 0 )
设置注意力处理器以使用分块前馈层。
forward
< source 查看源文件 >( hidden_states timestep encoder_hidden_states = None text_embedding_mask = None encoder_hidden_states_t5 = None text_embedding_mask_t5 = None image_meta_size = None style = None image_rotary_emb = None controlnet_block_samples = None return_dict = True )
参数
- hidden_states (
torch.Tensor，形状为(批次大小, 维度, 高度, 宽度)) — 输入张量。 - timestep (
torch.LongTensor, 可选) — 用于指示去噪步骤。 - encoder_hidden_states (
torch.Tensor，形状为(批次大小, 序列长度, 嵌入维度), 可选) — 交叉注意力层的条件嵌入。这是BertModel的输出。 - text_embedding_mask — torch.Tensor 形状为
(批次, 键令牌)的注意力掩码应用于encoder_hidden_states。这是BertModel的输出。 - encoder_hidden_states_t5 (
torch.Tensor，形状为(批次大小, 序列长度, 嵌入维度), 可选) — 交叉注意力层的条件嵌入。这是 T5 文本编码器的输出。 - text_embedding_mask_t5 — torch.Tensor 形状为
(批次, 键令牌)的注意力掩码应用于encoder_hidden_states。这是 T5 文本编码器的输出。 - image_meta_size (torch.Tensor) — 用于指示图像大小的条件嵌入
- style — torch.Tensor: 指示风格的条件嵌入
- image_rotary_emb (
torch.Tensor) — 在注意力计算期间应用于查询和键张量的图像旋转嵌入。 - return_dict — bool 是否返回字典。
HunyuanDiT2DModel 的 forward 方法。
启用融合 QKV 投影。对于自注意力模块，所有投影矩阵（即查询、键、值）都将融合。对于交叉注意力模块，键和值投影矩阵将融合。
此 API 是 🧪 实验性的。
设置注意力处理器
< source 查看源文件 >( processor: typing.Union[diffusers.models.attention_processor.AttnProcessor, diffusers.models.attention_processor.CustomDiffusionAttnProcessor, diffusers.models.attention_processor.AttnAddedKVProcessor, diffusers.models.attention_processor.AttnAddedKVProcessor2_0, diffusers.models.attention_processor.JointAttnProcessor2_0, diffusers.models.attention_processor.PAGJointAttnProcessor2_0, diffusers.models.attention_processor.PAGCFGJointAttnProcessor2_0, diffusers.models.attention_processor.FusedJointAttnProcessor2_0, diffusers.models.attention_processor.AllegroAttnProcessor2_0, diffusers.models.attention_processor.AuraFlowAttnProcessor2_0, diffusers.models.attention_processor.FusedAuraFlowAttnProcessor2_0, diffusers.models.attention_processor.FluxAttnProcessor2_0, diffusers.models.attention_processor.FluxAttnProcessor2_0_NPU, diffusers.models.attention_processor.FusedFluxAttnProcessor2_0, diffusers.models.attention_processor.FusedFluxAttnProcessor2_0_NPU, diffusers.models.attention_processor.CogVideoXAttnProcessor2_0, diffusers.models.attention_processor.FusedCogVideoXAttnProcessor2_0, diffusers.models.attention_processor.XFormersAttnAddedKVProcessor, diffusers.models.attention_processor.XFormersAttnProcessor, diffusers.models.attention_processor.XLAFlashAttnProcessor2_0, diffusers.models.attention_processor.AttnProcessorNPU, diffusers.models.attention_processor.AttnProcessor2_0, diffusers.models.attention_processor.MochiVaeAttnProcessor2_0, diffusers.models.attention_processor.MochiAttnProcessor2_0, diffusers.models.attention_processor.StableAudioAttnProcessor2_0, diffusers.models.attention_processor.HunyuanAttnProcessor2_0, diffusers.models.attention_processor.FusedHunyuanAttnProcessor2_0, diffusers.models.attention_processor.PAGHunyuanAttnProcessor2_0, diffusers.models.attention_processor.PAGCFGHunyuanAttnProcessor2_0, diffusers.models.attention_processor.LuminaAttnProcessor2_0, diffusers.models.attention_processor.FusedAttnProcessor2_0, diffusers.models.attention_processor.CustomDiffusionXFormersAttnProcessor, diffusers.models.attention_processor.CustomDiffusionAttnProcessor2_0, diffusers.models.attention_processor.SlicedAttnProcessor, diffusers.models.attention_processor.SlicedAttnAddedKVProcessor, diffusers.models.attention_processor.SanaLinearAttnProcessor2_0, diffusers.models.attention_processor.PAGCFGSanaLinearAttnProcessor2_0, diffusers.models.attention_processor.PAGIdentitySanaLinearAttnProcessor2_0, diffusers.models.attention_processor.SanaMultiscaleLinearAttention, diffusers.models.attention_processor.SanaMultiscaleAttnProcessor2_0, diffusers.models.attention_processor.SanaMultiscaleAttentionProjection, diffusers.models.attention_processor.IPAdapterAttnProcessor, diffusers.models.attention_processor.IPAdapterAttnProcessor2_0, diffusers.models.attention_processor.IPAdapterXFormersAttnProcessor, diffusers.models.attention_processor.SD3IPAdapterJointAttnProcessor2_0, diffusers.models.attention_processor.PAGIdentitySelfAttnProcessor2_0, diffusers.models.attention_processor.PAGCFGIdentitySelfAttnProcessor2_0, diffusers.models.attention_processor.LoRAAttnProcessor, diffusers.models.attention_processor.LoRAAttnProcessor2_0, diffusers.models.attention_processor.LoRAXFormersAttnProcessor, diffusers.models.attention_processor.LoRAAttnAddedKVProcessor, typing.Dict[str, typing.Union[diffusers.models.attention_processor.AttnProcessor, diffusers.models.attention_processor.CustomDiffusionAttnProcessor, diffusers.models.attention_processor.AttnAddedKVProcessor, diffusers.models.attention_processor.AttnAddedKVProcessor2_0, diffusers.models.attention_processor.JointAttnProcessor2_0, diffusers.models.attention_processor.PAGJointAttnProcessor2_0, diffusers.models.attention_processor.PAGCFGJointAttnProcessor2_0, diffusers.models.attention_processor.FusedJointAttnProcessor2_0, diffusers.models.attention_processor.AllegroAttnProcessor2_0, diffusers.models.attention_processor.AuraFlowAttnProcessor2_0, diffusers.models.attention_processor.FusedAuraFlowAttnProcessor2_0, diffusers.models.attention_processor.FluxAttnProcessor2_0, diffusers.models.attention_processor.FluxAttnProcessor2_0_NPU, diffusers.models.attention_processor.FusedFluxAttnProcessor2_0, diffusers.models.attention_processor.FusedFluxAttnProcessor2_0_NPU, diffusers.models.attention_processor.CogVideoXAttnProcessor2_0, diffusers.models.attention_processor.FusedCogVideoXAttnProcessor2_0, diffusers.models.attention_processor.XFormersAttnAddedKVProcessor, diffusers.models.attention_processor.XFormersAttnProcessor, diffusers.models.attention_processor.XLAFlashAttnProcessor2_0, diffusers.models.attention_processor.AttnProcessorNPU, diffusers.models.attention_processor.AttnProcessor2_0, diffusers.models.attention_processor.MochiVaeAttnProcessor2_0, diffusers.models.attention_processor.MochiAttnProcessor2_0, diffusers.models.attention_processor.StableAudioAttnProcessor2_0, diffusers.models.attention_processor.HunyuanAttnProcessor2_0, diffusers.models.attention_processor.FusedHunyuanAttnProcessor2_0, diffusers.models.attention_processor.PAGHunyuanAttnProcessor2_0, diffusers.models.attention_processor.PAGCFGHunyuanAttnProcessor2_0, diffusers.models.attention_processor.LuminaAttnProcessor2_0, diffusers.models.attention_processor.FusedAttnProcessor2_0, diffusers.models.attention_processor.CustomDiffusionXFormersAttnProcessor, diffusers.models.attention_processor.CustomDiffusionAttnProcessor2_0, diffusers.models.attention_processor.SlicedAttnProcessor, diffusers.models.attention_processor.SlicedAttnAddedKVProcessor, diffusers.models.attention_processor.SanaLinearAttnProcessor2_0, diffusers.models.attention_processor.PAGCFGSanaLinearAttnProcessor2_0, diffusers.models.attention_processor.PAGIdentitySanaLinearAttnProcessor2_0, diffusers.models.attention_processor.SanaMultiscaleLinearAttention, diffusers.models.attention_processor.SanaMultiscaleAttnProcessor2_0, diffusers.models.attention_processor.SanaMultiscaleAttentionProjection, diffusers.models.attention_processor.IPAdapterAttnProcessor, diffusers.models.attention_processor.IPAdapterAttnProcessor2_0, diffusers.models.attention_processor.IPAdapterXFormersAttnProcessor, diffusers.models.attention_processor.SD3IPAdapterJointAttnProcessor2_0, diffusers.models.attention_processor.PAGIdentitySelfAttnProcessor2_0, diffusers.models.attention_processor.PAGCFGIdentitySelfAttnProcessor2_0, diffusers.models.attention_processor.LoRAAttnProcessor, diffusers.models.attention_processor.LoRAAttnProcessor2_0, diffusers.models.attention_processor.LoRAXFormersAttnProcessor, diffusers.models.attention_processor.LoRAAttnAddedKVProcessor]]] )
设置用于计算注意力的注意力处理器。
禁用自定义注意力处理器并设置默认注意力实现。