Optimum 文档
优化
并获得增强的文档体验
开始使用
优化
转换
class optimum.fx.optimization.Transformation
< source >( )
一个 torch.fx 图转换。
它必须实现 transform() 方法，并用作可调用对象。
__call__
< source >( graph_module: GraphModule lint_and_recompile: bool = True ) → torch.fx.GraphModule
get_transformed_nodes
< source >( graph_module: GraphModule ) → List[torch.fx.Node]
将一个节点标记为已通过此转换进行转换。
transform
< source >( graph_module: GraphModule ) → torch.fx.GraphModule
transformed
< source >( node: Node ) → bool
可逆转换
class optimum.fx.optimization.ReversibleTransformation
< source >( )
一个可逆的 torch.fx 图转换。
它必须实现 transform() 和 reverse() 方法，并用作可调用对象。
__call__
< source >( graph_module: GraphModule lint_and_recompile: bool = True reverse: bool = False ) → torch.fx.GraphModule
将节点标记为已恢复到原始状态。
reverse
< source >( graph_module: GraphModule ) → torch.fx.GraphModule
optimum.fx.optimization.compose
< source >( *args: Transformation inplace: bool = True )
参数
- args (Transformation) — 要组合的转换。
- inplace (
bool, 默认为True) — 结果转换是否应该就地执行，或者创建一个新的图模块。
组合一系列转换。
示例
>>> from transformers import BertModel
>>> from transformers.utils.fx import symbolic_trace
>>> from optimum.fx.optimization import ChangeTrueDivToMulByInverse, MergeLinears, compose
>>> model = BertModel.from_pretrained("bert-base-uncased")
>>> traced = symbolic_trace(
... model,
... input_names=["input_ids", "attention_mask", "token_type_ids"],
... )
>>> composition = compose(ChangeTrueDivToMulByInverse(), MergeLinears())
>>> transformed_model = composition(traced)转换
class optimum.fx.optimization.MergeLinears
< source >( )
将相同输入的线性层合并为一个大线性层的转换。
示例
>>> from transformers import BertModel
>>> from transformers.utils.fx import symbolic_trace
>>> from optimum.fx.optimization import MergeLinears
>>> model = BertModel.from_pretrained("bert-base-uncased")
>>> traced = symbolic_trace(
... model,
... input_names=["input_ids", "attention_mask", "token_type_ids"],
... )
>>> transformation = MergeLinears()
>>> transformed_model = transformation(traced)
>>> restored_model = transformation(transformed_model, reverse=True)class optimum.fx.optimization.FuseBiasInLinear
< source >( )
将偏差融合到 torch.nn.Linear 中的权重的转换。
示例
>>> from transformers import BertModel
>>> from transformers.utils.fx import symbolic_trace
>>> from optimum.fx.optimization import FuseBiasInLinear
>>> model = BertModel.from_pretrained("bert-base-uncased")
>>> traced = symbolic_trace(
... model,
... input_names=["input_ids", "attention_mask", "token_type_ids"],
... )
>>> transformation = FuseBiasInLinear()
>>> transformed_model = transformation(traced)
>>> restored_model = transformation(transformed_model, reverse=True)class optimum.fx.optimization.ChangeTrueDivToMulByInverse
< source >( )
当分母为静态时，将 true除法节点更改为乘以逆节点的转换。例如，在注意力层中，缩放因子有时就是这种情况。
示例
>>> from transformers import BertModel
>>> from transformers.utils.fx import symbolic_trace
>>> from optimum.fx.optimization import ChangeTrueDivToMulByInverse
>>> model = BertModel.from_pretrained("bert-base-uncased")
>>> traced = symbolic_trace(
... model,
... input_names=["input_ids", "attention_mask", "token_type_ids"],
... )
>>> transformation = ChangeTrueDivToMulByInverse()
>>> transformed_model = transformation(traced)
>>> restored_model = transformation(transformed_model, reverse=True)class optimum.fx.optimization.FuseBatchNorm2dInConv2d
< source >( )
将 `nn.Conv2d` 后面的 `nn.BatchNorm2d` 融合到一个 `nn.Conv2d` 中的转换。只有当卷积的唯一后续节点是批归一化时，才会进行融合。
例如，在以下情况下将不会进行融合：
示例
>>> from transformers.utils.fx import symbolic_trace
>>> from transformers import AutoModelForImageClassification
>>> from optimum.fx.optimization import FuseBatchNorm2dInConv2d
>>> model = AutoModelForImageClassification.from_pretrained("microsoft/resnet-50")
>>> model.eval()
>>> traced_model = symbolic_trace(
... model,
... input_names=["pixel_values"],
... disable_check=True
... )
>>> transformation = FuseBatchNorm2dInConv2d()
>>> transformed_model = transformation(traced_model)class optimum.fx.optimization.FuseBatchNorm1dInLinear
< source >( )
将 `nn.Linear` 之后或之前的 `nn.BatchNorm1d` 融合到一个 `nn.Linear` 中的转换。只有当线性层的唯一后续节点是批归一化，或者批归一化的唯一后续节点是线性层时，才会进行融合。
例如，在以下情况下将不会进行融合：
示例
>>> from transformers.utils.fx import symbolic_trace
>>> from transformers import AutoModel
>>> from optimum.fx.optimization import FuseBatchNorm1dInLinear
>>> model = AutoModel.from_pretrained("nvidia/groupvit-gcc-yfcc")
>>> model.eval()
>>> traced_model = symbolic_trace(
... model,
... input_names=["input_ids", "attention_mask", "pixel_values"],
... disable_check=True
... )
>>> transformation = FuseBatchNorm1dInLinear()
>>> transformed_model = transformation(traced_model)