支持的模型

Brevitas 是一个 AMD 库，用于神经网络量化。 🤗 Optimum-AMD 与 Brevitas 集成，以便更容易通过 Brevitas 量化 Transformers 模型。

此集成还允许将通过 Brevitas 量化的模型导出到 ONNX。

要回顾量化，请查看此文档。

有关所有可用选项，请参阅 ~BrevitasQuantizer 和 ~BrevitasQuantizationConfig。

目前，仅测试和支持以下架构

Llama
OPT

动态量化

from optimum.amd import BrevitasQuantizationConfig, BrevitasQuantizer
from transformers import AutoTokenizer

# Prepare the quantizer, specifying its configuration and loading the model.
qconfig = BrevitasQuantizationConfig(
    is_static=False,
    apply_gptq=False,
    apply_weight_equalization=False,
    activations_equalization=False,
    weights_symmetric=True,
    activations_symmetric=False,
)

quantizer = BrevitasQuantizer.from_pretrained("facebook/opt-125m")

model = quantizer.quantize(qconfig)

静态量化

from optimum.amd import BrevitasQuantizationConfig, BrevitasQuantizer
from transformers import AutoTokenizer

# Prepare the quantizer, specifying its configuration and loading the model.
qconfig = BrevitasQuantizationConfig(
    is_static=True,
    apply_gptq=False,
    apply_weight_equalization=True,
    activations_equalization=False,
    weights_symmetric=True,
    activations_symmetric=False,
)

quantizer = BrevitasQuantizer.from_pretrained("facebook/opt-125m")

tokenizer = AutoTokenizer.from_pretrained("facebook/opt-125m")

# Load the data for calibration and evaluation.
calibration_dataset = get_dataset_for_model(
    "facebook/opt-125m",
    qconfig=qconfig,
    dataset_name="wikitext2",
    tokenizer=tokenizer,
    nsamples=128,
    seqlen=512,
    split="train",
)

model = quantizer.quantize(qconfig, calibration_dataset)

将 Brevitas 模型导出到 ONNX

可以使用 Optimum 将 Brevitas 模型导出到 ONNX

import torch
from optimum.amd.brevitas.export import onnx_export_from_quantized_model

# Export to ONNX through optimum.exporters.
onnx_export_from_quantized_model(model, "llm_quantized_onnx")

完整示例

完整示例可在 https://github.com/huggingface/optimum-amd/tree/main/examples/quantization/brevitas 获取。

< > 在 GitHub 上更新