推理工具包API

支持的任务

SageMaker Hugging Face 推理工具包在 inputs 键中接受输入，并在 parameters 键中支持额外的 pipelines 参数。您可以将 pipelines 支持的任何 kwargs 作为 parameters 提供。

推理工具包 API 支持的任务包括

文本分类
情感分析
词元分类
特征提取
掩码填充
摘要
翻译_xx_到_yy
文本到文本生成
文本生成
音频分类
自动语音识别
对话
图像分类
图像分割
目标检测
表格问答
零样本分类
零样本图像分类

参阅以下任务的一些请求示例

文本分类

{
  "inputs": "This sound track was beautiful! It paints the senery in your mind so well I would recommend it
  even to people who hate vid. game music!"
}

情感分析

{
  "inputs": "Don't waste your time.  We had two different people come to our house to give us estimates for
a deck (one of them the OWNER).  Both times, we never heard from them.  Not a call, not the estimate, nothing."
}

词元分类

{
  "inputs": "My name is Sylvain and I work at Hugging Face in Brooklyn."
}

问题回答

{
  "inputs": {
    "question": "What is used for inference?",
    "context": "My Name is Philipp and I live in Nuremberg. This model is used with sagemaker for inference."
  }
}

零样本分类

{
  "inputs": "Hi, I recently bought a device from your company but it is not working as advertised and I would like to get reimbursed!",
  "parameters": {
    "candidate_labels": ["refund", "legal", "faq"]
  }
}

表格问答

{
  "inputs": {
    "query": "How many stars does the transformers repository have?",
    "table": {
      "Repository": ["Transformers", "Datasets", "Tokenizers"],
      "Stars": ["36542", "4512", "3934"],
      "Contributors": ["651", "77", "34"],
      "Programming language": ["Python", "Python", "Rust, Python and NodeJS"]
    }
  }
}

参数化请求

{
  "inputs": "Hugging Face, the winner of VentureBeat’s Innovation in Natural Language Process/Understanding Award for 2021, is looking to level the playing field. The team, launched by Clément Delangue and Julien Chaumond in 2016, was recognized for its work in democratizing NLP, the global market value for which is expected to hit $35.1 billion by 2026. This week, Google’s former head of Ethical AI Margaret Mitchell joined the team.",
  "parameters": {
    "repetition_penalty": 4.0,
    "length_penalty": 1.5
  }
}

环境变量

推理工具包实现了各种额外的环境变量以简化部署。Hugging Face 特定的环境变量完整列表如下所示

HF_TASK

HF_TASK 定义了所使用的 🤗 Transformers 流水线的任务。有关任务的完整列表，请参阅此处。

HF_TASK="question-answering"

HF_MODEL_ID

HF_MODEL_ID 定义了模型 ID，在创建 SageMaker 端点时，该 ID 会自动从 hf.co/models 加载。🤗 Hub 的 10,000 多个模型均可通过此环境变量访问。

HF_MODEL_ID="distilbert-base-uncased-finetuned-sst-2-english"

HF_MODEL_REVISION

HF_MODEL_REVISION 是 HF_MODEL_ID 的扩展，允许您定义或固定模型修订版，以确保您的 SageMaker 端点始终加载相同的模型。

HF_MODEL_REVISION="03b4d196c19d0a73c7e0322684e97db1ec397613"

HF_API_TOKEN

HF_API_TOKEN 定义了您的 Hugging Face 授权令牌。HF_API_TOKEN 用作远程文件（如私有模型）的 HTTP 持有者授权。您可以在 Hugging Face 账户的设置中找到您的令牌。

HF_API_TOKEN="api_XXXXXXXXXXXXXXXXXXXXXXXXXXXXX"

< > 在 GitHub 上更新

在 AWS 上部署

推理工具包API

支持的任务

环境变量