多模态
我们目前支持 IDEFICS(托管在 TGI 上)、OpenAI 和 Anthropic Claude 3 作为多模态模型。您可以通过在 MODELS
配置中设置 multimodal: true
来启用它。对于 IDEFICS,您必须拥有 PRO HF API 令牌。对于 OpenAI,请参见 OpenAI 部分。对于 Anthropic,请参见 Anthropic 部分。
MODELS=`[
{
"name": "HuggingFaceM4/idefics-80b-instruct",
"multimodal" : true,
"description": "IDEFICS is the new multimodal model by Hugging Face.",
"preprompt": "",
"chatPromptTemplate" : "{{#each messages}}{{#ifUser}}User: {{content}}{{/ifUser}}<end_of_utterance>\nAssistant: {{#ifAssistant}}{{content}}\n{{/ifAssistant}}{{/each}}",
"parameters": {
"temperature": 0.1,
"top_p": 0.95,
"repetition_penalty": 1.2,
"top_k": 12,
"truncate": 1000,
"max_new_tokens": 1024,
"stop": ["<end_of_utterance>", "User:", "\nUser:"]
}
}
]`