AI Pulse
📄 论文解读

给AI纠错,不用重写,改一个词就行

给AI标注训练数据,过去最笨的办法是等它生成完,再整段手动改——慢,而且改完的句子已经不是AI自己的风格了。这篇工具换了个思路:你读到AI回答里第一个不对劲的词,直接在那个位置改掉,系统自动把后面的内容全部截断、从你改的地方重新生成,不满意就再改下一个词,循环到满意为止。小规模对照实验里,这个「改一个词就续写」的方式比整段手动改中位时间省了52%。更关键的是,最终答案里绝大多数词还是AI自己生成的,数据保留了它的原始分布,适合用来做对齐训练;而你改过的每个位置,天然就是一对「错词vs对词」的监督信号。它不是你明天能直接用的产品,但给做AI训练数据的人指了个省力方向:与其教AI重写,不如教它改。

📄 原文摘要(英文)

We present onPanda, an interactive tool for efficiently annotating LLM alignment data and agent trajectories. onPanda adopts token-level correction as its core interaction: while reading a model response, the annotator locates the first inappropriate token and either picks a substitute from the model's candidate tokens or types the correct text via free-form editing. The system then truncates everything after that position and continues generation from the corrected prefix, repeating this locate-correct-continue loop until a satisfactory response is obtained. This mechanism lets annotators precisely steer model outputs at low cost: a small controlled study suggests that onPanda reduces median annotation time by 52% over manual post-editing. Since the vast majority of tokens in the final response are generated by the model itself, the resulting data largely preserves the model's sampling distribution and is well suited for constructing on-policy SFT and preference data. Furthermore, the token-level corrections recorded during annotation provide fine-grained supervision with precise positions and naturally paired positive--negative samples. onPanda also connects to external tools and harnesses, enabling interactive trajectory annotation in realistic environments. In addition, we release Panda-CVL, a dataset annotated with onPanda, together with a benchmark for token-level correction.

arXiv 原文

订阅 AI Pulse

每天 08:00 · 12:30 · 18:30 · 23:50 更新