给机器人说希腊语，先别信测试分数
给机器人加一门新语言，最难的不是翻译，而是怎么知道它真的听懂了。研究者给一个开源机器人模型加希腊语，只把指令机器翻译过去，不动架构。结果发现：常用的评测方法会骗人——按颜色直方图打分，模型瞎猜反而得分高；换个单目标测试，故意给错误指令也能拿 82.6%，和正确希腊语的 84.6% 几乎没差；训练损失也预测不了实际表现。真正靠谱的是 90 个任务的区分性测试、每个配置跑 3 次取平均，这时只学希腊语的模型和给错误指令的底线差不多，而希腊语+英语双语训练稳定领先 6.7-7.1 分，达到英语水平的约四成。还有个坑：模型会死记翻译器的措辞，同一个任务换 7 种说法训练，能把这种过拟合的损失减半。结论很实在：给低资源语言做机器人本地化，先造一个「故意给错指令」的对照组来验证评测工具，再跨随机种子重复实验，否则你看到的分数可能是噪音。
📄 原文摘要(英文)
Robot foundation models are trained and evaluated predominantly in English, and robot demonstration corpora do not exist for most languages. We study the addition of Greek to an open vision-language-action stack using only machine-rephrased instructions and no architecture changes. The main challenge is measurement rather than translation. Several plausible instruments produce false conclusions: a color-histogram metric rewards noise, a single-goal benchmark scores 84.6% under correct Greek and 82.6% under deliberately wrong instructions, training loss fails to predict Greek success, and single-run comparisons are dominated by seed variation. On a discriminative ninety-task suite with three seeds per arm, a multilingual text tower without Greek demonstrations remains at its wrong-instruction floor, while Greek-only training exceeds its control by at most 2.7 points. Bilingual training yields a consistent 6.7-7.1 point margin over its control and reaches about two fifths of English performance. The policy also overfits the translator's phrasing; training on seven phrasings per task approximately halves this penalty. Warm-starting from a language-adapted world model and unfreezing the text tower both degrade performance. The results support two practical requirements for low-resource robot-policy localization: build a guaranteed null before trusting a metric, and replicate low-resource-language results across seeds.