修好AI界面代码的一个瑕疵,却弄坏另一处?
让AI照着网页截图生成代码,它改一个局部往往带崩另一块:动了布局、连累样式。研究者发现病根是「视觉修复耦合」——一个改动沿依赖关系传染,修好一个错、整出两个新错。他们给AI配了「评分准则」当作修复指南:每轮只挑一个最要紧的错去修,并且把前面用过的准则记下来,防止重复改、乱改。在6个大模型和3套基准里,这套方法让修正过程更稳,也更少出现改到一半彻底崩掉的情况。它不是你明天能直接装上的功能,但解释了为什么AI改界面总是按下葫芦浮起瓢,也给这个方向指了条路。
📄 原文摘要(英文)
Large vision-language models have shown strong progress in UI-to-code generation, yet their test-time self-evolution remains unstable. We first identify a fundamental obstacle, termed visual repair coupling: a local code edit may propagate through layout, style, and component dependencies, correcting one visual mismatch while degrading regions that were previously faithful. To address this issue, we present RubSE, a Rubric-guided Self-Evolution framework that uses rubrics to represent visual feedback as a structured visual-repair context. At each refinement round, RubSE generates typed candidate rubrics, selects one prioritized repair target, and stores previously selected rubrics as history, thereby steering each revision toward a well-scoped visual repair while discouraging repeated or over-broad changes. Evaluations across six VLMs and three UI-to-code benchmarks demonstrate that RubSE substantially outperforms naïve self-evolution in final-round and best-round settings, achieving more stable refinement trajectories and a higher trajectory-level performance ceiling. Further analysis shows that RubSE mitigates trajectory collapse by improving recovery from severe visual regressions, and that stronger rubric generators can transfer effective visual-repair guidance to weaker code improvers.