让AI写代码时先看一眼效果图
写代码的AI不懂审美,画图的AI不懂内容。这篇把两者接起来:让代码模型在生成幻灯片、海报这类设计代码时,先拿同一段文字生成一张参考图,再照着这张图调整布局和配色。它把整个画面拆成一张设计图,每个属性都能追责,再用两套奖励分别打分:能解析的靠规则验证,不能解析的交给视觉大模型按图打分。结果,8B和32B的开源模型视觉质量分别提升31.1和22.4个百分点,追平了GPT-5.5和Claude Opus 4.8。这不是你明天能用的工具,但它指了个方向:让AI设计时,先看见,再动手。
📄 原文摘要(英文)
Generating information-rich visual artifacts such as slides, posters, and dashboards as code is appealing because the outputs are structured, editable, and verifiable. Yet code models cannot see how their choices of layout and color come together on the canvas, often limiting spatial planning and aesthetic quality. Image generators excel at visual design but can misrender requested content, making their outputs unreliable targets for direct reproduction. We therefore introduce AesCode, which generates code grounded in the content prompt while using an image generated from the same prompt as an aesthetic guidance. To do so, AesCode represents the target as a design graph that describes the whole canvas and makes each property attributable. From this graph, we derive complementary cross-modal rewards: deterministic verifiers score properties parsable from code and rendering, while a VLM judge assesses the non-parsable ones with a sample-specific Visual Graph Rubric (VGR) tied to graph elements and relations. We optimize these decoupled rewards through reinforcement learning with per-channel normalization to jointly improve spatial planning, aesthetic quality, and verifiable correctness. To evaluate this setting, we build a benchmark that scores verifiable correctness and visual quality separately, with visual scores highly consistent with human judgments. We find that reference images substantially improve the visual quality of code generated by existing models, and ablations show that AesCode benefits from references both as model input and as a source of visual rewards. Relative to same-scale Qwen3-VL baselines given the same references, AesCode-8B and AesCode-32B improve visual quality by 31.1 and 22.4 percentage points, reaching scores comparable to those of closed-source frontier models such as GPT-5.5 and Claude Opus 4.8.