开源模型终于能看懂自己写的代码长什么样
闭源模型早就做到「看着截图写出网页」,开源一直卡在数据上:一段代码要正确、要写得好、渲染出来还得好看,可现有信号只能检查其中一项。这篇把「质量」拆成一张可勾选的评分表,既拿它筛数据、又拿它当奖励信号训练模型,造出目前最大最全的多模态代码数据集,9B 和 35B 两个开源模型在视觉编程任务上大幅领先同类、逼近闭源。它不是你明天能用上的,但这是开源社区第一次在「让 AI 边写边看」这件事上真正追上闭源的门槛。
📄 原文摘要(英文)
Multimodal code intelligence connects the symbolic world of programs with the visual world their execution creates. Interest in this capability is burgeoning, as models are increasingly asked to produce charts, webpages, animations, and semi-structured documents. Recently, proprietary models have pushed far ahead, producing interactive artifacts even from a text description or a screenshot alone. Yet their data and training remain closed, and open progress is held back by the quantity and quality of available data and by the pipelines and methods for producing it. Quality is the harder of the two, since a sample must be correct as a program, well built, and above all visually right once rendered. No existing signal covers all three. Execution often returns just a pass or fail, code similarity does not map to the visual output, and task-specific verifiers do not generalize. To push open progress further, we start from the data side with PrismaForge, a hierarchical data pipeline centered on a synthesized rubric that decomposes these factors into checkable criteria. The same rubric serves as both filter and reward: it curates the supervised corpus, then drives reinforcement learning on visual-coding tasks. Leveraging PrismaForge for data synthesis and filtering, we build PrismaCoder-2M, the most diverse open dataset for multimodal code intelligence and the largest to date. On the model side, we train PrismaCoder-9B and PrismaCoder-35B on this data and release both. They lead the open state of the art by a wide margin and rival leading proprietary systems. Our analyses further show that the proposed rubric-based pipeline improves multimodal coding performance across model backbones. We release our code and data at https://qiushisun.github.io/PrismaCoder-Home/.