一个模型读懂所有格式：文档、图片、音频、视频全变结构化数据
现在的 AI 读文档、看图、听音频、看视频，用的是四套不同的系统，互相不认账。这篇把四样东西塞进同一个编码器，用同一套坐标(空间+时间)来组织，视频帧还能实时参考同时间的音频——跨模态融合发生在感知层，而不是事后拼接。输出也统一成一套 schema，十个解析任务共用同一个训练目标，模型自己生成数据、自己给自己当老师，一轮轮迭代。最终一个 5B 参数的模型，在文档解析、图表、化学结构式、乐谱上都达到或接近专用模型水平，音频和视频优势最明显。代码和权重都开源。它不是你明天就能用的产品，但这是「一个模型通吃所有模态」这条路上少见的真进展。
📄 原文摘要(英文)
We present Youtu-Parsing-Omni, an omni-modal parser that converts documents, images, audio, and video into structured records grounded in space and time, providing a perception layer for agents that read, locate, and cite evidence across modalities. Its design rests on one encoder for every modality and one schema for every task. Youtu-Omni-Encoder, initialized from a pre-trained language model to strengthen text reading, encodes all modalities with shared weights in a unified space-time coordinate system, and a few fusion layers let video frames attend to concurrent audio, bringing cross-modal fusion and temporal alignment into perception itself. OmniSchema unifies the outputs of all ten parsing tasks, allowing a single self-evolving post-training procedure, OmniSchema-Aware On-Policy Distillation (OSAD), to provide a shared token-role-based objective across tasks. In each round, the model is trained on its own rollouts against token-level targets from a reference-conditioned copy of itself, which is then refreshed with the trained weights, so that the parser and its supervision co-evolve. With a single 5 B-parameter checkpoint, Youtu-Parsing-Omni achieves state-of-the-art performance on OmniDocBench v1.6 (96.96 Overall) and leads all open-weight models on OmniParsingBench (75.08 Avg.), with its advantage concentrated on audio and video. It also achieves the best overall chart-parsing score (95.02) and the highest exact chemical-fingerprint agreement among compared systems, and matches a dedicated optical-music model in character-level transcription of typeset sheet music while staying far ahead of general-purpose systems. Code and model weights are publicly available.