直播带货的AI终于能同时看、听、读
你刷直播时，AI助手只能听懂主播在说什么，却看不见屏幕上闪过的商品图、弹幕里的提问、或者背景里展示的实物。这篇论文把直播里的语音、视频画面、商品图片、屏幕文字、用户提问全塞进同一个模型，让AI能同时处理这些信息，而不是分开理解再拼凑。关键做法是给每一帧画面配上同一时刻的音频，打上时间戳，让模型知道“这个画面和这句话是同时发生的”。然后分三步训练：先让模型学会识别商品、听清语音、看懂文字，再教它回答用户问题，最后用强化学习逼它给出更准确、更自然的回答，同时保证不卡顿。在电商直播的测试集上，它比现有模型强出一截，在通用任务上也没掉队。它不是你明天就能用的工具，但直播带货的AI助手、自动生成直播摘要、实时回答弹幕提问这类场景，会因为它从“半聋半瞎”变成“全感官在线”。
📄 原文摘要(英文)
E-commerce live streaming requires omni-modal understanding of noisy, temporally extended streams, where product facts are distributed across speech, video frames, product images, overlaid text, and user queries. We present TLive-Omni, an omni-modal understanding model tailored to live-commerce scenarios. It maps image, video, audio, and text inputs into a unified representation space. For long-form live streaming analysis, we introduce Per-vGrid, a timestamped token organization that groups each video grid with its temporally corresponding audio within explicit boundary tokens to facilitate temporal alignment. We design a three-stage supervised training recipe that progressively develops live-commerce understanding, from omni-modal perception to instruction-following responses. We then propose Faithful-RFT, a reinforcement fine-tuning stage that further improves answer faithfulness and expression quality while meeting real-time demands, scoring final responses directly with task-verifiable feedback rather than optimizing for reasoning-style exploration during rollout. Moreover, TLive-Omni is supported by a scenario-oriented atomic capability taxonomy and a compact data production engine that converts live-commerce audio, image, and video streams into training signals for speech recognition, speaker analysis, product visual grounding, text recognition, temporal grounding, video dense caption, and omni-modal QA, etc. For scalable training, a synchronized length-grouped sampler reduces padding while preserving comparable workloads across workers, while a lightweight dynamic sampling strategy regenerates rollout groups with near-zero reward variance to maintain meaningful relative advantages for GRPO. Experiments on e-commerce live streaming benchmarks demonstrate strong performance across live-commerce domain tasks, together with excellent generalization on general benchmarks.