AI训练读视频，卡在解码上
AI训练要随机抽视频帧，但现有视频格式是为顺序播放优化的，抽一帧得先解一串依赖帧，慢且费算力。这篇论文把解码调度和视频参考结构一起改：只解你抽的那帧和它必需的参考帧，跳过无关重建；再配合一种可选的编码方式，把每帧的依赖限制在两个锚点内，最多解3帧就能拿到目标。在机器人数据集上，加载速度提升1.45倍，媒体数据量还少了58.7%。这不是你明天能用的工具，但它指出了AI训练基础设施里一个被忽视的瓶颈——数据读取可能比模型计算更拖后腿。
📄 原文摘要(英文)
Large-scale AI training increasingly relies on massive video datasets with random temporal sampling patterns. However, existing video storage paradigms are optimized for sequential playback and employ sophisticated inter-frame reference structures to maximize compression efficiency. When applied to the random-sampling workloads of AI training, these paradigms face substantial decoding-efficiency bottlenecks. Workarounds such as independent JPEG frames and short groups of pictures (GOPs) reduce decoding work at the cost of larger storage and transfer volumes. In this paper, we identify two sources of decode amplification: sequential reconstruction of unrelated frames and reconstruction of the reference chains required by sampled targets. We address both sources by co-designing the decoding schedule and video reference structure. Dependency-aware sparse decoding reconstructs only requested frames and their required references, skipping unrelated reconstruction without transcoding the source videos. Optional offline Dependency-Decoupled Random Access (DDRA) encoding further reduces reference reconstruction by restricting each interior frame to two bounding intra-coded anchors. Together, sparse decoding and DDRA retain inter-frame prediction while bounding reconstruction to at most three frames per requested target, independent of the intra period. Experiments on a large-scale robotics corpus show that, at comparable reconstruction quality, our method achieves 1.45times the loading throughput of LeRobot v3 with 58.7% less media payload.