AI听120分钟音频还能精确报时
现在的AI听音频只能理解内容,但不知道事情发生在第几秒。这篇让AI在长达2小时的录音里,不仅能听懂问题,还能精确回答『在第几分钟说了什么』。做法是在音频里每隔一段就插入一个时间标记,像给录音打上时间戳,然后用大量合成数据训练。它在短片段和长录音的测试上都表现不错,还能根据时间点描述或总结片段。这不是你明天就能用的功能,但它让AI从『听懂』进化到『知道什么时候发生的』,对会议记录、播客检索这类场景是真正的进步。
📄 原文摘要(英文)
Temporal grounding in long recordings remains challenging for audio-conditioned LLMs. We present a time-aware audio LLM that answers questions with explicit timestamps over up to 120 minutes of input. Our approach interleaves periodic time markers with continuous audio tokens using large-scale synthetic supervision from a cascaded pipeline. Our model achieves strong temporal-grounding accuracy on short and long benchmarks and supports time-anchored fragment descriptions and summaries. Extensive ablations examine how time representation, marker frequency, tokenization, and duration-mixture design affect accuracy and computational cost. We release model weights and datasets to support further research on time-aware audio understanding, available at https://huggingface.co/ai-sage/GigaChat3.1-Audio-10B-A1.8B.