AI Pulse
📄 论文解读

AI听120分钟音频还能精确报时

现在的AI听音频只能理解内容,但不知道事情发生在第几秒。这篇让AI在长达2小时的录音里,不仅能听懂问题,还能精确回答『在第几分钟说了什么』。做法是在音频里每隔一段就插入一个时间标记,像给录音打上时间戳,然后用大量合成数据训练。它在短片段和长录音的测试上都表现不错,还能根据时间点描述或总结片段。这不是你明天就能用的功能,但它让AI从『听懂』进化到『知道什么时候发生的』,对会议记录、播客检索这类场景是真正的进步。

📄 原文摘要(英文)

Temporal grounding in long recordings remains challenging for audio-conditioned LLMs. We present a time-aware audio LLM that answers questions with explicit timestamps over up to 120 minutes of input. Our approach interleaves periodic time markers with continuous audio tokens using large-scale synthetic supervision from a cascaded pipeline. Our model achieves strong temporal-grounding accuracy on short and long benchmarks and supports time-anchored fragment descriptions and summaries. Extensive ablations examine how time representation, marker frequency, tokenization, and duration-mixture design affect accuracy and computational cost. We release model weights and datasets to support further research on time-aware audio understanding, available at https://huggingface.co/ai-sage/GigaChat3.1-Audio-10B-A1.8B.

arXiv 原文

订阅 AI Pulse

每天 08:00 · 12:30 · 18:30 · 23:50 更新