AI Pulse
📄 论文解读

给企业文档做RAG,先花72分钟把一切变成PDF

企业知识库里的RAG系统,最大的坑不是AI不够聪明,而是文档本身:PDF、Word、PPT、扫描件,内容锁在复杂版式、多栏页面和密集表格里。传统抽取和OCR会毁掉阅读顺序、拍扁表格、丢掉标题层级;让AI直接读原文再切块,又贵又容易幻觉。这篇提出D-RAC:先把任何文档统一转成PDF——因为几乎所有格式都有忠实的PDF渲染——然后用一次多模态大模型扫描,把页面转成检索优化的Markdown,表格改写成自包含的陈述句,保留标题层级,之后切块就完全走已有的W-RAC流程:确定性解析成可寻址单元,再用轻量LLM基于标识符做切块规划,全程不重新生成原文。在236份文档、795页的企业基准上,整个语料72分钟转换+切块完成,零错误,产出1748个检索块;相比用前沿大模型做agentic切块,输出token减少95.7%,成本降77.8%到85.6%,时间省75%,且能线性扩展到500+页文档。它不是你明天就能用上的产品,但给了一个信号:企业RAG的瓶颈正在从「AI能力」转向「文档工程」,而把一切先归一化成PDF,可能是最笨也最稳的解法。

📄 原文摘要(英文)

Retrieval-Augmented Generation (RAG) systems over enterprise knowledge bases must ingest heterogeneous document formats -- PDFs, Word documents, presentations, and scans -- whose content is locked inside complex visual layouts, multi-column pages, and dense tables. Rule-based extraction and OCR destroy reading order, flatten tables, and lose heading hierarchy, while fully agentic chunking over extracted text incurs high token costs and hallucination risk. We present Document Retrieval-Aware Chunking (D-RAC), an extension of our Web Retrieval-Aware Chunking (W-RAC) framework to arbitrary document formats. D-RAC first normalizes any input document into PDF, exploiting the fact that virtually every format has a faithful, deterministic PDF rendering. A single multimodal LLM pass then converts rendered pages into retrieval-optimized Markdown -- rewriting tables as self-contained prose statements and preserving heading hierarchy -- after which chunking proceeds exactly as in W-RAC: deterministic parsing into ID-addressable units followed by lightweight LLM-based chunk planning over identifiers rather than text. Source text is never regenerated during chunking, preserving W-RAC's cost, determinism, and observability benefits while unlocking every renderable format as a first-class input. On the 236-document, 795-page PDF subset of the RAG-Multi-Corpus benchmark spanning five enterprise domains, D-RAC converts and chunks the entire corpus in 72 minutes with zero errors, producing 1,748 retrieval-ready chunks. Compared to agentic chunking with frontier LLMs, D-RAC reduces chunking-stage output tokens by 95.7%, cutting chunking cost by 77.8% (GPT-4.1 pricing) to 85.6% (Gemini 2.5 Pro pricing) and chunking time by 75%. D-RAC scales linearly to documents of 500+ pages.

arXiv 原文

订阅 AI Pulse

每天 08:00 · 12:30 · 18:30 · 23:50 更新