AI Pulse
📄 论文解读

给企业文档做RAG,先花72分钟把PDF变成AI爱读的格式

企业想让AI回答内部知识库的问题,得先把PDF、Word、PPT这些文档切成AI能检索的小块。传统做法要么用规则和OCR硬拆,把表格和层级结构弄乱;要么让AI自己边读边切,又贵又慢还可能编造。这篇提出一个中间路线:先把任何文档统一转成PDF,再用一次多模态大模型把每页转成结构化的Markdown——表格改写成自包含的陈述句、保留标题层级——之后的分块就完全走确定性规则,不再让AI碰原文。在236份文档、795页的企业基准测试上,整个语料库72分钟切完、零错误,产出1748个检索块;相比让前沿大模型自己切,输出token减少95.7%,成本降77.8%到85.6%,时间省75%,而且能线性扩展到500页以上的文档。它不是你明天就能用上的现成工具,但给了一个信号:企业RAG的瓶颈不在模型多聪明,而在文档进AI之前那一步——把格式先收拾干净,比让AI硬啃原始文件划算得多。

📄 原文摘要(英文)

Retrieval-Augmented Generation (RAG) systems over enterprise knowledge bases must ingest heterogeneous document formats -- PDFs, Word documents, presentations, and scans -- whose content is locked inside complex visual layouts, multi-column pages, and dense tables. Rule-based extraction and OCR destroy reading order, flatten tables, and lose heading hierarchy, while fully agentic chunking over extracted text incurs high token costs and hallucination risk. We present Document Retrieval-Aware Chunking (D-RAC), an extension of our Web Retrieval-Aware Chunking (W-RAC) framework to arbitrary document formats. D-RAC first normalizes any input document into PDF, exploiting the fact that virtually every format has a faithful, deterministic PDF rendering. A single multimodal LLM pass then converts rendered pages into retrieval-optimized Markdown -- rewriting tables as self-contained prose statements and preserving heading hierarchy -- after which chunking proceeds exactly as in W-RAC: deterministic parsing into ID-addressable units followed by lightweight LLM-based chunk planning over identifiers rather than text. Source text is never regenerated during chunking, preserving W-RAC's cost, determinism, and observability benefits while unlocking every renderable format as a first-class input. On the 236-document, 795-page PDF subset of the RAG-Multi-Corpus benchmark spanning five enterprise domains, D-RAC converts and chunks the entire corpus in 72 minutes with zero errors, producing 1,748 retrieval-ready chunks. Compared to agentic chunking with frontier LLMs, D-RAC reduces chunking-stage output tokens by 95.7%, cutting chunking cost by 77.8% (GPT-4.1 pricing) to 85.6% (Gemini 2.5 Pro pricing) and chunking time by 75%. D-RAC scales linearly to documents of 500+ pages.

arXiv 原文

订阅 AI Pulse

每天 08:00 · 12:30 · 18:30 · 23:50 更新