数据智能体评测：最强模型准确率仅66%
数据智能体——能听懂人话、在数据库、表格、文档、视频里翻找答案并给出表格的AI——有了第一个贴近真实工作场景的硬核考场。这个叫DataSpace的评测集塞进了410个任务、7439份文件，横跨六种格式，连视频都有。结果：目前最强的多模态模型，准确率也只有66.34%；而且只要换一个AI的“外壳”（调用框架），成绩能差出15个百分点。更反直觉的是，让AI同时看文字和视频、做跨格式关联，反而会让所有模型都变笨。这说明数据智能体离“可靠”还远，也点出了它最该补的短板。
📄 原文摘要(英文)
Data agents enable natural-language analytics over organizational workspaces, where relevant evidence may be scattered across databases, structured files, long documents, and multimedia. Existing benchmarks largely isolate structured querying, retrieval, or open-ended analysis, leaving heterogeneous evidence discovery, complete tabular outputs, and deterministic evaluation insufficiently unified. We introduce DataSpace, a benchmark in which data agents produce verifiable tabular results from task-local heterogeneous workspaces. It contains 410 cross-language tasks and 7,439 artifacts totaling 15.01 GB across CSV, JSON, SQLite, Markdown, PDF, and video. DataSpace also served as the official evaluation benchmark for the KDD Cup 2026 Data Agents for Complex Data Analysis competition. Each agent receives only a question and workspace and returns the complete requested tabular result. We construct DataSpace with DataSpace-Builder, an execution-grounded framework comprising cross-language transformation, constraint-aware relational sampling, modality routing and artifact rendering, and human review and task repair by 11 domain experts. A deterministic evaluator performs header-invariant column alignment, type- and precision-aware normalization, and order-aware row comparison. Across six recently released frontier multimodal models and five widely used agent harnesses, the best accuracy reaches 66.34%, while harness choice creates a 15.36-point spread with the backbone fixed. Multimodal evidence integration and joins consistently reduce accuracy across all six backbones. These results show that DataSpace remains unsaturated and identify key challenges for improving data-agent reliability.