AI Pulse
📄 论文解读

AI 想问题时,终于学会把废话藏进脑子里

大模型现在回答前会先“想”一大段,但这段思考全写在明面上,既费算力又费时间。这篇把思考过程当成一条轨迹,用几何方向判断每一步的价值:和最终答案方向一致的步骤保留成文字,跑偏的、纠错的、试探的步骤压缩成“脑子里”的连续向量,不再占用输出。结果是在 Qwen 两个模型上,准确率最高提升 2.6%,回答长度砍掉一半,单位算力的准确率提升 2.29 倍。它不是你明天能用上的东西,但它指向一个更本质的方向:AI 的思考不必全部摊开给你看,该省的地方省,该想的地方想。

📄 原文摘要(英文)

Chain-of-Thought (CoT) improves the reasoning ability of Large Language Models (LLMs) but incurs substantial computation and context costs. Existing methods either lose intermediate information through hard pruning or lack a principled criterion for continuous compression. We present A*-Thought-V2, a geometric dynamics of LLM guided framework that models CoT as a hidden-state trajectory and replaces hard deletion with an explicit-implicit interleaved latent architecture. After projecting question, step, and solution representations into a 3D PCA space, it measures alignment between each local transition and global question-to-solution direction. Aligned steps remain explicit text, whereas deviating steps are compressed into continuous latent tokens. Directional angles capture both local semantics and reasoning dynamics: small angles indicate direct execution and answer formation, while large angles more frequently involve checking, correction, and branch exploration; their temporal variation reveals exploration, convergence, and refinement stages. To train this architecture, we introduce stepwise embedding forcing, which pools each redundant step into a single latent embedding, and label forcing, which supervises that latent token with a soft multi-modal vocabulary distribution instead of a hard one-hot label. Experiments on Qwen3.5-9B and Qwen3.6-27B across six in-domain and out-of-domain benchmarks show that A*-Thought-V2 improves average accuracy by up to 2.6% while reducing response length by up to half, increasing Accuracy per Computation Unit by 2.29times, and reducing preprocessing and training time by 94.6% and up to 80.3%, respectively. Representation analyses suggest that latent states form a compact region distinct from textual states, while higher entropy at latent-token positions reflects broader soft targets that encourage richer step-level feature learning.

arXiv 原文

订阅 AI Pulse

每天 08:00 · 12:30 · 18:30 · 23:50 更新