AI 推理变强，账单却藏在你没看的地方
让 AI 多算几遍再回答，准确率确实涨了——但同样的「多算几遍」，执行方式不同，电费和等待时间能差出 5 倍。研究者把 8 个候选答案一次性批量生成，和拆成 8 次逐个生成做对比：后者耗电是前者的 4.6 到 4.9 倍，延迟更是高出近 6 倍，而准确率完全一样。也就是说，你为「更聪明」付的钱，可能大半花在了低效的调度上。这篇不是给你明天调参用的，但它提醒了一件事：AI 的进步账单，往往藏在没人报的工程细节里。
📄 原文摘要(英文)
Test-time scaling can improve large language model reasoning by generating and combining multiple candidate responses. In sampling-based methods, the inference budget is often described by the number of generated candidates, N. However, N tells us how many candidates are generated, not how they are executed. The same candidate budget can be produced in one batched generation call or split across several sequential calls with smaller batch sizes. We first study the effect of increasing N on reasoning accuracy using Phi-3-mini and Qwen2.5-1.5B on 500 GSM8K prompts. As expected, increasing N from 1 to 8 improves accuracy by 8.4 percentage points for Phi-3-mini and 18.4 points for Qwen2.5-1.5B. However, accuracy alone does not show the systems cost of using a larger candidate budget. We therefore fix N = 8 and compare four generation schedules: 1x8, 2x4, 4x2, and 8x1, where axb denotes a generation calls with b candidates per call. We measure latency, throughput, GPU-hours, and gross GPU-device energy while keeping the total candidate count fixed. On A100 GPUs, eight serial calls use 4.64-4.86x as much gross GPU-device energy and have 5.77-6.12x the P95 latency of one batched call with eight candidates. The same pattern appears across three independently scheduled A100 nodes per model and in short-output SciQ/V100 experiments. These results show that candidate count alone is not enough to describe the systems cost of multi-candidate test-time scaling. When candidates are independent and memory allows it, fewer generation calls with larger batch sizes are more efficient. Evaluations should therefore report not only candidate count and accuracy, but also generation schedule and GPU-level systems metrics.