一个模型，想用多深用多深
大模型有个尴尬：训练时定死了深度，推理时想多花算力换更准答案，只能重新训练一个更大的模型。Dory1.0-4B 把「深度」变成了可调节的旋钮——同一个模型，推理时可以让核心模块循环 1 次、2 次还是 4 次，循环越多越准，而且多出来的计算花在「内部思考」上，不是多写废话。在编程、智能体、长文本任务上，循环 2 次以上就超过了比它大好几倍的模型。它还顺手解决了循环带来的内存翻倍问题：只缓存最后一轮，内存占用和普通模型一样。这不是你明天能下载来用的产品，但它指向一个趋势：未来的模型可能不再按参数大小分档，而是同一个模型按任务难度动态决定思考多深。
📄 原文摘要(英文)
We introduce Dory1.0-4B, a 4-billion-parameter looped (latent-recurrent) language model with elastic depth: a single checkpoint serves different inference compute budgets. Dory1.0-4B is built on a normalized Transformer (nGPT) backbone with interleaved sliding-window and full-attention layers, organized into an input block, a shared recurrent block, and an output block. At inference, the recurrent block can be applied a variable number of times. We pre-trained Dory1.0-4B on 6 trillion tokens. The first 4.5 trillion tokens were trained in NVFP4, and additional pre-training then extended the context length to 256K tokens. Next, we post-trained the model using Supervised Fine-Tuning (SFT) and Reinforcement Learning with Verifiable Rewards (RLVR), with a difficulty-aware loop curriculum that routes harder prompts to deeper loops. Increasing the loop count improves accuracy, with the largest gains on coding, agentic, and long-context benchmarks. With two or more loops, Dory1.0-4B outperforms larger models such as Qwen3.5-9B, Gemma 4 12B, and Gemma 4 26B-A4B on average across 12 benchmarks. Looping also makes Dory1.0-4B more token-efficient on agentic benchmarks: with four loops, it generates 1.3-2.5x fewer tokens than with one loop, as the extra computation is spent in latent space rather than on longer outputs. To reduce memory during looping, Dory1.0-4B caches keys and values only from the last loop, keeping its KV-cache footprint equal to that of a non-looped model at any loop count while retaining most of the accuracy gains. We open-source the BF16, MXFP8, NVFP4, and GGUF quantized checkpoints.