让机器人看视频学走路：不靠真实训练，靠互联网
机器人学走路，通常得在真实世界里摔很多次，或者用模拟器反复练，但模拟和现实总有差距。这篇论文换了个思路：让机器人看互联网上大量第一人称视频——比如人拿着手机走向某个目标——从中学会“当目标在视野里时，画面会怎么变化”，再用这些规律直接生成自己的动作。他们在模拟测试里成功率最高到94.9%，还真的放到四足机器人上跑通了。它不是你明天就能买回家的功能，但指向一个更便宜、更通用的训练方向：让机器人从人类已有的生活录像里学，而不是每次都为它专门造一个世界。
📄 原文摘要(英文)
We study language-conditioned local navigation, where a robot must navigate to a text-specified target visible in its initial observation and stop at the described location. End-to-end navigation policies require diverse training data to generalize across scenes, yet real-robot data is costly to collect and simulated data leaves a sim-to-real gap. Internet egocentric videos offer a scalable source of real-world navigation experience. Our key insight is to learn goal-directed visual dynamics from Internet egocentric videos, and use these dynamics to support robot action generation. To this end, we propose ScaleNav, a unified world-action framework with three-stage training. We first perform instruction-conditioned video pretraining on these videos to learn how observations evolve as the camera approaches instructed targets across diverse real-world scenes. Action alignment then uses action-labeled simulated data to train the model to predict executable action chunks. Joint fine-tuning subsequently optimizes all trainable components together for navigation. ScaleNav achieves success rates of 86.5% on a single-view adaptation of Short-Horizon OVON and 94.9% on our HabitatGS-LN benchmark, outperforming the evaluated baselines on both benchmarks. We further deploy and evaluate ScaleNav on a real quadruped robot, with results supporting the model's ability to generalize to real-world navigation. Code and videos are available at https://zju3dv.github.io/ScaleNav/