让AI学开车：先画一张俯视地图，再脑补前方路况
现在的自动驾驶世界模型有个死穴：它只能从司机的视角看路，于是旁边车的运动跟它的长相、遮挡全搅在一起，学不干净。这篇叫 BirdDreams 的模型换了个思路——它同时生成两路视频：一路是正常的挡风玻璃视角，另一路是像游戏小地图那样的俯视示意图，路上每辆车就是一个色块。俯视图把位置和运动讲得清清楚楚，透视图把红绿灯和细节补上，两路互相参考一起生成。更妙的是，你想让车往哪开，不用重新生成，直接把俯视图的最后一帧按你的轨迹平移旋转，再喂回去当条件就行。在 PhysicalAI-AV 和 nuScenes 两个数据集上，它比之前的模型更听话地跟着你的轨迹走，人眼评测也觉得它生成的车流更自然。它不是你明天能装进车里的东西，但它指向一个更省力的方向：让模型先学会画地图，再学会看路。
📄 原文摘要(英文)
Driving world models aim to simulate how a scene unfolds under different ego actions. This requires plausible motion of surrounding vehicles as well as precise ego-trajectory control. Learning vehicle motion from perspective video alone is challenging because motion is entangled with appearance and vehicles can be occluded. Prior work therefore conditions video generation on layouts that specify vehicle motion, providing control over surrounding vehicles but requiring their future trajectories as inputs. We introduce BirdDreams, an autoregressive driving world model that jointly generates perspective video and a schematic bird's-eye-view (BEV) video, making scene layout part of the predicted future. The BEV, rendered from scene annotations for training, depicts roads and vehicles as simple shapes at a fixed metric scale, providing a top-down view of their positions and motion with reduced occlusion between vehicles. Perspective video, by contrast, retains scene details that the BEV omits, such as traffic signals and object appearance. Joint attention between two video diffusion transformers, one per stream, lets each stream inform the other during generation. The predicted BEV also provides a geometric interface for ego control: we translate and rotate its latest frame along an input ego trajectory and use the resulting frames as a soft condition that shows how the scene would appear to move if only the ego vehicle moved. At inference, BirdDreams generates both videos from a single camera image without requiring an HD map or vehicle-state inputs. On PhysicalAI-AV and nuScenes, BirdDreams follows input ego trajectories more faithfully than prior driving world models and generates more plausible ego motion without ego-action conditioning. Human evaluation shows that BirdDreams outperforms a perspective-only baseline in generating realistic surrounding-vehicle behavior.