给机器人装上一双通用的手
机器人摸东西这件事,一直卡在一个很笨的地方:触觉传感器各家各做各的,压阻的、磁的、视觉的,物理原理完全不同,模型换一种传感器就得重学一遍。这篇把「摸到的感觉」从具体硬件里剥出来,做成一套统一的触觉表示,三种传感器喂进去都变成同一种 token,新传感器也能接。更关键的是这些 token 即插即用:塞进扩散策略、塞进 π0.5 和 LingBot-VLA 这类视觉-语言-动作模型,复杂操作任务都变稳,仿真和真机、多个硬件平台都验过,算力开销也没炸。它离你家还远,但它指向的方向很具体:以后机器人换手、换传感器不用重训,触觉数据能像视觉数据一样攒起来规模化。
📄 原文摘要(英文)
Tactile sensing is an indispensable modality for contact-rich dexterous manipulation, a capability that hints at the ultimate frontiers of embodied intelligence. However, commercially available tactile sensors are still at a relatively early stage of development, with each type of sensor offering irreplaceable strengths alongside its own drawbacks. A key challenge is thus to free tactile understanding from sensor-specific physics, so that embodied models can effectively leverage heterogeneous tactile data. To this end, we develop LingBot-Touch, a tactile foundation model characterized by the following properties. (i) Our model yields a unified representation across different tactile sensors (i.e., piezoresistive, magnetic, and visuotactile sensors), and is easily extensible to new sensor types. (ii) The resulting tactile tokens are plug-and-play: they can be seamlessly integrated into policy models across paradigms, from diffusion policy to vision-language-action (VLA) models such as π_{0.5} and LingBot-VLA, consistently enhancing manipulation dexterity. (iii) Extensive experiments, spanning simulation and real-robot deployments across multiple hardware platforms, demonstrate that our tactile representation substantially benefits complex manipulation tasks while maintaining strong computational efficiency. We believe the findings in this work demonstrate the potential of leveraging cross-sensor data to scale tactile learning, and offer encouraging insights for the design of future hardware and embodied models.