你的电脑能跑 753B 大模型了
大模型通常被认为只能在数据中心跑，但 FreeToken 让个人电脑也能运行超大模型：一台 8GB 显存的笔记本能跑 35B 模型，游戏台式机能跑 284B，工作站显卡甚至能跑 753B 的 GLM-5.2。它不把个人电脑当小 GPU，而是当成一个弹性推理平台，根据你机器上 CPU、GPU、内存的实际配比，动态分配计算和模型数据，而不是固定一种卸载策略。它支持 20 多种 MoE 模型和真实的编码、工具调用智能体。这不是你明天就能装上的东西，但它意味着开源模型正在从云端走向你的本地设备，未来你的电脑可能就是你的 AI 服务器。
📄 原文摘要(英文)
Frontier open-weight models are increasingly available, but serving them still largely assumes datacenter infrastructure. We present FreeToken, an edge-native MoE serving system that treats a personal machine not as a small GPU, but as a unified, elastic inference platform. FreeToken co-designs the full serving stack, including model layout and loading, expert residency, CPU--GPU execution, agentic state reuse, and runtime memory management, around two realities of local AI: agent workloads continuously change their execution pattern, and edge hardware exposes heterogeneous resources whose balance differs from machine to machine. Rather than committing to a fixed offloading strategy, FreeToken continuously maps computation and model state onto the resources actually available. FreeToken supports more than 20 MoE models and real coding and tool-using agents across hardware ranging from an 8GB laptop GPU to a single workstation GPU. More importantly, it changes what these machines can practically serve, from a 35B model on a laptop to a 284B model on a gaming desktop and the 753B GLM-5.2 on a single workstation GPU. FreeToken turns open weights into deployable local software, making the machines users already own a practical platform for frontier-scale intelligence. We release the system at flashml.ai.