手机本地小模型,自动为你定制
大模型跑在手机上,隐私和离线都解决了,但真正卡住人的是另一件事:同一个模型,换个任务、换台手机,效果就塌。过去这得靠专家反复调。这篇让一个「编排 LLM」自己当厂长:给它任务说明和目标机型,它自己决定是调超参、挖真实数据还是造合成样本,规则负责把掉准确率的改动回滚;它从小到大搜模型和量化方案,边搜边改自己的准确率目标,还真机实测不同推理引擎。十个任务、四台安卓机,它造出的端侧模型在五个任务上打赢两个五次示例的前沿大模型(最高领先 0.34),另三个打平;搜索效果离「把 27 个变体全训一遍」只差 0.03,GPU 时间省 19 倍。还有个细节:最佳配置因手机而异,不给真机测量,连 Claude Opus 5.5 拿着完整硬件参数也会在 25% 的简单/中等场景、60% 的困难场景里不达标。想把某个小模型塞进自家 App、又不想养一个调参团队的人,这是能直接抄的流水线。
📄 原文摘要(英文)
Small language models (SLMs) running on mobile devices can keep user data private and work without connectivity. But they perform well only when fine-tuned for the target task and sized for the target device, a process that today requires repeated expert tuning. We present {\it Mobile AI Factory}, an agentic system that builds a fine-tuned, quantized SLM for a given task and phone. Given a task specification and a target phone, an orchestrator LLM chooses whether to tune hyperparameters, mine real data, or generate synthetic examples, while fixed rules roll back changes that hurt accuracy. The factory searches from small to large models across base models and quantizations, adapts its accuracy goal as it searches, and uses real measurements of the target phone across inference engines and backends. We evaluate Mobile AI Factory on ten tasks and four Android phones. On an unseen testset, its on-device models outperform two five-shot frontier models on five tasks, by up to 0.34, and match them on three more. Its search comes within 0.03 of exhaustively training all 27 model variants on an entity extraction task, while using 19times less GPU time. Phone measurements also matter: the best configuration changes across phones. Without phone measurements, Claude Opus 5.5, given the full hardware and model specifications, misses the app's requirements in 25\% of easy and moderate cases and in 60\% of hard ones. Code is available at: https://github.com/evanly-gh/SLM_Factory.