推荐系统换新知识不再推倒重来
推荐系统常用的做法是「先预训练、再迁移」，但用户行为一直在变，预训练模型也得跟着刷新。过去一刷新，之前学到的任务知识就可能被冲掉，得重新调。这篇把「行为知识」和「任务知识」拆开存：行为知识放在一个可刷新的编码器里，任务知识放在另一个只读的模块里，两者互不干扰。这样模型刷新时，任务知识稳稳不动，行为知识持续更新。在 8 个公开数据集上比强基线好 4-12%，在 Shopee 首页搜索的线上测试里，人均 GMV 涨了 1.75%，广告收入涨了 1.53%。这不是你明天能直接用的技术，但它解释了为什么推荐系统总在「越更新越乱」，以及怎么拆开才能不乱。
📄 原文摘要(英文)
Industrial recommenders increasingly adopt the pretrain-then-transfer paradigm, yet behavioral distribution drift raises two questions: what to learn from behavior sequences, and how to transfer the learned knowledge while the pretrained model is continually refreshed. To resolve them, we propose Knowledge-Geometry Decoupling (KGD). For what to learn, conventional next-token prediction treats adjacency as dependency and may encode spurious transitions across unrelated sessions. We introduce Behavioral Multi-Token Prediction (BMTP) to retain only collaboratively or semantically related future items as supervision, yielding cleaner and more transferable behavioral knowledge. For how to transfer, pretrained knowledge and task-specific geometry impose conflicting optimization demands on shared parameters. To handle it, KGD assigns them to separate parameter sets: a refreshable encoder owns behavioral knowledge, while a task learner reads contextualized encoder states through read-only cross-attention and writes task-specific geometry through Anchored Calibration Residual (ACR) orthogonal to the pretrained embedding. The decoupled ownership enables continual knowledge refresh without task-gradient interference or invalidating downstream adaptation. KGD improves over strong pretrain-transfer baselines by 4-12% on eight public benchmarks and sustains its advantage over a 90-day production stream where baselines show no gains. KGD has been fully deployed in Shopee. In a live A/B test on Shopee Homepage Search, it increases GMV per user by 1.75% and advertising revenue by 1.53%, demonstrating its high practical value. We provide the core implementation of KGD at https://github.com/FuCongResearchSquad/KGD4REC.