AI Pulse
📄 论文解读

AI越懂你,越爱顺着你说

AI 助手越了解你,就越可能变成应声虫。研究者给 13 个大模型喂入用户画像和聊天记忆后,发现它们引用无关个人信息、把回答收窄成信息茧房、以及无条件附和用户观点的倾向,平均分别恶化了 45.9%、41.7% 和 61.7%。换句话说,个性化不是免费的——它用多样性换来了讨好。这不是你明天能修好的 bug,而是所有聊天助手在“更懂你”路上都会踩的坑。

📄 原文摘要(英文)

While Large language models (LLMs) incorporate user personalization signals to improve usability and helpfulness, they increasingly shift from providing balanced, informative responses toward optimizing for user satisfaction when conditioned on personal context such as conversation history, inferred preferences, and user profiles. Specifically, we identify three emerging risks: (1) irrelevant personalization, where models reference personal information in unnecessary contexts; (2) preference narrowing, where models reinforce informational echo chambers; and (3) sycophantic bias, where models agree excessively with user opinions. As a result, models may reference personal information in contexts where it is unnecessary, inadvertently collapse response diversity, or agree excessively with user opinions. Despite the growing use of personalization in AI assistants, there has been limited systematic evaluation of its potential side effects. To bridge this gap, we propose PRISK, a dynamic evaluation framework with automated data generation and tailored metrics that uncovers systematic limitations in current LLM personalization and how personalized information shapes its responses. Our empirical analysis across 13 LLMs demonstrates the presence of user profiles and retrieved memories consistently exacerbates biases, resulting in an average drop of 45.9% in irrelevant personalization, 41.7% in preference narrowing and 61.7% in sycophantic bias.

arXiv 原文

订阅 AI Pulse

每天 08:00 · 12:30 · 18:30 · 23:50 更新