同一张图,这个AI看政策决定封不封
现在的AI安全审核默认一张图要么安全要么危险,但现实是同一张图在不同产品、不同时期政策下可能截然不同。研究者发现现有模型在政策变化时表现脆弱,于是构建了PolicyShiftBench——265张图每张配7.55条政策条件,测试模型是否真按政策判断而非凭直觉。他们提出的PolicyShiftGuard用两阶段训练:先随机配对政策做微调,再用边界对策略让模型学会区分“该放行”和“该拦截”的边界。7B模型在基准上F1达76.9,且能迁移到其他安全基准。这不是你明天能用的工具,但它揭示了AI安全审核的一个关键盲区:政策适应性。
📄 原文摘要(英文)
Image guardrails are typically trained and evaluated under a fixed safety policy, implicitly treating safety as an intrinsic property of an image. Real deployments are different: the same image may be allowed in one product, restricted in another, and newly disallowed when a policy boundary changes. We study policy-adaptive image guardrailing, where a model must decide whether an image violates the currently supplied policy and generalize to held-out policy definitions. We introduce PolicyShiftBench, a comprehensive benchmark with 2,000 policy-discriminative instances over 265 images, where each image is paired with 7.55 policy-conditioned prompts on average to test whether models adapt to the active policy rather than relying on image-level safety priors. We then propose PolicyShiftGuard, a compact policy-conditioned guardrail trained with a two-stage training recipe that combines Randomized Policy SFT (RP-SFT) with Boundary-Pair Policy Adaptation (BP-Adapt). BP-Adapt trains matched prompts for the same image and risk category using standard label supervision and a pairwise comparison loss that separates blocking policies from passing policies. Experiments show that existing VLMs and specialized guardrails remain brittle under policy shifts, while PolicyShiftGuard substantially improves policy-sensitive performance. The 7B model achieves SOTA performance of 76.9 Avg. F1 and 72.1 Avg. PSS on PolicyShiftBench, transfers well to UnSafeBench and SafeEditBench, and improves the latency-performance trade-off with a concise output format. Ablations confirm that matched pass/block boundary pairs are essential for stable policy adaptation.