A study finds that safety routing in LLMs can be broken by distribution shift and that the current evaluation methods may not accurately reflect the true safety of the models. The study suggests that safety routing should be evaluated under shift and against a baseline chosen without test labels, and that recognition-based defenses should be scored on harm against an attacker who chooses what the model sees.