This paper explores the impact of masking-based token pruning on the robustness of CLIP models. It proposes a pre-deployment diagnostic called the Spurious Inversion Metric (SIM) to predict whether masking helps or hurts worst-group robustness. The study finds that masking can have a significant impact on model performance and introduces a batched GPU segmentation routine to mitigate its drawbacks.