This paper proposes a new approach to continual audio deepfake detection, called RF-Prompt, which learns to detect new deepfake methods while retaining the ability to recognize previously encountered speech. The method uses a shared real prompt and inherited fake experts with orthogonal residuals to preserve and expand knowledge.