Multimedia forensics is an important field addressing the increasing misuse of digital content, such as deepfakes and face-swapping technologies. This paper focuses on detecting face swapping. Our goal is not to decide whether face swapping has occurred. We assume that we execute a forensic investigation in which it needs to be learned which photo of a person’s face has been used for the face swap. We take a number of potential source face photographs and compare their behavior when reproducing the face swap. We show that the photo used for the face swap can be identified even after lossy compression and scaling.