Many archival photos are unique, existed only in a single copy. Some of them are damaged due to improper archiving (e.g. affected by direct sunlight, humidity, insects, etc.) or have physical damage resulting in the appearance of cracks, scratches on photographs, non-necessary signs, spots, dust, and so on. This paper proposed a system for detection and removing image defects based on machine learning. The method for detecting damage to an image consists of two main steps: the first step is to use morphological filtering as a pre-processing, the second step is to use the machine learning method, which is necessary to classify pixels that have received a massive response in the preprocessing phase. The second part of the proposed method is based on the use of the adversarial convolutional neural network for the reconstruction of damages detected at the previous stage. The effectiveness of the proposed method in comparison with traditional methods of defects detection and removal was confirmed experimentally.