Udder ranking is one of the crucial traits and used extensively in cattle breeding. The analysis of the udder images is challenging due to the variations in the captured conditions of the non-rigid nature of the organ, the farm environment, and disturbances in the form of irrelevant segments of other cattle parts. To this end, we proposed a deep learning-based udder classification algorithm to enhance registrations’ precision within cattle breeding. We explore a convolution neural network (CNN), namely the VGG-16 model. The model is trained and validated on a cattle dataset that is collected in Norwegian dairy cattle farms. Expert technicians in the form manually annotate the dataset. We demonstrate that the VGG-16 model used as the backbone can efficiently give an acceptable performance with training and validation accuracy of 97% and 93% respectively on our custom dataset.