In a complex background, insulator fault is the main factor behind transmission accidents. With the wide application of unmanned aerial vehicle (UAV) photography, digital image recognition technology has been further developed to detect the position and fault of insulators. There are two mainstream methods based on deep learning: the first is the “two-stage” example for a region convolutional neural network and the second is the “one-stage” example such as a single-shot multibox detector (SSD), both of which pose many difficulties and challenges. However, due to the complex background and various types of insulators, few researchers apply the “two-stage” method for the detection of insulator faults in aerial images. Moreover, the detection performance of “one-stage” methods is poor for small targets because of the smaller scope of vision and lower accuracy in target detection. In this article, the authors propose an accurate and real-time method for small object detection, an example for insulator location, and its fault inspection based on a mixed-grouped fire single-shot multibox detector (MGFSSD). Based on SSD and deconvolutional single-shot detector (DSSD) networks, the MGFSSD algorithm solves the problems of inaccurate recognition in small objects of the SSD and complex structure and long running time of the DSSD. To resolve the problems of some target repeated detection and small-target missing detection of the original SSD, the authors describe how to design an effective and lightweight feature fusion module to improve the performance of traditional SSDs so that the classifier network can take full advantage of the relationship between the pyramid layer features without changing the base network closest to the input data. The data processing results show that the method can effectively detect insulator faults. The average detection accuracy of insulator faults is 92.4% and the average recall rate is 91.2%.