There are three critical problems that need to be tackled in target detection when both the target and the photodetector platform are in flight. First, the background is a sky–ground joint background. Second, the background motion is slow when detecting targets from a long distance, and the targets are small, lacking shape information as well as large in number. Third, when approaching the target, the photodetector platform follows the target in violent movements and the background moves fast. This article is comprised of three parts. The first part is the sky–ground joint background separation algorithm, which extracts the boundary between the sky background and the ground background based on their different characteristics. The second part is the algorithm for the detection of small flying targets against the slow moving background (DSFT-SMB), where the double Gaussian background model is used to extract the target pixel points, then the missed targets are supplemented by correlating target trajectories, and the false alarm targets are filtered out using trajectory features. The third part is the algorithm for the detection of flying targets against the fast moving background (DFT-FMB), where the spectral residual model of target is used to extract the target pixel points for the target feature point optical flow, then the speed of target feature point optical flow is calculated in the sky background and the ground background respectively, thereby targets are detected using the density clustering algorithm. Experimental results show that the proposed algorithms exhibit excellent detection performance, with the recall rate higher than 94%, the precision rate higher than 84%, and the F-measure higher than 89% in the DSFT-SMB, and the recall rate higher than 77%, the precision rate higher than 55%, and the F-measure higher than 65% in the DFT-FMB.