The rapid development of 3D display technologies allows consumers to enjoy the 3D world visually through different display systems such as the stereoscopic, multiview, and light field displays. Despite the hype of 3D display development, 3D contents for various display systems are very insufficient and the most of 3D contents are only stereo sequences for stereoscopic 3D. To handle with the lack of 3D contents for various display systems, various 3D display images are generated by stereo sequences. Howver, the conventional 3D display rendering algorithm suffers from increased computational complexity and memory usage with the increasing number of view in 3D displays to achieve a sufficient level of reality. This paper proposes an efficient method to generate 3D display images by 3D direct light field rendering. We propose new 3D image generation algorithm for hole filling, boundary matting, and postprocessing from common stereo input images.