The latest High Efficiency Video Coding (HEVC) standard achieves about 50% bit rate saving while maintaining the same subjective quality compared to H.264/AVC High Profile. However, the better coding efficiency is obtained at the cost of significantly increased encoder complexity. In this paper, a fast mode decision algorithm is proposed to relieve the computational burden. The proposed algorithm consists of two steps. Firstly, depth information of CU block is used to reduce the mode candidates, with the assumption that the exhaustive search for a large CU is unnecessary. Secondly, a directional ratio is applied to estimate the rough orientation of each CU, thus some unlikely selected directional modes can be further eliminated. On average, the proposed algorithm can achieve 34.1% encoder time saving while cause an negligible coding performance loss under the All-Intra configuration compared with HM 16.0.