The format of multi-view plus depth (MVD) is introduced in 3D-HEVC for improving coding efficiency. However, as the number of viewpoints increases, the computational complexity increases dramatically. To overcome this issue, a method based on RD-cost Bayesian Decision (RDBD) for 3D-HEVC fast depth map inter coding is proposed in this paper, including coding unit (CU) early termination and SKIP/DIS decision. First, an offline training model with the minimum risk Bayesian decision rule is used to predict optimal CU size in depth map. Then, a minimum error rate Bayesian decision rule is proposed to determine whether SKIP or DIS is the best mode and ignores the rate distortion optimization (RDO) process of other modes. Experimental results show that, with the proposed algorithm, 45.7% depth map encoding time saving is achieved compared to the original 3D-HEVC, and outperforms state-of-the-art fast inter coding methods for 3D-HEVC.