In this paper, we propose a simple but efficient video segmentation scheme for real-time video applications. First, we temporally separate video frames into scenes, comparing the chisquare distance between consecutive frames. To partition each frame into disjoint regions, then, a pixel-wise color clustering scheme is employed, which is based on K-means clustering and EM algorithm. Finally, we regularize computational complexity to apply the proposed scheme into embedded video processing system. Due to pixel-wise video segmentation with very low complexity, the proposed scheme yields a realistic framework for real-time video applications.