Establishing dense correspondence fields between images is an important issue with many computer vision and computational photography applications. Although there have been significant advances in estimating dense correspondence fields, it is still difficult to find reliable correspondence fields between a pair of images because of their geometric and photometric variations. In this paper, we propose an unified framework for establishing dense correspondences, consisting of sparse matching, multilevel segmentation, and derivation of affine transformations. Dense correspondence fields are estimated via winner-takes-all (WTA) optimization by utilizing affine transformations, derived from spare matching and multilevel segmentation. The proposed method reduces a size of label search space dramatically, and further extends the dimension of label search space, by leveraging affine transformation with the multilevel segmentation scheme. Our robust dense correspondence estimation is evaluated on extensive experiments, which show that our approach outperforms the state-of-the-art methods both qualitatively and quantitatively.