This paper presents a novel method for 3D scene modeling using stereo vision, with an application to image registration. The method constists of two steps. First, disparity estimates are refined, by filling gaps of invalid disparity and removing halos of incorrectly assigned disparity. A coarse segmentation is obtained by identifying depth slices, after which objects are clustered based on color and texture information using Gabor filters. The second step consists of reconstructing the resulting objects in 3D for scene alignment by fitting a planar region. A 2D triangle mesh is generated, and a 3D mesh model is obtained by projecting each triangle onto the fitted plane. Both of these extensions result in improved alignment quality with respect to the state of the art, and operate in near real time using multi-threading. As a bonus, the refined disparity map can also be used in combination with the existing method.