In this article we address the problem of the recovery of a realistic textured model of an image sequence without any prior knowledge either about the parameters of the cameras, or about their motion. Firstly, using various computer vision tools, we establish correspondences between the image pairs and estimate the fundamental matrix. Secondly, using epipolar geometry constraints, we can obtain the rectified image pairs by a novel rectification method, where the epipolar lines coincide with the image scan-line. Furthermore, we can make dense stereo matching original image pairs rapidly and simply. Thirdly, in self-calibration, the prior knowledge of orthogonal wall planes and the orthogonal and parallel lines are formulated as constraints on the absolute quadric. Finally, the 3D Euclidean model can be built through self-calibration, matching Delaunay triangulation. In producing a 2-D mesh for VRML, an algorithm for the depth adaptive mesh is proposed to give a more detailed description of the reconstructed 3-D surface of a building. A large number of experimental results show that this method increases the speed and accuracy of the reconstructed 3D model and the obtained 3D models are more realistic.