Computer vision applications are able to model and reconstruct three dimensional scenes from several pictures. In this work, we are interested in the group of algorithm that register each image with respect to the model and aim at constructing a model of the scene. At the lowest level, most of these algorithms are comparing the pixel values of the image to the ones predicted by the model to refine the result. As research advances, the models are getting better and better, but no matter how complex they are, there will always be unpredictable situations that cannot be handled by the model. A recurring example is when an object appears in one image of the set, but in none of the others. The situation occurs, for example, when a moving entity crosses rapidly the field of view of the camera. In this work, we study the error generated by such an unexpected object at a pixel level and how colour can improve the estimation. We will derive the expected error distribution that this hypothetical object may cause.Our model is primarily intended as a basis for outlier removal in scene modelling algorithms. It gives a clear answer to whether, and with which confidence, a part of the image can be considered as part of the model or should be discarded, without using any dedicated thresholding scheme.The model is demonstrated on a trivial example where we match two images of a scene using a static camera. The example shows that the outlier distribution can be predicted by using the histograms of both images. We also show that by considering not only greyscale information, but also colour information, the outlier detection performance improves. We want to emphasise that the central part of this paper is the outlier modelling and not the outlier rejection scheme, which could be solved—for the trivial examples we are showing—by many other techniques.