Some Retinex-based implementations explore the images by using paths or computing ratios with neighbours in a multilevel framework [3, 15–19] or using models of Brownian motions to analyse the image along paths [20, 21]. Other implementations compute values over the given image with convolution masks or weighting distances [22–26]. In a study [10], the path-based scanning was substituted by a new approach using random sampling of a cloud of points.

A similar approach was followed for the STRESS algorithm [9]. Here, the visual context was characterised using two envelopes Emax and Emin. For each pixel, p0, the values of the maximum and minimum envelopes at the corresponding positions, are computed through N iterations. In every iteration, M pixel values pi in each channel are sampled at random with a probability proportional to 1∕d, d being the Euclidean distance in the image from the sampled pixel to the pixel in question. The value of the centre pixel is not eligible for random sampling but is always included in the sampled set. From these samples, the maximum and minimum samples in the spray are found: smax = max(pj), smin = min(pj). Since p0 is always among the sample points, it is granted that smin ≤ p0 ≤ smax. These maximum and minimum points could be taken as direct estimates for the envelopes. However, better results were achieved when the relative position vi of the pixel p0 within the range ri = smax − smin was used.

The final envelopes were computed as where v̄ is the average of the v values, and r̄ is the average of the r values over the N iterations. The new image is recomputed by stretching the image to these envelopes, It should be noted that this is a heavy computational technique requiring O(NMP) operations, where N is the number of iterations, M is the number of samples per iteration, and P is the number of pixels in the image.

The sampling technique also introduces a significant chromatic noise. The type of noise is quite particular to STRESS. To the best of our knowledge, this type of noise has not been characterised in the literature. A histogram of the noise in each colour channel, produced by computing the difference between STRESS images with 100 (noisy) versus 1000 (virtually noiseless) iterations, is shown in Figure 1. It is symmetric, but far from Gaussian. There is no correlation between the noise of the different image channels.

Since this method produces significant chromatic noise, another approach is taken in the STRETV algorithm [11]. Here, the constrained total variation method is used to calculate the envelopes. Total variation minimisation results in a process very similar to non-linear isotropic diffusion. This implementation showed promising results when used in contrast enhancement and in automatic colour correction, with significantly reduced noise levels. However, due to the non-linearity of total variation, the method requires a tiny time-step and thus many iterations to converge. Although it behaves nicely near edges in the original image, it creates some artefacts near corners and lines of high curvature.

Various diffusion techniques have been widely used in computer vision and image processing to reduce image noise without removing significant information from the image. One important technique is the Perona–Malik diffusion [27], which is an isotropic, local, non-linear diffusion technique, not too different from total variation minimisation. Unfortunately, Perona–Malik diffusion was misnomed “anisotropic diffusion”. Real anisotropic diffusion for image processing was first described by Tschumperle and Deriche [28] as a non-linear process.

The starting point for Tschumperle and Deriche non-linear anisotropic diffusion is the 2 × 2 structure tensor S of the original image [29], whose components for every single pixel can be expressed as Here u0 denotes the original image, the indexes μ and ν denote the colour channels, and xi and xj denote the two spatial directions. The eigenvalues of the structure tensor S are denoted λ+ and λ−, and the corresponding normalised eigenvectors e+ and e− are stored as columns in the orthonormal eigenvector matrix V , such that the structure tensor can be written S = V Tdiag(λ+, λ−)V  [14]. From this, the diffusion tensor is then defined as where d(λ) is a nonlinear diffusion coefficient function (Eq. (6)) [14] whose task is to suppress the diffusion across the edges while preserving it along the edges, and κ is a suitably chosen numeric constant. Higher values of κ will give more edge preservation in the image.

Having defined the diffusion tensor, D, anisotropic diffusion results from solving the Euler–Lagrange equations for minimising its eigenvalues by gradient descent using the artificial time parameter t (corresponding to the iterations when discretised),

It has been ascertained in various studies (see, e.g., [13]) that the structure tensor, and thus the diffusion tensor, can be computed once and for all from the original image. Then, the diffusion equation, Eq. (7) becomes linear. In the same study, it was also found that the anisotropic diffusion was better than the isotropic one (Perona–Malik-type) for preserving edges and, in particular, corners.

It should be noted that, even after linearisation, the solution of the anisotropic diffusion equation employing iterative gradient descent is, like STRESS, a computationally heavy procedure. The diffusion lengths, and thus the number of iterations needed, are based on the image size, and the diffusion tensor that reduces the diffusion locally increases the need for iterations further.

The envelopes of STRESS are images Emax and Emin with the property that, for each image channel in the original image u0, 0 ≤ Emin ≤ u0 ≤ Emax ≤ 1. The basic idea here, is to use the diffusion tensor of anisotropic diffusion, Eq. (5) to compute alternative versions of the STRESS envelopes. This can be achieved in two different ways. Either, the original image is used as the initial value for both envelopes, or a black image is used as the initial value for the minimum envelope and a white image for the maximum one. In both cases, adding a data attachment term to the diffusion equation will ensure that the envelopes will stay reasonably close to the image. The resulting equations for the envelopes are

Normal boundary conditions are applied to the envelopes to make sure that there are no problems when calculating with the pixels at the border of the image. The equations are solved by the explicit Euler method for the time integration, and centred differences for the spatial derivatives. For each iteration, Emax and Emin are constrained so that they are greater than or smaller than the original image, respectively. The envelopes and the original image are finally used to recompute the image exactly as in the original STRESS algorithm.

In Fig. 2, the envelopes for STRESS and STREAD are shown. The envelopes contain some of the original image content, always brighter (for Emax) or darker (for Emin), and preserve the edges of the image. A plot of the corresponding scan lines is shown in Figure 3. One can see that the envelopes of STREAD are smoother than those of STRESS, and also that they, in general, are not so close to the original image. This means that STREAD will lead to less dramatic changes of the images. Moreover, both algorithms can follow sharp edges in the image (left part of the graphs). The less sharp transitions for the STREAD envelpes on the right part of the graph can be explained by other image information transversal to the scan line that diffuses into the shown line of the image.

As with the STRESS algorithm, the behaviour of STREAD varies with the values of its parameters. The influence of these parameters on the output of the algorithm is explained in detail.

Although both STRESS and STREAD converge to the final solution, iterations in STRESS and STREAD have quite different effects. STRESS iterates the stochastic sampling and computes the average to reduce the sampling chromatic noise, while iterations in STREAD leads to the stationary solution of the constrained PDEs.

The algorithm was tested with N ∈{100,200,300, 400,500,600,700,800,900,1000} iterations. With this test, it was easier to see the influence of higher iterations on the output. In Figure 4 the envelopes are shown for the extreme values of N in this set, and Figure 5 shows the resulting images. The edges are much sharper in the image with the most iterations. One can also notice some artefacts starting to appear near the edges.

As for the iterations, experiments were performed with κ to determine what value it had to be to give the best output. While testing the highest value for iterations, a value of κ = 1000 was used. When the number of iterations became higher than 700, artefacts like halos started to appear on edges in the image. Results for different values of κ are shown in Figures 6 and 7 for the envelopes and resulting images, respectively. In Fig. 7(a), we can see how the halos appear around the edges in the image (see, e.g., the greenish area in the sky close to the red/pink caps, number four from the left). To counter this, a higher value of κ is needed, since higher values of κ will give more edge preservation in the image. Even with κ = 100000 and N = 1000, some artefacts still appear around some edges, but lower number of iterations does not result in these artefacts. Based on this, κ was set to 10000 and N ∈{300,350,400,450,500,550,600} for the remaing studies.

The last variable is λ, which is used to control the strength of the data attachment. The data attachment term λ(u − u0) in Eqs. (8) and (9) is a regularisation term that incorporates the prior information about the original image into the enhancement process, and acts as a constraint that minimises the discrepancy between the envelope and the original image [30, 31]. If λ = 0, the envelopes will be flat, resulting in the algorithm becoming spatially independent or global. If λ →∞, the envelopes will be the same as the original image. Results for different values of λ can be seen in Figure 8.

Figure 9 shows a comparison between STREAD and STRESS for different numbers of iterations. It can be seen that STREAD is better at enhancing the vertical lines on the wall. STRESS also amplified the noise present in the more uniform areas.

One challenge with the preliminary experiments was that while STREAD kept more or less the same colour balance as the original, STRESS made the images a bit brighter and a bit blue, see Figure 11. In order for the images to become more comparable, the last stage of STRESS was modified to counteract this. Two linear scalings for preserving white and gray and a gamma correction for preserving gray were added. The difference between these two can be seen in Figure 10. Linear scaling to preserve gray was chosen, as it was the one that looked most similar to the original and STREAD. The scaling is performed for each channel separately by multiplying the result with the mean of the original image, and then dividing it by the mean of the resulting image. After these preliminary experiments, the final image set was chosen.

In a preliminary experiment, we compared visually the images resulting from STREAD to those of STRETV [11], and found that the STREAD images were so much better than the STRETV ones, that STRETV was left out of the experiments. The images generated by STREAD and STRESS were compared using a subjective survey and objective image metrics. Initial tests indicated that STREAD performed well with κ = 10000, λ = 0.001 and N ranging from 300 to 600. A sample of 10 standard RGB test images was selected for a paired comparison evaluation. The selected images are standard test images with a good range of different properties in terms of level of detail, contrast, and colours. The original images are shown in Figure 12. Seven pairs (STRESS and STREAD) were computed for each original image, with iterations N ∈{300,350,400,450,500,550,600}. All the resulting images are available as supplementary material.

QuickEval [32] was used for setting up and running the experiment in the lab. A gray background was used behind the images and a 200 ms delay was added before going to the next image to avoid the memory effect from the previous stimulus. The experiment was run in a controlled room to make sure that there were no other disturbances during the experiment. The computer screen, an Asus PA32UCG with a resolution of 3840×2160, was calibrated using an i1 Display Pro. The colours for the screen were in sRGB and the luminance 300 cd/m2. The distance from the participants was 70 cm, with the room fully lit. The dynamic ranges of the display and the selected images were not so high that glare was a relevant factor.

The experiment was set up on campus and students from the university were picked for the experiment. The students had different academic backgrounds and genders. A total of 22 observers participated in the experiment. The experiment was conducted in two parts. The first part analysed image preference and the second part analysed image noise. For both parts, the observers were given instructions on what to do. For the image noise part, the observers were first shown an image with chromatic noise, Figure 13, to indicate the kind of noise sought. To make sure only results of comparable computational complexity were compared, the participants were shown only the pair of images that had the same number of iterations. So, e.g., STRESS_300 was compared to STREAD_300. The image pairs that were shown had their placement randomized, to avoid bias toward one of the images.

To assess how well STREAD performs in comparison with the original images, we set up a pair-comparison experiment where the 10 original images (Fig. 12) were compared to STREAD with 300, 450 and 600 iterations. This resulted in 30 pairs for observers to evaluate, shown in random order. Observers were asked to select the image they preferred. The experiment was carried out using QuickEval [32], as an uncontrolled online experiment. A total of 31 observers participated in the experiment.

The raw data from QuickEval for the image preference part of the subjective experiment is shown in Table I. The table contains the number of times the given image was preferred by the participants. The table is set up such that the image on the y-axis is the one selected. For example, for the Alley image at N = 300 iterations, the STRESS image was selected by 6 observers and the stress image STREAD image by 16. The same data is shown graphically in Figure 14.

To analyse the statistical significance of the results from the experiment, a two-sided binomial test with the null hypothesis H0 : p = 1∕2 was conducted. The resulting p-values for the individual comparisons are given in Table II. Using a threshold of p < 0.05, as shown in the coloured cells of Table II, we can see that many of the results are statistically significant, all in favour of STREAD. Even with thresholds of p < 0.01 and p < 0.001, many results are still significant. Even though not all the individual comparisons are statistically significant, it should be noted that there is not one single occurrence of a statistically significant preference of STRESS over STREAD.

Combining all iterations for STREAD and STRESS for each image, the binomial test gives the p-values shown in Table III. Here we see that the p-values are so small that we can easily draw conclusions about which algorithm is best for image preference. Finally, a last binomial test was done on all STREAD versus all STRESS images, resulting in a p-value p = 5.5 × 10−153.

The raw data from QuickEval for the subjective experiment on resulting image noise is shown in Table IV. It is set up similar to Table I, and shows the number of times one image was perceived to be more noisy than the other. The corresponding graphical representation is shown in Figure 15. A brief look at the table shows us that the images created with STRESS are selected by far the most, and in some cases, they are the only one selected.

The trend is even more pronounced than for the image preference, which is also confirmed by the individual p-values for the binomial tests shown in Table VI. Contrary to the image preference, there is no p-value for this test that has a value larger than 0.05. Using a threshold of p < 0.05, the coloured cells, we can infer that the result is statistically significant. However, when performing so many statistical tests from the same data, a Bonferroni correction should be applied, leading to a lower p-value threshold. Even with stricter thresholds of p < 0.01 and p < 0.001, the results are still robust. A binomial test was also conducted on all iterations for STREAD and STRESS for each image, and the results are displayed in Table V. Here, we observe that the p-values are so small that we can confidently conclude which algorithm is superior for image preference. Finally, a binomial test was carried out on all STREAD versus all STRESS images, which yielded a p-value of p = 8.99 × 10−288.

Comparison of STREAD with the original images is shown in Table VII, where the number of times STREAD or the original is preferred is shown. This reveals that overall for the 10 images, STREAD is, according to a binomial test, significantly better than the original (p = 1.73 × 10−7). Conducting the binomial test on 300, 450 and 600 iterations gives a similar conclusion, being statistically significant at p < 0.05 for all p-values are p = 0.0005, p = 0.0007, and p = 0.0354 for 300, 450 and 600 iterations, respectively. We can notice that with higher iterations, the preference towards the original is increasing slightly. Analysis of the images reveals that in 7 images STREAD has a higher number than the original and lower in 3 images (Alley, Sunrise, and White flower). This can indicate that content plays a role.

We evaluated the performance of the suggested STREAD and compared it to STRESS using objective metrics. The Visual-Signal-to-Noise-Ratio (VSNR) [33] is a full-reference metric for the detection of distortions in natural images; this is done by using contrast thresholds and visual masking to determine if the distortion is visible. VSNR is therefore a good metric to compare visible distortions in STREAD and STRESS. Default parameters for VSNR are used, these being Alpha = 0.04 and viewing parameters equal to b = 0, k = 0.02874, g = 2.2, r = 138, v = 27.5, num_levels = 5 and filter_gains =2.̂ (1:num_levels) The second metric is the no-reference Natural Image Quality Evaluator (NIQE) [34], which is a no-reference metric based on analysis of statistical features from natural scene statistics and correlates well with subjective scores on various distortions, including noise [35]. For NIQE, we calculate the results for each colour channel and average them. The last objective metric is a contrast metric, the RSC [36] which is a weighted multilevel contrast metric. For RSC, we have used the optimal parameters from Simone et al. [36]. We calculate the difference in contrast between the contrast enhanced images and the original image, so a higher value will indicate increased contrast and a lower value indicates decreased contrast compared to the original.

The results for VSNR, NIQE, and RSC are shown in Figure 16. Higher VSNR values are better, given in dB in the range 0 - Inf, while for NIQE, lower values are better, where it has been found that values higher than 40 rarely occur [37]. We see that for VSNR, all values for STREAD are higher than STRESS, while for NIQE, all values for STREAD are lower than STRESS. For all images, VSNR indicates that STREAD has less visible distortions compared to STRESS. It can also be seen that the difference between STREAD and STRESS is image-dependent, and consistent with the results from the subjective experiment. For the contrast metric RSC, we notice that increasing the number of iterations produces images with higher contrast compared to the original. The exception is the image Church, where the STREAD algorithm produces very uniform areas without noise, which lowers the local contrast and therefore leads to a lower RSC value. It can also be seen that STRESS produces higher RSC values, which is partly due to the global contrast change because of the colour shift in STRESS.

To supplement the analysis, we have also calculated the difference between the original and the enhanced images. This has been done by taking the normalized sum of the absolute difference between the original and enhanced images for each pixel for both STREAD and STRESS. We visualize the results for the church image with 600 iterations in Figure 17. It can be seen that the higher differences that STRESS produces in the sky, the noise increases. The result for the caps image is shown for 400 iterations in Figure 18. We can notice the same observation regarding noise in the sky for STRESS, and that STREAD, in general, makes fewer changes to the images and treats the image more locally. STRESS increases the edge of the shadows of the caps, while STREAD can also enhance the areas between the shadows.