The surface of planet Mars has been studied for years by geologists seeking evidence of possible ancient life and to improve our understanding of crustal processes and climates variations. With this aim, Mars is steadily observed by orbiters developed by numerous space agencies worldwide: NASA, ESA, ROSCOSMOS, ISRO, UAESA, and CNSA, leading to recurring rover mission developments in order to perform closer surveys of different landing sites. Viking was the first spacecraft to land on Mars in 1975 and to provide color images of the planet’s landscapes. The following ten years have seen the development of rover missions (able to move on the ground) such as Sojourner in 1996, followed by Spirit and Opportunity (Mars Exploration Rovers - MER) in 2003, Curiosity (Mars Science Laboratory - MSL) in 2011, and Perseverance in 2020, all providing images of the martian ground while visiting their landing sites surroundings. Although on Mars’ surface, these rovers cannot provide an unlimited amount of scientific images. This is related to bandwidth limitations for interplanetary missions on one hand, and power availability of the rover on the other. Therefore any full-resolution image sent to Earth has to contain as much information as possible and the zone pictured has to be carefully selected for effective decision-making. In practice, this is performed through context images allowing planning for future (following days) navigation tasks toward zones of interest. In this context, it is crucial to show images that are as practical and meaningful as possible, and most of all, to ensure correct interpretations. This requirement expressed by geologists raises the need for color correction of images taken on Mars and motivates the need in investigating methods of (i) measuring the in-situ illuminant, and (ii) estimating how the Human Visual System (HVS) behaves in such an extra-terrestrial environment. This paper investigates partially the HVS using the results on the martian illuminant published in [1].

Images from cameras whose sensitivity functions are centered around 450 nm, 550 nm, and 600 nm - usually referred to as RGB cameras - may be color-corrected to become approximately colorimetric i.e. with pixel values that are sensor-independent. Such values are expressed in a standard color space such as CIEXYZ [2] not related to the spectral sensitivities of the camera anymore, rather related to standard Color Matching Functions (CMFs) such as those of the CIE1931 observer [3]. In practice, this conversion from sensor-dependent to sensor-independent color spaces is not perfect, as the Luther-Ives condition is not fulfilled [4]. The advantage of colorimetric images is the power of comparison they offer [5]. For Mars images applications, it means any image from any camera could be compared to one another thus bringing invariant information on the colors of any object of the scenes that are imaged. Reaching colorimetric images requires an in-situ colorchecker with a large number of patches (or excellent knowledge of the camera spectral sensitivities) and the spectral power distribution of the illuminant of the scene. Then, as soon as basic colorimetric - as defined in [6] - values are recovered from the images, the question of displaying them occurs and so the question of advanced colorimetry - as defined in [6] - and in particular chromatic adaption. It has been shown through an illumination discrimination task (IDT) in [7] that discrimination of illumination changes is less effective along the daylight locus than along an atypical locus, and in particular for bluer illumination changes. Assuming small discrimination indicates full adaptation from the HVS, while larger discrimination indicates HVS does not fully adapt, it demonstrates that surface color constancy is best for blue daylight illumination, and suggests chromatic adaptation occurs more easily for such stimuli. It brings us to wonder what insights this type of experiment could bring on chromatic adaptation on Mars.

Perseverance rover carries twenty-three cameras (https://mars.nasa.gov/mars2020/spacecraft/rover/cameras/) including five on Supercam mast. The Remote Micro Imager (RMI) is one on them, and is equipped with a calibration target [8] placed at the rear of the Perseverance’s deck. Figures 1 and 2 show images of two specific sols i.e martian days, respectively sol 77 and sol 478. For each, three types of images are shown: PDS color images i.e as published by NASA (https://planetarydata.jpl.nasa.gov/img/data/mars2020/mars2020_imgops/data_rmi_imgops/) [9], Basic colorimetric images i.e without chromatic adaption performed, and Advanced colorimetric images i.e with chromatic adaptation (Bradford [10]) performed. Basic and Advanced terms refer to [6] concepts now CIE vocabulary [2].

The basic colorimetric correction is applied through a Color Correction Matrix (CCM) computed as the minimum least square regression between the X, Y, Z tristimulus values and the R,G,B pixels values. The X, Y, Z tristimulus values were computed for the CIE1931 standard observer, M54, the martian illuminant from [1], and the 24 reflectances spectrum of the ColorChecker Classic. The R, G, B raw values were computed rather than taken from the image because no ColorChecker is available in situ - aside from 3 patches shown figs. 1 and 2. The same illuminant and reflectances as X, Y, Z computation were used, with the end-to-end spectral sensitivities of the camera system. After the CCM is applied to the raw image, and though no correction for chromatic adaptation is done, the XYZ to sRGB matrix optimized for D65 standard illuminant is applied to the image. This deviation from the usual way of doing is for illustration purposes only, as chromatic adaptation for Mars is the main aspect and unknown parameter of this study. Comparing PDS color images and Basic colorimetric image on figs. 1 and 2 first shows the relative high variations in colorimetry when the weather changes on Mars. Sol 77 is known as a clear day whereas sol 478 was a stormy day. These weather data are not yet published but eight-year climatology of dust optical depth on Mars was studied and published [11] showing how fast the variations of dust in the atmosphere may occur and alter the optical depth and so in-situ illumination. This methodology alone, poses several difficulties when color correction has to be performed on Mars images. First, the illuminant is varying fast and its spectral power distribution is not known at all time. Second, the end-to-end spectral sensitivities of the imaging system might be partially unknown as often measured with low resolution for some parts. Third, no color target is available except for the three red, green, blue patches, known to be insufficient for color correction computation [12].

The advanced colorimetric correction on both Figs. 1 and 2 uses the same CCM for basic colorimetric correction with an additional step performing chromatic adaption “as on earth”. A Bradford adaptation [10] is applied with D65 as adapted white. It allows the visualization of colorimetric images assuming that the HVS performs chromatic adaptation the same way under daylight and under solight. This last uncertainty is another barrier when geologists require faithful images of Mars: there is no study concerning color perception on Mars. Note Bradford adaptation was applied with the same amplitude on images from sol 77 and sol 478.

Figure 3 shows the D55 daylight standardized spectrum as well as two illuminants on Mars locus [1]: M46 (CCT 4600K, considerd as bad weather illuminant) and M54 (CCT 5400K, considered as clear sky illuminant). It highlights spectral differences between daylight and solight. Comparing all three types of images on Fig. 1 also shows how color-corrected images allow to better render real colors: the dust around the patches appears more yellowish brown (ISCC-NBS designation) as measured through multispectral radiometrically calibrated images by [13].

The aim of this study is to approach color constancy/inconstancy under Mars light compared to Earth light and therefore bring the first stone toward faithful color images for martian geologists. In this domain, the question is fundamental concerning the goal of the correction to apply: should we display Mars images in the way a human standing on Mars would see the scene, and therefore account for chromatic adaptation, or should we provide color constant images i.e with no dependency on the reflected light? An Illumination Discrimination Task (IDT) was therefore conducted. This method, first introduced by Pearce et al. [7], is approaching color constancy/inconstancy in measuring global lightning variations discrimination on a scene showing objects with invariant reflectances. The level of notice of the difference (i.e. discrimination), as a consequence of some HVS mechanisms, is described in the concept of color appearance and gives useful information on the occurrence or not of chromatic adaptation when Mars illumination is part of the viewing conditions of a scene.

Our study compares stimuli along the daylights locus to stimuli along the solight locus. The following four sections detail the multiprimary lighting system that is used, followed by the stimuli under study, and finally the observers who participated in the experiment.

Figures 7 and 8 (a figure per booth condition as titled) show individual results of the illumination discrimination task i.e. the accuracy averaged on ten repetitions at each stimulus tested. Red curves are for results along solight locus, whereas blue curves are for results along the daylight locus. In these plots, high accuracy shows the ease of the observers to discriminate the tested illuminants, whereas low accuracy shows the difficulty of discrimination. Fig. 7 shows thirteen individual results as two observers didn’t manage to perform the second part of the study. Observer #15 experienced a failure of the automatized stimuli generating system while performing the solight/ColorChecker task so the results were simply withdrawn as seen in Fig. 7.

Figure 9 show global results averaged over the observers with error bars. The same color coding is kept: blue lines for IDTs along daylight locus, red lines for IDTs along solight locus. Both booth conditions are differentiated by dotted lines (with ColorChecker) and full lines (no ColorChecker). Due to the low number of observers in the dataset, the standard error on the sample variance needs to be accounted for in the error bars. For a centered distribution of standard deviation σ, its value is given by V ar(V ar(X)) = 2∗σ4 : (n − 1) where n is the number of samples (> 0). In our case, this accounts for up to ∼10% of the computed value, which which could be withdrawn from the standard deviations on Fig. 9. The global (averaged over observers) results for conditions no ColorChecker/daylight and ColorChecker/solight almost perfectly overlap. The results for no ColorChecker/solight are slightly more accurate, while the results for ColorChecker/daylight show lower accuracy and therefore lower illumination discrimination ability. It conversely demonstrates that chromatic adaptation is better optimized for the ColorChecker/daylight condition with small adaptation difference for the solight locus tested.

Our results agree with [7] pointing out that the discrimination of illumination is less effective along the daylight locus than along an atypical (orthogonal) locus. They especially show that illumination discrimination variations follow the trend of the most common lights i.e. bluish, then yellowish, reddish, and finally (less common) greenish. As they demonstrate that the Human Visual System (HVS) is adapted to typical environments, our study shows that the observers didn’t experience an important difference in illumination discrimination being under Earth’s bluish illuminant or under Mars slightly reddish illuminant. This is true for the reduced loci used from [1] in our study. Mars illuminants used in our experiment were developed from a Principal Component Analysis (PCA) based on fifty measures on Mars through radiometrically calibrated multispectral images of two rovers (Spirit and Opportunity) [1]. To stay in the range of the Mars illuminants measured data, it was decided to focus the study on a reduced locus. An extension may be considered in forthcoming investigations currently at the stage of collection of more illuminant SPDs from all RAD-calibrated images available for all rovers on Mars.

Our overall results, considering a reduced locus, demonstrate that solight was perceived in a similar way as daylight. Indeed, the observers clearly stated that the illuminant displayed as the martian was “less red than expected”. Through a quantitative study whose results are shown in Figure 10, they were nearly 70% to answer above (strictly) 6 to the question “Between 1 and 10, how red did you think Mars light was before you did the experiment?”. They were nearly 85% to answer below (strictly) 7 to the question “How red did you perceive the stimuli along solight (Mars) locus ?”.

Overlapped JNDs in solight locus (see Fig. 4) do not look to affect the results as the first four stimuli concerned (CCT 5400K to 5100K) show accuracies higher than for the ColorChecker/Daylight condition.

The expertise in color sciences of most of the observers may be part of the explanation for the result observed for the ColorChecker/Daylight condition. The ColorChecker Classic is an object very often stared at by such scientists, making this condition even more typical. Together with the large error bars, this suggests performing the experiment with more observers and with different expertise. However a sample of fifteen observers is already larger than several other IDTs repo in the literature [7, 17].

Similar results in illumination discrimination suggest comparable HVS abilities to chromatic adaptation on Mars and on Earth. Therefore the same processes might be applied to Mars and Earth images i.e. bringing colors toward a rendering under D65 illuminant (roughly, beyond the scope of this paper). This could bring geologists closer to what they would see if standing on Mars. However, more aspects of color appearance [18] should be studied such as the impact of the chromaticity of the scene [17] as Mars landscapes show an homogeneous distribution of reflectance in the visible spectrum [19].

The illumination discrimination ability of the Human Visual System (HVS) proved to be optimized for typical environments in an earlier study [7]. The study conducted in this paper reports similar outcomes while showing small differences between results under daylight and solight. It suggests that some of the Mars illuminants tested here are experienced as typical by the observers as they show similar ability to discriminate illuminations along both loci. This implies, for the reduced part of the loci tested, that Mars images might be corrected from chromatic adaptation in a similar way to what is done for Earth scene images. However further data is under collection and study in order to determine the extent of the Mars illuminants; from blue sunsets to dust storms. Pending on future results on this topic, another illumination discrimination task with more non-expert observers and stimuli more sparsed on a longer loci is perhaps required. Assuming the crucial need for color-corrected images for geologists of Mars exploration missions, and taking into account the data transmission limitations in space imaging, Spectral Filter Arrays (SFA) stand as very promising solutions [20, 21].