Back to articles
Regular Articles
Volume: 4 | Article ID: jpi0131
Low-contrast Acuity Under Strong Luminance Dynamics and Potential Benefits of Divisive Display Augmented Reality
  DOI :  10.2352/J.Percept.Imaging.2021.4.1.010501  Published OnlineJanuary 2021

Understanding and predicting outdoor visual performance in augmented reality (AR) requires characterizing and modeling vision under strong luminance dynamics, including luminance differences of 10000-to-1 in a single image (high dynamic range, HDR). Classic models of vision, based on displays with 100-to-1 luminance contrast, have limited ability to generalize to HDR environments. An important question is whether low-contrast visibility, potentially useful for titrating saliency for AR applications, is resilient to saccade-induced strong luminance dynamics. The authors developed an HDR display system with up to 100,000-to-1 contrast and assessed how strong luminance dynamics affect low-contrast visual acuity. They show that, immediately following flashes of 25× or 100× luminance, visual acuity is unaffected at 90% letter Weber contrast and only minimally affected at lower letter contrasts (up to +0.20 LogMAR for 10% contrast). The resilience of low-contrast acuity across luminance changes opens up research on divisive display AR (ddAR) to effectively titrate salience under naturalistic HDR luminance.

Subject Areas :
Views 57
Downloads 23
 articleview.views 57
 articleview.downloads 23
  Cite this article 

Chou P. Hung, Chloe Callahan-Flintoft, Paul D. Fedele, Kim F. Fluitt, Barry D. Vaughan, Anthony J. Walker, Min Wei, "Low-contrast Acuity Under Strong Luminance Dynamics and Potential Benefits of Divisive Display Augmented Realityin Journal of Perceptual Imaging,  2021,  pp 010501-1 - 010501-9,

 Copy citation
  Copyright statement 
Copyright © Society for Imaging Science and Technology 2021
  Article timeline 
  • received October 2019
  • accepted October 2020
  • PublishedJanuary 2021

Preprint submitted to:
Journal of Perceptual Imaging
J. Percept. Imaging
J. Percept. Imaging
Society for Imaging Science and Technology
Augmented Reality (AR) overlays information onto a user’s visual field, to boost human performance and alleviate cognitive load. How to overlay text, icons, or symbols within real visual scenes is a non-trivial problem that must be solved to create a usable interface (Figure 1A) [3, 7, 9, 11, 35, 44, 50]. Our ability to predict human performance in the context of AR and other heads-up displays is limited by fundamental scientific and technical gaps in our ability to display, measure, and model perception of high dynamic range (HDR) visual stimuli [16, 39, 45]. Most of our knowledge about the visual system has been acquired through experiments using computer displays which are typically limited to a max-to-min luminance range of about 100-to-1. By comparison, outdoor environments have much wider variations in luminance, and it is not uncommon for outdoor and mixed indoor/outdoor scenes to have a luminance range of 10,000-to-1 at a single glance [15, 55]. Because of this discrepancy between laboratory experimental conditions and the natural environment, it is difficult to extrapolate classic models of vision to real world scenarios.
Figure 1.
(A) Two examples of mixed indoor/outdoor scenes, where low-contrast text may be displayed against an outdoor luminance exceeding 10,000-to-1 max-to-min luminance ratio. Optimal performance requires that the text be legible after a gaze shift from a brighter area, but that the text not be too salient that it captures attention, distracting from other visual tasks. (B) An ETDRS acuity test with 5-letter sets at high and low luminance contrast. Test stimuli were shown via a commercial off the shelf HDR projector. To maintain steady peak luminance, four 1×1 light anchors are shown at the corners of the screen. (C) Time course of the dynamic luminance acuity test. In each trial, a background flash of 400 cd/m2 for 100 ms is immediately followed by a blank screen at 4 or 16 cd/m2, and 100 ms later, by a set of 5 letters for 100 ms at 90%, 20%, 15%, or 10% Weber contrast. The sequence of flash and letter presentation is repeated at 1 Hz until the subject verbally reports the 5 letters.
Choosing appropriate display parameters is important for optimizing overall visual performance, e.g., for aided target recognition (AiTR) in which a target is highlighted to aid detection. Visual performance depends on automatic bottom-up mechanisms of saliency [17, 25, 48], top-down task-driven mechanisms in which abstract representations compete for attentional resources [18, 36, 54], and temporal autocorrelation of spatially localized signals [5]. As a result, in the context of real-world AR performance, high contrast and temporally discontinuous displays have the potential to capture attention and reduce the availability of attentional resources across other parts of the scene, delaying the warfighter’s ability to perceive and react [31, 32].
An obvious approach to avoid attentional capture in AiTR is to titrate salience by lowering the contrast of the target highlight. Both additive displays (bright letters or icons superimposed on the background) and divisive displays (e.g., dark letters, in which the transmitted luminance is divided via variable occlusion) can produce low contrast. Current AR devices have additive displays which are optimized for indoor gaming environments, but they have limited luminance range, becoming invisible on sunny days, and limited ability to display low contrast relative to the background scene. A divisive display AR (ddAR) would have several potential advantages for outdoor tasks including increasing the effective dynamic range, efficiently matching AR to the scene for low-contrast display, ease of controlling salience against complex backgrounds, and potential to reduce size, weight, and power requirements.
The potential of low-contrast highlighting to improve overall AiTR performance raises a possible concern under real-world luminance. Gaze shifts in visual search often induce strong 10- to 100-fold luminance changes in the retinal image. Could a low-contrast target highlight be used to titrate saliency under such strong luminance dynamics, or would the low-contrast visibility be drowned out?
Previous reports on contrast sensitivity thresholds found that, in the static condition, the log of contrast sensitivity is predicted by a linear function of spatial frequency, retinal eccentricity, and adapting luminance, i.e., a “pyramid of visibility” [52]. There is also a mostly linear dependence of contrast sensitivity on temporal frequency and retinal eccentricity, consistent with the distribution of parvocellular versus magnocellular neurons in central versus peripheral visual fields, and these relationships hold across a wide range of static luminances [53].
With dynamic stimuli, it is known that contrast thresholds are elevated, e.g., to 20% contrast after a strong 100% contrast forward masking stimulus, consistent with a rapid decrease in response gain and contrast sensitivity at very low spatial frequencies [4]. However, a 100% versus 20% contrast difference is only a 5-fold change in stimulus strength, without a change in average luminance or spatial frequency, and to our knowledge the issue of how strong luminance dynamics affects low-contrast acuity has not been addressed. Given the strong temporal coupling between luminance-sensitive (very low spatial frequency sensitive) neurons and orientation-selective (high spatial frequency sensitive) neurons in early visual cortex [23, 24], it is reasonable to ask whether mechanisms for low-contrast spatial acuity are resilient to strong luminance dynamics.
We investigated the effect of strong luminance dynamics on low-contrast acuity by examining letter acuity, because letters and shapes (Gabors) engage similar neural filtering mechanisms in early visual cortex [45]. We tested how 25× and 100× flashes in luminance affect LogMAR letter acuity across letter Weber contrasts of 10%, 15%, 20%, and 90%. By characterizing how the visual system performs across large and small luminance changes, we will be better able to design AR systems which effectively titrate saliency for overall visual search performance.
Fifteen subjects (11 male) 18 to 70 years old participated in the experiment. Potential subjects were excluded if they self-reported that they, their parents, or their siblings had photosensitive epilepsy, or that they previously had head trauma or other disorders thought to be associated with excitatory/inhibitory balance (epilepsy, schizophrenia, autism, depression, attention deficit hyperactivity disorder) [26], atypical brain development, or used mind-altering drugs in the past week. Potential subjects were also screened via the Canadian Longitudinal Study on Aging – Epilepsy Algorithm “CLSA-EA” [27]. All experiments were conducted in the MIND lab at the Army Research Laboratory at Aberdeen Proving Ground, MD, according to a protocol approved by the Army’s Human Research Protection Program.
Vision Screening
Prior to beginning experimental tasks, subjects were screened for normal or corrected-to-normal (at least 20/40) visual acuity and normal color vision via a Titmus i500 Vision Screener [34].
HDR Display
All images were projected from a JVC DLA-RS600U 4K Reference Projector (software version u83.2, PS version 100310) and displayed biocularly on an HD projection screen [21]. The projector was positioned just above and behind the seated subject, and the test was conducted in a darkened room (0.000598 cd/m2).
Images spanned 1920 × 1080 pixels in resolution (48.7 × 27.3 cm w × h) and were observed from a chinrest-stabilized viewing distance of 78 cm, thus spanning 34.7×19.9 viewing angle with pixel size 0.0181×0.0184.
Gaze and pupil size were tracked monocularly via an infrared eye tracker (EyeLink 1000 Plus), synchronized via Lab Streaming Layer software (Swartz Center for Computational Neuroscience, UCSD) [30]. To maintain a constant peak luminance in the visual field, all tasks included static 400 cd/m2 “light anchors” [14] sized 1×1 at the four corners of the screen. In two subjects, we repeated these experiments without the light anchors and observed the same pattern of results (see Discussion).
Images were displayed at 60 Hz and pseudo 11 bits (10.7 bits, i.e., 11 bits red, 11 bits green, but only 10 bits blue, because all the color information needs to fit into 32 bits) precision via a framebuffer procedure using Psychtoolbox 3.0 [28] for GNU/Linux X11 software (version 3.0.14 – Build date: May 8, 2017) running under MATLAB 64-bit version 2016b on Ubuntu 16.04 (seen by Psychtoolbox as Linux version 4.4.0-31-generic). We used the AMD FirePro W8100 graphics card and applied an 11-bit grayscale Gamma correction by measuring the luminance via spectrophotometer (Photo Research PR-745) at over 75 luminance indices and applying log-linear interpolation. The resulting linearized Gamma spanned a range of 636.4 (u, v = 0.1953, 0.3199; x, y = 0.3200, 0.3494, 6037K) to 0.006055 cd/m2, for a maximum contrast ratio of over 100,000:1 in a single uniform image. We used these measurements to set the luminances of the light anchors, the flash, and the text background.
An advantage of our display system is that it allowed us to present smaller letters, by avoiding potential spatial misalignment and inhomogeneity of stacked LCD projection displays [42]. Nevertheless, our projection system has some spatial inhomogeneity primarily due to light scatter, limiting the maximum contrast ratio within a single image depending on distance and average display luminance (ADL, percent of black screen occupied by white squares) [21]. By sampling the luminance of 1, 2, and 4 deg. black disks against a sparsely checkered background (Figure 2A), we measured effective maximum contrast ratios of 5690:1 at 1% ADL and 1590:1 at 5% ADL, corresponding approximately to the 4 and 16 cd/m2 background conditions tested here. Our MATLAB code for generating ADL test images is available at We calibrated our text contrast by taking digital photos at the same aperture setting across multiple exposure durations ranging from 1/60 to 1 sec (CANON 5D Mark IV), then using MATLAB’s “makehdr” function (tone mapping off) and referencing to spectrophotometer measurements of the background. We measured letter luminances by averaging the luminances of letter pixels selected using MATLAB’s “imfreehand/createMask” function (Fig. 2B). Across most letter sizes including 0.5 LogMAR, letter luminances were within 2% of the requested value. At 0.3 LogMAR, letter contrast was reduced to 90% of the requested value, e.g., the contrast of the “90% contrast” letters was actually 81.5% contrast. At 0.1 and −0.1 LogMAR, letter contrasts were reduced to 80% and 66% of the requested value, respectively.
Figure 2.
(A) A test image used to measure effective contrast ratio as a function of average display luminance. A 2 deg. black disk is overlaid on an array of white squares at 1% density (1% ADL). (B) Measurement of effective letter luminance by combining digital photos at multiple exposure durations into a single HDR image, then segmenting the letter versus the background. Background luminance is measured by photometer, then letter luminance is inferred from the letter-to-background ratio in the combined image.
Dynamic Luminance Visual Acuity Task
We measured visual acuity based on the ETDRS LogMAR chart, which improves performance consistency compared to the Snellen chart via proportional letter spacing (consistent crowding effects), improved font, and improved balancing for letter set difficulty [2, 12]. We modified a MATLAB version of the ETDRS test [46] by adding dynamic luminance and adjustable letter contrast. An example of the letter display is shown in Fig. 1B. For each trial, a set of 5 dark letters was presented, pseudorandomly chosen from a predetermined pool of 5-letter sets that were balanced for difficulty. A block comprised 20 trials, in decreasing letter sizes from 50 arcmin down to 3.125 arcmin.
We considered whether our smallest letter size of 3.125 arcmin might be too small for line (or pixel) thickness of 1.1 arcmin. Carkeet and Lister [6] described a maximum pixel size, in minutes of angle, needed to accurately assess finer visual acuity. This information is critical to accurately assessing visual performance at the threshold of perception. They also showed that a slightly larger pixel size can be extended to accurately assess visual acuity if anti-alias filtering is used to average across neighboring pixels. They reported that using a pixel size 0.35 times the smallest minutes of arc used in testing will ensure that pixelation does not compromise acuity measurements. They also showed that when pixels are smoothed by one pixel width in the vertical and horizontal directions, a larger pixel size can be used: 0.65 times the width of the smallest minutes of arc applied in testing. Our applied pixel size (0.0181×0.0184) is at the unfiltered pixel limit (3.125arcmin ×0.35=0.018) and below the filtered pixel limit (3.125arcmin ×0.65=0.034).
In one block, letters were shown at nominally 90% Weber contrast (1.3 cd/m2 letters at the larger letter sizes) against a 16.2 cd/m2 background. On six blocks, letters were presented against a uniform background of 4.1 or 16.2 cd/m2, at Weber contrasts of 20%, 15%, and 10% for each background (e.g., for the 16.2 cd/m2 background, the letters were 13.1, 13.8, and 14.5 cd/m2; for the 4.1 cd/m2 background, the letters were 3.30, 3.44, and 3.65 cd/m2). Figure 3 shows the approximate appearance of these test conditions based on sRGB gamma. We note that these letter contrasts could be produced at 8-bit luminance, making the test accessible to a wider community. Each set of 5 letters was presented for 0.1 s. On seven additional blocks, the set of letters was preceded by a 400 cd/m2 full screen flash (i.e., 25 or 100 times the background luminance) of duration 0.1 s, with an interstimulus interval of 0.1 s (Fig. 1C). The sequence of background flash and letters was repeated at 1 Hz until the subject read the letters aloud and the experimenter recorded the number of correct letters (0 to 5) via keyboard entry. The flashed blocks were presented first, in order of decreasing background luminance and letter Weber contrast, followed by the non-flashed blocks in the same sequence. The entire test lasted 20 to 30 minutes.
Figure 3.
Approximate appearance of ETDRS test letters. To be more visible in print, nominal backgrounds of 16 cd/m2 and 4 cd/m2 are shown here as 32 cd/m2 and 8 cd/m2, based on sRGB gamma for a 100 cd/m2 display.
LogMAR acuity was calculated via the 2-letter acuity method, to improve consistency across subjects. As is commonly defined, 0.0 LogMAR corresponds to 20/20 Snellen acuity. Each line in which 3 or more letters were correctly reported (2 or fewer letters were incorrectly reported) was counted as all correct, i.e., −0.10 LogMAR. Starting after the last “all correct” line, each correct letter in the next line was counted as −0.02 LogMAR (Figure 4). Subjects were told to “try hard to guess each letter” and were informed that the scoring was based on the number of correct letters, without penalty for incorrect guesses.
Figure 4.
Calculation of LogMAR acuity for subject ME based on the 2-letter method, awarding −0.1 LogMAR for correct lines (at least 3 of 5 letters correct, black squares) and −0.02 LogMAR for each correct letter in the subsequent line (black circles). In this example, the resulting acuities are 0.06 LogMAR without flash and −0.02 LogMAR with flash, for 90% letter contrast.
The tests were conducted in two batches separated by one year. We previously reported preliminary results from the first batch, in which we tested nine subjects (six male) without the 15% letter contrast level [22]. This report includes additional results from a second batch tested at all contrast levels, in which we retested five of these subjects and recruited six additional subjects. Because the test and retest results were similar for the five retested subjects, we report only their retest results.
We recently described the setup of a projection display system with over 100,000-to-1 luminance contrast ratio and pseudo 11-bit depth (2048 shades of gray), enabling the study of visual perception under HDR luminance . Using this system, we showed that the apparent orientation of a central target, a contrast mixture of two orthogonal Gabors, depends on the conjunction of the luminances and orientations in the surrounding context and that the effect was specific to HDR luminance [1921]. Here, used this display system to investigate how luminance dynamics affect visual letter acuity (Fig. 1B, 1C). We extended a previously developed computerized ETDRS acuity test with static luminance [46] by adding strong luminance dynamics. Specifically, we tested the effect of bright (25× to 100× luminance) flashes on the acuity of dark letters at different background and letter luminance contrasts, as might be induced by shifting gaze from a brighter area of a scene to a darker area with a low-contrast target highlight. We tested a total of 14 blocks of varying background and letter contrasts, 7 with and 7 without a preceding luminance flash.
The resulting acuity measurements are shown in Figure 5. The vertical axis shows acuity expressed as the Logarithm of the Minimum-Angle-of-Resolution in minutes of arc (LogMAR), and as the equivalent Snellen ratio, which shows the resolution of the test participant’s vision at 20 feet (numerator value) compared to the distance at which a person with normal vision would have the same line resolution ability (denominator). With normal vision, a Snellen score of 20/20, the minimum angle of resolution is 1 arc minute, corresponding to a LogMAR score of zero.
Figure 5.
Dependence of visual acuity on letter Weber contrast, background luminance, and background flash. LogMAR acuity was calculated based on the 2-letter method, awarding −0.1 LogMAR for correct lines (at least 3 of 5 letters correct) and −0.02 LogMAR for each correct letter in the subsequent line. Open squares and black lines indicate means across 15 subjects (4 subjects marked by were not tested at 15% contrast). Significance of background flash on LogMAR acuity is based on two tailed paired t test, uncorrected for multiple comparisons.
As expected, acuity generally decreased as letter contrast and background luminance decreased. At 90% letter Weber contrast and 16 cd/m2 background luminance, adding a 400 cd/m2 background flash (a 25× flash) before presenting the set of 5 letters resulted in a slight improvement in acuity from 0.0453 to −0.0027 LogMAR, a difference of −0.048 LogMAR (CI −0.01 to −0.08, p = 0.013, N = 15 subjects). This slight increase may be related to a previously reported slight increase in contrast sensitivity at low temporal frequencies under strong retinal illuminance [53].
The same 400 cd/m2 background flash had negligible effect on letter acuity at a lower text contrast of 20%. At 16 cd/m2 background luminance, acuity was 0.3493 LogMAR without flash and 0.3320 LogMAR with flash, an average difference of −0.017 LogMAR (CI −0.07 to 0.03, p = n.s., N = 15 subjects). At 4 cd/m2 letter background (a 100× flash), acuity was 0.3680 without the flash and 0.4187 with the flash, a difference of +0.051 LogMAR (CI −0.02 to 0.12, p =  n.s., N = 15 subjects).
At 15% letter contrast with 16 cd/m2 background, adding a 400 cd/m2 background flash weakly but significantly decreased acuity from 0.458 to 0.511 LogMAR, a difference of 0.053 LogMAR (CI 0.01 to 0.10, p = 0.033, N = 11 subjects). At the same letter contrast but 4 cd/m2 letter background, the 400 cd/m2 background flash decreased acuity from 0.43 to 0.55 LogMAR, a difference of 0.12 LogMAR (CI 0.08 to 0.16 LogMAR, p = 0.000049, N = 11 subjects).
At further reduced letter contrast of 10% with 16 cd/m2 letter background, adding the background flash weakly but significantly decreased acuity from 0.47 to 0.63 LogMAR, a difference of 0.16 LogMAR (CI 0.11 to 0.21, p = 6.9 × 10−6). At the same letter contrast but 4 cd/m2 letter background, the background flash decreased acuity from 0.57 to 0.77 LogMAR (from 3.7 to 5.9 arcmin), a difference of 0.20 LogMAR (CI 0.15 to 0.25 LogMAR, p = 5.9 × 10−7). For comparison, LogMAR values of 0.4 and 0.8 correspond to Snellen ratios of 20/50 and 20/125.
Reproducibility of Test Results
Our results were generally consistent across the 15 subjects, with weak flash-induced decreases in acuity at 15% and 10% contrast for nearly all subjects (one subject had an improvement in acuity at 15% contrast, 16 cd/m2 background). At 20% contrast, there was no average change in acuity, but about a quarter of subjects had slight increases in acuity, and a quarter to half of the subjects had decreases in acuity, depending on the background luminance. At 90% contrast, only 5 of 15 subjects showed ∼0.1 LogMAR improvement in acuity with flashing, and the remainder were unchanged. Overall, this suggests that the number of subjects is not the limiting factor to the significance or reproducibility of these results for average changes in acuity. However, individuals vary slightly in baseline visual acuity and in flash-induced changes in acuity, in conditions where low contrast is not the limiting factor.
Figure 6.
Dependence of acuity on presence of light anchors. Acuity with anchors (solid lines) and without anchors (dashed lines) are shown for subjects CH (maroon) and AH (gold). Acuity with anchors, mean across all 15 subjects, is shown as gray.
The main experiment was conducted with static 400 cd/m2 “light anchors” [14] sized 1×1 at the four corners of the screen. We retested two subjects without the light anchors to determine whether their presence affected these results (Fig. 6). We did not observe any consistent differences for the presence versus absence of the light anchors, except that at the most challenging condition with 10% letter Weber contrast, the absence of the light anchors may have weakened the decrease in acuity after the background flash. However, the differences were within the range of results across the 15 subjects. We interpret these results to mean that removing the light anchors did not increase the effect of the background flash. Conversely, adding the light anchors may have worsened the effect of the background flash in the most challenging condition, possibly due to light scatter within the eye, but had negligible effect otherwise. Overall, this suggests that low-contrast acuity is mostly unaffected by the scene’s highest luminance under these conditions, but it may be worthwhile to investigate in future studies whether this holds true for the most challenging conditions.
Generalizability to Visual Search
The generalizability of these results is limited by several factors. First, letter discrimination against a blank background and (mostly) steady fixation is quite different from shape discrimination against a naturalistic background during visual search. Although letters and shapes are thought to engage similar mechanisms in early visual cortex [45], it would be worthwhile to reinvestigate this issue with an AR device in a shape discrimination and search task. In a related study with the same HDR projection display [1921], we reported that strong flashes can affect shape discrimination (detecting the orientation of overlapping Gabors against a background of Gabors overlaid on a wide 100-to-1 luminance range of patches) consistent with grouping by orientation and luminance, hinting that strong luminance dynamics engage early visual mechanisms such as horizontal fibers in V1 for contextual pre-attentive processing. We have also examined additive versus transparent text against an HDR natural scene, and the preliminary results are consistent with this report in suggesting that the text visibility declines from 70% to 30% contrast [22].
Several reports have examined the visibility of transparent text against uniform and artificial textured backgrounds, including backgrounds with naturalistic 1/f power [3], and found that text contrast can be a good predictor of search times and reading speed [3, 44]. Our results are consistent with a previous report that reading speed is well preserved down to 15% contrast against a uniform background [33]. Reading speed is decreased for complex backgrounds tested with root-mean-square (RMS) contrast up to 0.27 [44] and for cases where the text-to-background contrast ratio was below 1.6-to-1 [3]. However, we note that retinal luminance dynamics during reading differ from large-saccade-induced strong retinal luminance dynamics during HDR visual search. Besides textured backgrounds, we also did not test stronger luminance flashes, darker backgrounds, or examine timing effects such as adaptation, stimulus duration, or latency. Performance may differ under such conditions, for example, due to differences in crowding or other factors at mesopic versus photopic luminance [40, 47, 56].
Our test range of over 100× luminance captures the key transition between indoor and outdoor illumination, and our brief 100 msec flash and 100 msec letter presentations approximates the saccade frequency of naturalistic visual search (4 saccades per second) while avoiding higher cognitive effects. Our results suggest that lower contrast may not necessarily mean poor visibility when viewed in HDR luminance environments.
Another limitation is that in our task, the letter positions remained fixed while the luminance changed, whereas such luminance changes typically occur across gaze shifts in a static scene. An important difference is that, in gaze shifts, the visual system has pre-saccadic information about the luminances and patterns at the target post-saccade [13, 38]. Also, because contrast sensitivity depends on retinal eccentricity, spatial frequency [52], and temporal dynamics [29, 49, 51], titration of salience via contrast would need to be via gaze-contingent display. It is comforting that the interaction of these factors appear to be mostly linear, at least at static luminance and standard dynamic range, and that the temporal contrast sensitivity appears to be constant across eccentricity when corrected for cortical magnification [43]. However, it is possible that foveal and peripheral temporal dynamics differ markedly under strong luminance dynamics [49]. It would be worthwhile to investigate under controlled free-viewing, where the subject shifts his gaze from bright to dark regions in a static HDR image.
Lastly, in our test scenario, the flash and the letters are on the same viewing surface, which may not be the case in AR, especially for indoor environments and mixed indoor/outdoor environments such as cockpits. Binocular disparity is thought to aid in depth perception to at least 18 m [1]. Notably, adding a slight defocus to the background was reported to improve reading rate for transparent text, suggesting a possible way to mitigate the luminance limitations of additive displays [3]. Thus, the generalizability of our results may be limited to outdoor uses, where both the display and the environment are effectively at infinite distance (over 100 m, typical for dismounted warriors), and it would be worthwhile to investigate how low-contrast visibility and defocusing interact for search performance in HDR scenes.
Potential Implications for Heads-Up Displays and Augmented Reality
Augmented reality is a promising tool to improve performance by synergizing human and machine capabilities. Current AR devices are descendants of VR technology, optimized for indoor settings and gaming. For AR to augment performance in outdoor settings, the limitations of AR display must be addressed.
One limitation of current AR displays is luminance dynamic range (up to about 1000 cd/m2) compared to outdoor scenes (0.001 cd/m2 at dusk to 108 cd/m2 during the day). This is because current AR devices have additive displays (bright icons) that are limited in their light output. Increasing the luminance range would consume exponentially more power. Conversely, a divisive display (transparency-reducing icons) would automatically match the background luminance range without requiring complex or powerful processing.
Another limitation of current AR is that, unlike indoor gaming experience, outdoor tasks such as AiTR require titration of visual salience so that labeled icons do not capture attention, reducing the availability of attentional resources across other parts of the scene, e.g., to detect unhighlighted targets that were missed by computer vision algorithms. Whereas additive displays are too salient at low scene luminance, invisible at high scene luminance, and would require complex calculations to titrate transparency and salience relative to the scene, a divisive display would automatically provide fine control of salience, because the icons’ luminance is relative to its background.
These potential benefits of divisive display AR (ddAR) motivated our investigation of the effect of luminance dynamics on low-contrast letter acuity. We were particularly interested in whether low-contrast letters would remain visible after a bright flash, because such luminance dynamics on the retinal image frequently occur when scanning a naturalistic scene (i.e., a luminance change or “flash” resulting from the gaze shifting from light-to-dark areas or across a bright area, not because of a physical flash in the environment). Our results suggest that visual acuity is resilient to large gaze-shift-induced luminance dynamics on the retina, indicating that icons such as target highlights may not need to be of high contrast to be visible during the visual search of outdoor and mixed indoor/outdoor scenes with HDR luminance.
Implications for Low Vision
The ability to adapt to changing light levels via pupillary constriction (the pupillary light reflex) declines with age [10] and is a predictor of eye disease [37]. Reports on brightness perception, luminance normalization, and contextual mechanisms hint that visual cortex may be able to partially compensate for this decline [8, 23, 41]. Our results show that visual acuity to low-contrast letters is mostly unaffected by strong flashes, for our subject pool ranging in age from 18 to 70 years old. We did not detect noticeable age-related differences in flash-induced changes in acuity, but our approach suggests a way to investigate this systematically in a larger and more elderly population.
We showed that low-contrast visual acuity is resilient to large changes in luminance, such that flashing has negligible impact on letter acuity within 200 ms immediately following the flash, for flashes up to 25× the background luminance and letter Weber contrast above 20%. At our most challenging condition of flashes 100× the background luminance and letter contrast of 10%, flashing induced a mild acuity loss of 0.2 LogMAR, i.e., a 59% increase in the minimum angle of resolution from 3.7 to 5.9 arcmin. The resilience of low-contrast acuity to strong luminance dynamics opens new directions for research at the intersection of salience, attention, and performance in real world scenes. These results advance our capability to develop effective AR displays for real-world luminance dynamics.
We acknowledge the generous support of Dr. Nigel Davies and Mario Kleiner, who provided advice and code for the LogMAR and HDR display, respectively. Research was sponsored by the Army Research Laboratory and was accomplished under CAST 076910227001, ARL-74A-HR53, and ORAU cooperative agreement W911NF-16-2-0008. The views and conclusions contained in this document are those of the authors and should not be interpreted as representing the official policies, either expressed or implied, of the Army Research Laboratory or U.S. Government. The U.S. Government is authorized to reproduce and distribute reprints for Government purposes notwithstanding any copyright notation herein.
1AllisonR. S.GillamB. J.VecellioE.2009Binocular depth discrimination and estimation beyond interaction spaceJ. Vis.91010.1167/9.1.10
2BaileyI. L.Lovie-KitchinJ. E.2013Visual acuity testing. From the laboratory to the clinicVis. Res.90210.1016/j.visres.2013.05.004
3Blanc-GoldhammerD. R.MacKenzieK. J.2018The effects of natural scene statistics on text readability in additive displaysProc. Human Factors and Ergonomics Society Annual Meeting128112851281–5SAGE PublicationsSage, CA
4BurrD. C.MorroneM. C.RossJ.1994Selective suppression of the magnocellular visual pathway during saccadic eye movementsNature37151110.1038/371511a0
5Callahan-FlintoftC.HolcombeA. O.WybleB.2020A delay in sampling information from temporally autocorrelated visual stimuliNature Commun.111111–1110.1038/s41467-020-15675-1
6CarkeetA.ListerL. J.2018Computer monitor pixelation and Sloan letter visual acuity measurementOphthalmic Physiol. Opt.3814410.1111/opo.12434
7CarneyT.KleinS. A.TylerC. W.SilversteinA. D.BeutterB.LeviD.WatsonA. B.ReevesA. J.NorciaA. M.ChenC.-C.Development of an image/threshold database for designing and testing human vision modelsHuman Vision and Electronic Imaging IV1999International Society for Optics and PhotonicsBellingham, WA542551542–51
8ChenC.-C.TylerC. W.2002Lateral modulation of contrast discrimination: Flanker orientation effectsJ. Vis.2810.1167/2.6.8
9CornelisK.PollefeysM.VergauwenM.Van GoolL.2000Augmented reality using uncalibrated video sequencesEuropean Workshop on 3D Structure from Multiple Images of Large-Scale Environments144160144–60SpringerNew York, NY
10DumbletonK.GuillonM.TheodoratosP.WooleyC. B.MoodyK.2018The effects of age and refraction on pupil size and visual acuity: Implications for multifocal contact lens design and fittingContact Lens and Anterior Eye41S2310.1016/j.clae.2018.04.153
11EdstromJ.ChenD.WangJ.GuH.VazquezE. A.McCourtM. E.GongN.Luminance-adaptive smart video storage system2016 IEEE Int’l. Symposium on Circuits and Systems (ISCAS)2016IEEEPiscataway, NJ734737734–7
12ElliottD. B.2016The good (logMAR), the bad (Snellen) and the ugly (BCVA, number of letters read) of visual acuity measurementOphthalmic Physiol. Opt.3635510.1111/opo.12310
13FreemanJ.SimoncelliE. P.2011Metamers of the ventral streamNat. Neurosci.14119510.1038/nn.2889
14GilchristA.KossyfidisC.BonatoF.AgostiniT.CataliottiJ.LiX.SpeharB.AnnanV.EconomouE.1999An anchoring theory of lightness perceptionPsychol. Rev.10679510.1037/0033-295X.106.4.795
15GrahamD. J.2011Visual perception: Lightness in a high-dynamic-range worldCurrent Biol.21R91410.1016/j.cub.2011.10.003
16GutsteinW.SinclairS. H.PrestiP.NorthR. V.2015Interactive thresholding of central acuity under contrast and luminance conditions mimicking real world environments: 1, evaluation against LogMAR chartsJ. Comput. Sci. Syst. Biol.822510.4172/jcsb.1000210
17HarrisonA.GreenM.HungC.RaglinA.ioM-HDR: A High Dynamic Range adapted model of visual salienceSBP-BRiMS: Int’l. Conf. on Social Computing, Behavioral-Cultural Modeling and Prediction and Behavior Representation in Modeling and Simulation2019SpringerWashington, DC171–7
18HungC. P.CuiD.ChenY.-p.LinC.-p.LevineM.“Correlated activity supports efficient cortical processing,” Front Comput. Neurosci.8, 171 (2014)
19HungC. P.Callahan-FlintoftC.FedeleP. D.FluittK. F.OdoemeneO.WalkerA. J.HarrisonA. V.VaughanB. D.JaswaM. S.WeiM.“An 11-bit High Dynamic Range (HDR) luminance display and its use to discover contextual mechanisms of real-world luminance normalization for visual acuity and target discrimination,” ARL Technical Report, ARL-TR-8744, 1–44 (2019)
20HungC. P.Callahan-FlintoftC.FedeleP. D.FluittK. F.OdoemeneO.WalkerA. J.HarrisonA. V.VaughanB. D.JaswaM. S.WeiM.2020Abrupt darkening under high dynamic range (HDR) luminance invokes facilitation for high contrast targets and grouping by luminance similarityJ. Vis.20910.1167/jov.20.7.9
21HungC. P.Callahan-FlintoftC.WalkerA. J.FedeleP. D.FluittK. F.OdoemeneO.HarrisonA. V.VaughanB. D.JaswaM. S.WeiM.2020A 100,000-to-1 high dynamic range (HDR) luminance display for investigating visual perception under real-world luminance dynamicsJ. Neurosci. Methods33810868410.1016/j.jneumeth.2020.108684
22HungC. P.LarkinG. B.GeussM. N.Callahan-FlintoftC.FedeleP. D.FluittK. F.VaughanB.WeiM.2020Divisive display augmented reality (ddAR) for real-world warfighter performanceSPIE Defense + Commercial Sensing, sub-conference Virtual, Augmented, and Mixed Reality (XR) Technology for Multi-Domain Operations 11426-191131–13
23HungC. P.RamsdenB. M.RoeA. W.2007A functional circuitry for edge-induced brightness perceptionNat. Neurosci.10118510.1038/nn1948
24HungC. P.RamsdenB. M.RoeA. W.Inherent Biases in Spontaneous Cortical Dynamics2010Oxford University PressOxford
25IttiL.KochC.NieburE.1998A model of saliency-based visual attention for rapid scene analysisIEEE T Pattern Anal.20125410.1109/34.730558
26JiangX.BollichA.CoxP.HyderE.JamesJ.GowaniS. A.HadjikhaniN.BlanzV.ManoachD. S.BartonJ. J. S.GaillardW. D.RiesenhuberM.2013A quantitative link between face discrimination deficits and neuronal selectivity for faces in autismNeuroImage: Clin.232010.1016/j.nicl.2013.02.002
27KeezerM. R.PelletierA.StechysinB.VeilleuxM.JettéN.WolfsonC.2014The diagnostic test accuracy of a screening questionnaire and algorithm in the identification of adults with epilepsyEpilepsia55176310.1111/epi.12805
28KleinerM.BrainardD.PelliD.2007What’s new in Psychtoolbox-3?Perception 36 ECVP Abstract Supplement
29KoenderinkJ. J.BoumanM. MesquitaA. E. B.SlappendelS.1978Perimetry of contrast detection thresholds of moving spatial sine wave patterns. II. The far peripheral visual field (eccentricity 050)J. Opt. Soc. Am.6885010.1364/JOSA.68.000850
30KotheC.Lab streaming layer (LSL), Accessed on October 26, 2015
31LarkinG. B.GeussM.YuA.RexwinkleJ.Callahan-FlintoftC.BakdashJ.SwobodaJ.LiebermanG.HungC.MooreS.“Emerging recommendations for initial automated target recognition (ATR) display considerations: Implications for Concept Development and Evaluation,” PsyArXiv 1–8 (2019)
32LarkinG. B.GeussM.YuA.RexwinkleJ.Callahan-FlintoftC.BakdashJ. Z.SwobodaJ.LiebermanG.HungC. P.MooreS.LanceB.2020Augmented target recognition display recommendationsDSIAC J.728
33LeggeG. E.RubinG. S.LuebkerA.1987Psychophysics of reading—V. The role of contrast in normal visionVis. Res.27116510.1016/0042-6989(87)90028-9
34Lipin-Dietz, Titmus i500 Vision Screener,, accessed 2019-1-1
35McDermottK. C.MalkocG.MulliganJ. B.WebsterM. A.2010Adaptation and visual salienceJ. Vis.101710.1167/10.13.17
36MoranJ.DesimoneR.1985Selective attention gates visual processing in the extrastriate cortexScience22978210.1126/science.4023713
37MüllerS.HeerenT. F.BonelliR.FruttigerM.IssaP. C.EganC. A.HolzF. G.2019Contrast sensitivity and visual acuity under low light conditions in macular telangiectasia type 2Br. J. Ophthalmology10339810.1136/bjophthalmol-2017-311785
38NakamuraK.ColbyC. L.2002Updating of the visual representation in monkey striate and extrastriate cortex during saccadesProc. Natl. Acad. Sci. USA99402610.1073/pnas.052379899
39NormarkC. J.GärlingA.“Assessment of automotive visual display guidelines and principles: A Literature review,” Des. J. 14, 446 (2011)
40PluháčekF.SiderovJ.2018Mesopic visual acuity is less crowdedGraefe’s Archive for Clinical and Experimental Ophthalmology256173910.1007/s00417-018-4017-6
41RadonjicA.AllredS. R.GilchristA. L.BrainardD. H.2011The dynamic range of human lightness perceptionCurr. Biol.21193110.1016/j.cub.2011.10.013
42RadonjicA.BroussardC.BrainardD.“Characterizing and controlling the spectral output of an HDR display,” UPenn Brainard Lab Technical Report 1 (2011)
43RovamoJ.MustonenJ.NäsänenR.1995Neural modulation transfer function of the human visual system at various eccentricitiesVis. Res.3576710.1016/0042-6989(94)00171-H
44ScharffL. F.AhumadaA. J.2002Predicting the readability of transparent textJ. Vis.2710.1167/2.9.7
45SolomonJ. A.PelliD. G.1994The visual filter mediating letter identificationNature36939510.1038/369395a0
46StewartC. E.HusseyA.DaviesN.MoseleyM. J.2006Comparison of logMAR ETDRS chart and a new computerised staircased procedure for assessment of the visual acuity of childrenOphthalmic Physiol. Opt.2659710.1111/j.1475-1313.2006.00407.x
47ThompsonW. B.LeggeG. E.KerstenD. J.ShakespeareR. A.LeiQ.2017Simulating visibility under reduced acuity and contrast sensitivityJ. Opt. Soc. Am. A3458310.1364/JOSAA.34.000583
48TreismanA. M.GeladeG.1980A feature-integration theory of attentionCogn Psychol.129710.1016/0010-0285(80)90005-5
49TylerC. W.HamerR. D.1990Analysis of visual modulation sensitivity. IV. Validity of the Ferry–Porter lawJ. Opt. Soc. Am. A774310.1364/JOSAA.7.000743
50VaughanB. D.“Soldier-in-the-loop target acquisition performance prediction through 2001: integration of perceptual and cognitive models,” Army Research Laboratory Technical Report, ARL-TR-3833 (2006), pp. 1–124
51VirsuV.RovamoJ.LaurinenP.NäsänenR.1982Temporal contrast sensitivity and cortical magnificationVis. Res.22121110.1016/0042-6989(82)90087-6
52WatsonA. B.2018The field of view, the field of resolution, and the field of contrast sensitivityJ. Perceptual Imaging11050510.2352/J.Percept.Imaging.2018.1.1.010505
53WatsonA. B.AhumadaA. J.2016The pyramid of visibilityElectron. Imaging20161
54WilkeM.LogothetisN. K.LeopoldD. A.2003Generalized flash suppression of salient visual targetsNeuron39104310.1016/j.neuron.2003.08.003
55XiaoF.DiCarloJ. M.CatrysseP. B.WandellB. A.2002High dynamic range imaging of natural scenesProc. IS&T/SID CIC10: Tenth Color Imaging Conf.337342337–42IS&TSpringfield, VA
56YazdanieM.AlvarezJ.AgrónE.WongW. T.WileyH. E.Ferris IIIF. L.ChewE. Y.CukrasC.2017Decreased visual function scores on a low luminance questionnaire is associated with impaired dark adaptationOphthalmology124133210.1016/j.ophtha.2017.05.005