Fifteen subjects (11 male) 18 to 70 years old participated in the experiment. Potential subjects were excluded if they self-reported that they, their parents, or their siblings had photosensitive epilepsy, or that they previously had head trauma or other disorders thought to be associated with excitatory/inhibitory balance (epilepsy, schizophrenia, autism, depression, attention deficit hyperactivity disorder) [26], atypical brain development, or used mind-altering drugs in the past week. Potential subjects were also screened via the Canadian Longitudinal Study on Aging – Epilepsy Algorithm “CLSA-EA” [27]. All experiments were conducted in the MIND lab at the Army Research Laboratory at Aberdeen Proving Ground, MD, according to a protocol approved by the Army’s Human Research Protection Program.

Prior to beginning experimental tasks, subjects were screened for normal or corrected-to-normal (at least 20/40) visual acuity and normal color vision via a Titmus i500 Vision Screener [34].

All images were projected from a JVC DLA-RS600U 4K Reference Projector (software version u83.2, PS version 100310) and displayed biocularly on an HD projection screen [21]. The projector was positioned just above and behind the seated subject, and the test was conducted in a darkened room (0.000598 cd/m2).

Images spanned 1920 × 1080 pixels in resolution (48.7 × 27.3 cm w × h) and were observed from a chinrest-stabilized viewing distance of 78 cm, thus spanning 34.7∘×19.9∘ viewing angle with pixel size 0.0181∘×0.0184∘.

Gaze and pupil size were tracked monocularly via an infrared eye tracker (EyeLink 1000 Plus), synchronized via Lab Streaming Layer software (Swartz Center for Computational Neuroscience, UCSD) [30]. To maintain a constant peak luminance in the visual field, all tasks included static 400 cd/m2 “light anchors” [14] sized 1∘×1∘ at the four corners of the screen. In two subjects, we repeated these experiments without the light anchors and observed the same pattern of results (see Discussion).

Images were displayed at 60 Hz and pseudo 11 bits (10.7 bits, i.e., 11 bits red, 11 bits green, but only 10 bits blue, because all the color information needs to fit into 32 bits) precision via a framebuffer procedure using Psychtoolbox 3.0 [28] for GNU/Linux X11 software (version 3.0.14 – Build date: May 8, 2017) running under MATLAB 64-bit version 2016b on Ubuntu 16.04 (seen by Psychtoolbox as Linux version 4.4.0-31-generic). We used the AMD FirePro W8100 graphics card and applied an 11-bit grayscale Gamma correction by measuring the luminance via spectrophotometer (Photo Research PR-745) at over 75 luminance indices and applying log-linear interpolation. The resulting linearized Gamma spanned a range of 636.4 (u, v = 0.1953, 0.3199; x, y = 0.3200, 0.3494, 6037K) to 0.006055 cd/m2, for a maximum contrast ratio of over 100,000:1 in a single uniform image. We used these measurements to set the luminances of the light anchors, the flash, and the text background.

An advantage of our display system is that it allowed us to present smaller letters, by avoiding potential spatial misalignment and inhomogeneity of stacked LCD projection displays [42]. Nevertheless, our projection system has some spatial inhomogeneity primarily due to light scatter, limiting the maximum contrast ratio within a single image depending on distance and average display luminance (ADL, percent of black screen occupied by white squares) [21]. By sampling the luminance of 1, 2, and 4 deg. black disks against a sparsely checkered background (Figure 2A), we measured effective maximum contrast ratios of 5690:1 at 1% ADL and 1590:1 at 5% ADL, corresponding approximately to the 4 and 16 cd/m2 background conditions tested here. Our MATLAB code for generating ADL test images is available at https://github.com/USArmyResearchLab/ARL-Display-Metrics-and-Average-Display-Luminance. We calibrated our text contrast by taking digital photos at the same aperture setting across multiple exposure durations ranging from 1/60 to 1 sec (CANON 5D Mark IV), then using MATLAB’s “makehdr” function (tone mapping off) and referencing to spectrophotometer measurements of the background. We measured letter luminances by averaging the luminances of letter pixels selected using MATLAB’s “imfreehand/createMask” function (Fig. 2B). Across most letter sizes including 0.5 LogMAR, letter luminances were within 2% of the requested value. At 0.3 LogMAR, letter contrast was reduced to 90% of the requested value, e.g., the contrast of the “90% contrast” letters was actually 81.5% contrast. At 0.1 and −0.1 LogMAR, letter contrasts were reduced to 80% and 66% of the requested value, respectively.

We measured visual acuity based on the ETDRS LogMAR chart, which improves performance consistency compared to the Snellen chart via proportional letter spacing (consistent crowding effects), improved font, and improved balancing for letter set difficulty [2, 12]. We modified a MATLAB version of the ETDRS test [46] by adding dynamic luminance and adjustable letter contrast. An example of the letter display is shown in Fig. 1B. For each trial, a set of 5 dark letters was presented, pseudorandomly chosen from a predetermined pool of 5-letter sets that were balanced for difficulty. A block comprised 20 trials, in decreasing letter sizes from 50 arcmin down to 3.125 arcmin.

We considered whether our smallest letter size of 3.125 arcmin might be too small for line (or pixel) thickness of 1.1 arcmin. Carkeet and Lister [6] described a maximum pixel size, in minutes of angle, needed to accurately assess finer visual acuity. This information is critical to accurately assessing visual performance at the threshold of perception. They also showed that a slightly larger pixel size can be extended to accurately assess visual acuity if anti-alias filtering is used to average across neighboring pixels. They reported that using a pixel size 0.35 times the smallest minutes of arc used in testing will ensure that pixelation does not compromise acuity measurements. They also showed that when pixels are smoothed by one pixel width in the vertical and horizontal directions, a larger pixel size can be used: 0.65 times the width of the smallest minutes of arc applied in testing. Our applied pixel size (0.0181∘×0.0184∘) is at the unfiltered pixel limit (3.125arcmin ×0.35=0.018∘) and below the filtered pixel limit (3.125arcmin ×0.65=0.034∘).

In one block, letters were shown at nominally 90% Weber contrast (1.3 cd/m2 letters at the larger letter sizes) against a 16.2 cd/m2 background. On six blocks, letters were presented against a uniform background of 4.1 or 16.2 cd/m2, at Weber contrasts of 20%, 15%, and 10% for each background (e.g., for the 16.2 cd/m2 background, the letters were 13.1, 13.8, and 14.5 cd/m2; for the 4.1 cd/m2 background, the letters were 3.30, 3.44, and 3.65 cd/m2). Figure 3 shows the approximate appearance of these test conditions based on sRGB gamma. We note that these letter contrasts could be produced at 8-bit luminance, making the test accessible to a wider community. Each set of 5 letters was presented for 0.1 s. On seven additional blocks, the set of letters was preceded by a 400 cd/m2 full screen flash (i.e., 25 or 100 times the background luminance) of duration 0.1 s, with an interstimulus interval of 0.1 s (Fig. 1C). The sequence of background flash and letters was repeated at 1 Hz until the subject read the letters aloud and the experimenter recorded the number of correct letters (0 to 5) via keyboard entry. The flashed blocks were presented first, in order of decreasing background luminance and letter Weber contrast, followed by the non-flashed blocks in the same sequence. The entire test lasted 20 to 30 minutes.

LogMAR acuity was calculated via the 2-letter acuity method, to improve consistency across subjects. As is commonly defined, 0.0 LogMAR corresponds to 20/20 Snellen acuity. Each line in which 3 or more letters were correctly reported (2 or fewer letters were incorrectly reported) was counted as all correct, i.e., −0.10 LogMAR. Starting after the last “all correct” line, each correct letter in the next line was counted as −0.02 LogMAR (Figure 4). Subjects were told to “try hard to guess each letter” and were informed that the scoring was based on the number of correct letters, without penalty for incorrect guesses.

The tests were conducted in two batches separated by one year. We previously reported preliminary results from the first batch, in which we tested nine subjects (six male) without the 15% letter contrast level [22]. This report includes additional results from a second batch tested at all contrast levels, in which we retested five of these subjects and recruited six additional subjects. Because the test and retest results were similar for the five retested subjects, we report only their retest results.

Our results were generally consistent across the 15 subjects, with weak flash-induced decreases in acuity at 15% and 10% contrast for nearly all subjects (one subject had an improvement in acuity at 15% contrast, 16 cd/m2 background). At 20% contrast, there was no average change in acuity, but about a quarter of subjects had slight increases in acuity, and a quarter to half of the subjects had decreases in acuity, depending on the background luminance. At 90% contrast, only 5 of 15 subjects showed ∼0.1 LogMAR improvement in acuity with flashing, and the remainder were unchanged. Overall, this suggests that the number of subjects is not the limiting factor to the significance or reproducibility of these results for average changes in acuity. However, individuals vary slightly in baseline visual acuity and in flash-induced changes in acuity, in conditions where low contrast is not the limiting factor.

The main experiment was conducted with static 400 cd/m2 “light anchors” [14] sized 1∘×1∘ at the four corners of the screen. We retested two subjects without the light anchors to determine whether their presence affected these results (Fig. 6). We did not observe any consistent differences for the presence versus absence of the light anchors, except that at the most challenging condition with 10% letter Weber contrast, the absence of the light anchors may have weakened the decrease in acuity after the background flash. However, the differences were within the range of results across the 15 subjects. We interpret these results to mean that removing the light anchors did not increase the effect of the background flash. Conversely, adding the light anchors may have worsened the effect of the background flash in the most challenging condition, possibly due to light scatter within the eye, but had negligible effect otherwise. Overall, this suggests that low-contrast acuity is mostly unaffected by the scene’s highest luminance under these conditions, but it may be worthwhile to investigate in future studies whether this holds true for the most challenging conditions.

The generalizability of these results is limited by several factors. First, letter discrimination against a blank background and (mostly) steady fixation is quite different from shape discrimination against a naturalistic background during visual search. Although letters and shapes are thought to engage similar mechanisms in early visual cortex [45], it would be worthwhile to reinvestigate this issue with an AR device in a shape discrimination and search task. In a related study with the same HDR projection display [19–21], we reported that strong flashes can affect shape discrimination (detecting the orientation of overlapping Gabors against a background of Gabors overlaid on a wide 100-to-1 luminance range of patches) consistent with grouping by orientation and luminance, hinting that strong luminance dynamics engage early visual mechanisms such as horizontal fibers in V1 for contextual pre-attentive processing. We have also examined additive versus transparent text against an HDR natural scene, and the preliminary results are consistent with this report in suggesting that the text visibility declines from 70% to 30% contrast [22].

Several reports have examined the visibility of transparent text against uniform and artificial textured backgrounds, including backgrounds with naturalistic 1/f power [3], and found that text contrast can be a good predictor of search times and reading speed [3, 44]. Our results are consistent with a previous report that reading speed is well preserved down to 15% contrast against a uniform background [33]. Reading speed is decreased for complex backgrounds tested with root-mean-square (RMS) contrast up to 0.27 [44] and for cases where the text-to-background contrast ratio was below 1.6-to-1 [3]. However, we note that retinal luminance dynamics during reading differ from large-saccade-induced strong retinal luminance dynamics during HDR visual search. Besides textured backgrounds, we also did not test stronger luminance flashes, darker backgrounds, or examine timing effects such as adaptation, stimulus duration, or latency. Performance may differ under such conditions, for example, due to differences in crowding or other factors at mesopic versus photopic luminance [40, 47, 56].

Our test range of over 100× luminance captures the key transition between indoor and outdoor illumination, and our brief 100 msec flash and 100 msec letter presentations approximates the saccade frequency of naturalistic visual search (4 saccades per second) while avoiding higher cognitive effects. Our results suggest that lower contrast may not necessarily mean poor visibility when viewed in HDR luminance environments.

Another limitation is that in our task, the letter positions remained fixed while the luminance changed, whereas such luminance changes typically occur across gaze shifts in a static scene. An important difference is that, in gaze shifts, the visual system has pre-saccadic information about the luminances and patterns at the target post-saccade [13, 38]. Also, because contrast sensitivity depends on retinal eccentricity, spatial frequency [52], and temporal dynamics [29, 49, 51], titration of salience via contrast would need to be via gaze-contingent display. It is comforting that the interaction of these factors appear to be mostly linear, at least at static luminance and standard dynamic range, and that the temporal contrast sensitivity appears to be constant across eccentricity when corrected for cortical magnification [43]. However, it is possible that foveal and peripheral temporal dynamics differ markedly under strong luminance dynamics [49]. It would be worthwhile to investigate under controlled free-viewing, where the subject shifts his gaze from bright to dark regions in a static HDR image.

Lastly, in our test scenario, the flash and the letters are on the same viewing surface, which may not be the case in AR, especially for indoor environments and mixed indoor/outdoor environments such as cockpits. Binocular disparity is thought to aid in depth perception to at least 18 m [1]. Notably, adding a slight defocus to the background was reported to improve reading rate for transparent text, suggesting a possible way to mitigate the luminance limitations of additive displays [3]. Thus, the generalizability of our results may be limited to outdoor uses, where both the display and the environment are effectively at infinite distance (over 100 m, typical for dismounted warriors), and it would be worthwhile to investigate how low-contrast visibility and defocusing interact for search performance in HDR scenes.

Augmented reality is a promising tool to improve performance by synergizing human and machine capabilities. Current AR devices are descendants of VR technology, optimized for indoor settings and gaming. For AR to augment performance in outdoor settings, the limitations of AR display must be addressed.

One limitation of current AR displays is luminance dynamic range (up to about 1000 cd/m2) compared to outdoor scenes (0.001 cd/m2 at dusk to 108 cd/m2 during the day). This is because current AR devices have additive displays (bright icons) that are limited in their light output. Increasing the luminance range would consume exponentially more power. Conversely, a divisive display (transparency-reducing icons) would automatically match the background luminance range without requiring complex or powerful processing.

Another limitation of current AR is that, unlike indoor gaming experience, outdoor tasks such as AiTR require titration of visual salience so that labeled icons do not capture attention, reducing the availability of attentional resources across other parts of the scene, e.g., to detect unhighlighted targets that were missed by computer vision algorithms. Whereas additive displays are too salient at low scene luminance, invisible at high scene luminance, and would require complex calculations to titrate transparency and salience relative to the scene, a divisive display would automatically provide fine control of salience, because the icons’ luminance is relative to its background.

These potential benefits of divisive display AR (ddAR) motivated our investigation of the effect of luminance dynamics on low-contrast letter acuity. We were particularly interested in whether low-contrast letters would remain visible after a bright flash, because such luminance dynamics on the retinal image frequently occur when scanning a naturalistic scene (i.e., a luminance change or “flash” resulting from the gaze shifting from light-to-dark areas or across a bright area, not because of a physical flash in the environment). Our results suggest that visual acuity is resilient to large gaze-shift-induced luminance dynamics on the retina, indicating that icons such as target highlights may not need to be of high contrast to be visible during the visual search of outdoor and mixed indoor/outdoor scenes with HDR luminance.

The ability to adapt to changing light levels via pupillary constriction (the pupillary light reflex) declines with age [10] and is a predictor of eye disease [37]. Reports on brightness perception, luminance normalization, and contextual mechanisms hint that visual cortex may be able to partially compensate for this decline [8, 23, 41]. Our results show that visual acuity to low-contrast letters is mostly unaffected by strong flashes, for our subject pool ranging in age from 18 to 70 years old. We did not detect noticeable age-related differences in flash-induced changes in acuity, but our approach suggests a way to investigate this systematically in a larger and more elderly population.