Back to articles

Papers Presented at London Imaging Meeting: Material Appearance 2023

Volume: 67 | Article ID: 030405

Digital Pre-Distorted One-Step Phase Retrieval Algorithm for Real-Time Hologram Generation for Holographic Displays

DOI : **10.2352/J.ImagingSci.Technol.2023.67.3.030405** Published Online : **May 2023**

Abstract

In a computer-generated holographic projection system, the image is reconstructed via the diffraction of light from a spatial light modulator. In this process, several factors could contribute to non-linearities between the reconstructed and the target image. This paper evaluates the non-linearity of the overall holographic projection system experimentally, using binary phase holograms computed using the one-step phase retrieval (OSPR) algorithm, and then applies a digital pre-distortion (DPD) method to correct for the non-linearity. Both a notable increase in reconstruction quality and a significant reduction in mean squared error were observed, proving the effectiveness of the proposed DPD-OSPR algorithm.

Journal Title : **Journal of Imaging Science and Technology**

Publisher Name : **Society for Imaging Science and Technology**

Subject Areas :

Views 153

Downloads 17

Cite this article

Jinze Sha, Adam Goldney, Andrew Kadis, Jana Skirnewskaja, Timothy D. Wilkinson, "Digital Pre-Distorted One-Step Phase Retrieval Algorithm for Real-Time Hologram Generation for Holographic Displays" in *Journal of Imaging Science and Technology*, 2023, pp 030405-1 - 030405-7, https://doi.org/10.2352/J.ImagingSci.Technol.2023.67.3.030405

Article timeline

- received January 2023
- accepted April 2023
- PublishedMay 2023

1.

Introduction

In a computer-generated holographic projection system, images are generated via the controlled diffraction of coherent light, which is modulated by a spatial light modulator (SLM). Contemporary SLM’s can only modulate either phase or amplitude, hence algorithms are needed to compute amplitude-only or phase-only holograms. Among those, phase-only holograms are generally preferred due to inherently higher energy efficiency as there is no intentional blockage of light during modulation. Classic phase retrieval algorithms include direct binary search [1], simulated annealing [2], Gerchberg-Saxton [3], and recent years have also seen numerical optimization methods [4–8], but these methods are iterative so computation speed is the major challenge. Previous studies have demonstrated a real-time computer-generated holography (CGH) method called one-step phase retrieval (OSPR) [9], which was fast enough for real-time holography, but the reconstruction quality still has potential for improvement. Hence, a computationally inexpensive method is needed to improve the reconstruction quality whilst maintaining the real-time property of the OSPR algorithm.

There are several factors contributing to the non-linearities between the reconstruction of hologram and the target image, including the calculation and quantization of the hologram, modulation of the light and imperfections in the optical setup. This article proposes the digital pre-distorted one-step phase retrieval (DPD-OSPR) algorithm. The digital pre-distortion (DPD) is carried out on the holographic projection system using holograms computed by the OSPR algorithm by measuring the non-linearity experimentally and applying the corresponding pre-distortion curve on target images. DPD can be done via a one-to-one correction curve or a lookup table (LUT) which allows the relationship between the input and output to be adjusted without any heavy computation.

The intuition of the proposed DPD-OPSR algorithm for CGH comes from the gamma correction method for conventional displays, such as cathode-ray tube (CRT) monitor [10], plasma display panel television (PDP-TV) [11] and thin-film-transistor liquid-crystal display (TFT-LCD) [12, 13]. Gamma correction for conventional displays were originally developed to mimic the perceptual response of human vision [14]. The work presented here is a logical continuation of this approach applied to holographic displays.

2.

Method

The DPD-OSPR method builds on the OSPR algorithm [9]. The OSPR algorithm relies on the time multiplexing of holograms, exploiting the response time of eyes in order to reduce noise in the replay field, rather than computational optimization of the hologram. As the random noises are averaged by the eye while the target image stays, the perceived noise is lessened by the temporal average detected by the eye [15].

The OSPR algorithm is described in Algorithm 1, where the propagation function $\mathcal{P}$ is simply the Discrete Fourier Transform (DFT) for Fraunhofer diffraction [16]. The number of subframes S is set to 24 in this study. The OSPR algorithm generates each hologram subframe by taking the inverse Discrete Fourier Transform (iDFT) on the field concatenating the target image with a random phase, and then the phase-only constraint is applied by discarding the amplitude while keeping the phase. After computing the S subframes, they are then displayed on the SLM sequentially. For real-world SLM with limited bit depth, a further step of quantization on each subframe is needed.

3.

Experimental setup

The holographic projector used in this experiment is a Fourier projection system developed by Freeman [17], as shown in Figure 1. The design consists of a diode-pumped solid-state (DPSS) 532 nm 50 mW laser source, focussed down by an aspheric singlet (A), the focus of which becomes the diffraction limited point source (B) for the projector. The beam then passes through a polarizing beam splitter cube (C) to a collimating lens (D), which illuminates the SLM (E). The SLM is a binary phase SXGA-R2 ForthDD ferroelectric Liquid crystal on silicon (LCOS) micro-display with a refresh rate of 1440 Hz, a pixel pitch of 13.6 μm and a resolution of 1280 × 1024. An aperture at point (F) spatially filters out the other orders, leaving only one first order, which is then magnified up by a finite conjugate lens group (G) to produce an image, of the required size, on the screen (H) [17].

Figure 1.

Figure 2.

The mechanical components are listed in Figure 2 with parts numbered. The holograms displayed on the SLM are generated using the OSPR algorithm [9], and as the SLM is a binary-phase modulator, each hologram subframe needs to be binary quantized. Then each group of the 24 binary-phase hologram subframes are encoded as the 8-bit red, green, blue (RGB) channels of a 24-bit image to interface with the SLM driver electronics. The SLM displays each bit plane sequentially, with ones and zeros mapping to opposing phase modulations at each pixel. The reconstructions were captured using a Canon 550D camera with an EFS 18–55 mm lens. To ensure fair comparison, the camera was set to the same manual setting when comparing each pair of replay fields before and after DPD. It takes 24∕1440 = 1∕60 s to display all 24 subframes on a 1440 Hz SLM, so the camera shutter speed was set to 1∕30 s to capture all frames twice. The images captured are raw format in RGB colour, which are subsequently converted to grey-scale (using the rgb2gray function in Matlab [18]) when calculating normalized mean squared error (NMSE) against the target images, using equation $\text{NMSE}=\frac{\frac{1}{n}\ast \sum {\left(x-\widehat{x}\right)}^{2}}{\sum {\left(x\right)}^{2}}$, where x is the target, $\widehat{x}$ is the measured output and n is the dimension of x and $\widehat{x}$.

4.

Determining the DPD curve

To determine the DPD curve of the holographic projection system, the non-linearity needs to be measured first. The hologram in Figure 3(b) was first generated using OSPR algorithm for the linear grey-scale ramp of pixel value increasing linearly from 0 to 255, as shown in Fig. 3(a), along with a single pixel white (255) strip at the left end as a fiducial marker to demonstrate the beginning of the grey-scale region [15].

Figure 3.

The projection output of the linear grey-scale ramp was then captured and cropped as shown in Fig. 3(c), from which the non-linearity curve was determined, by averaging each column of pixels in the image and discarding the fiducial marker, forming the blue line in Fig. 3(d). A third-order polynomial fit was applied, generating a smoothed non-linearity curve (yellow line in Fig. 3(d)).

A high degree of non-linearity is exhibited. By taking the mean of the square of the error between the measured output (blue line) and the linear reference (green dashed line), the normalized mean squared error (NMSE) of the measured output was calculated to be 0.0858. To correct for the non-linearity, the DPD curve (red line) was formed by inverting the smoothed non-linearity curve (yellow line) in Fig. 3(d).

Figure 4.

Subsequently, the DPD curve (red line in Fig. 3(d)) was used to adjust the grey-scale ramp, achieving the pre-distorted grey-scale ramp as shown in Figure 4(a). The corresponding projection output was then captured as shown in Fig. 4(c). By using the same method of averaging columns of pixels, the measured output was plotted in Fig. 4(d). It can be seen that the corrected non-linearity was much closer to linear comparing to the original non-linearity, and the NMSE was calculated to be 0.0039.

Table I.

NMSE | Percentage | |
---|---|---|

Before DPD | 0.0858 | 100% |

After DPD | 0.0039 | 4.55% |

Thus, as demonstrated in Table I, DPD achieved a 95.45% reduction in MSE, which was a significant improvement in non-linearity, therefore the DPD curve measured is validated.

5.

Applying the DPD Curve

To qualitatively demonstrate the effectiveness of our approach, we project a simple test pattern of a graduated ramp test pattern consisting of 10-step strips in Figure 5(a), which is commonly employed in gamma-correction calibration of many display systems. As shown in the projection replay field captured in Fig. 5(c), before DPD, the right few strips are barely distinguishable. In comparison, after carrying out DPD, it can be seen that each pair of adjacent strips in Fig. 5(f) are much more distinguishable, qualitatively showing the effectiveness of the DPD method.

Figure 5.

Figure 6.

Then the DPD curve was applied to the two sample images as shown in Figure 6 (a) and (b), producing pre-distorted images in Fig. 6 (c) and (d). Holograms were generated for each image using the OSPR algorithm and loaded onto the SLM respectively. The corresponding replay fields were captured as shown in Figure 7.

Figure 7.

The replay fields of the holographic projection of original images are shown in Fig. 7 (a) and (b), and the replay fields of the holographic projection of images after DPD are shown in Fig. 7 (c) and (d).

As shown in Fig. 7(a), it can be seen that, before DPD, the edges between the buildings and the sky were quite ambiguous, with most detail of the sky being lost. In comparison, after DPD, the replay field in Fig. 7(c) provided not only sharper edges between buildings and the sky, but also more detail of clouds in the sky. The NMSE of the replay field for sample image 1 decreased from 0.06139 to 0.04920, which was a 19.86% reduction.

In Fig. 7(b), before DPD, the horse was difficult to distinguish from the background, especially around the horse’s back area. But after DPD, as shown in Fig. 7(d), contrast has been significantly boosted and the fine detail around this part of the horse is more evident. The NMSE of the replay field for sample image 2 decreased from 0.04309 to 0.03635, which was a 15.64% reduction.

Table II.

Sample image 1 | NMSE | Percentage |
---|---|---|

Before DPD | 0.06139 | 100% |

After DPD | 0.04920 | 80.15% |

Sample image 2 | MSE | Percentage |

Before DPD | 0.04309 | 100% |

After DPD | 0.03635 | 84.36% |

Hence, as summarised in Table II, DPD achieved a 19.86% reduction in NMSE for sample image 1 and a 15.64% reduction in NMSE for sample image 2, quantitatively proving the effectiveness of DPD method for CGH of real-world test images using OSPR algorithm.

Finally, as the DPD is a one-to-one mapping, the computation time is negligible. In practice, the computational overhead is too small to be measured against randomness between subsequent runs. DPD can also be further accelerated in hardware using a hardware LUT, so that the DPD can be carried out instantly. This approach is widely adopted in gamma correction for displays.

6.

Conclusion

The non-linearity between target image and reconstructed image was measured for the overall holographic projection system by projecting a linear grey-scale ramp. Then DPD was applied to the grey-scale ramp and successfully reduced the MSE by 95.45%. To examine its effectiveness on real-world images, the DPD method was applied on two sample images, it was observed that more details were shown in the replay field after DPD, and the MSE’s of the two example images were reduced by 19.86% and 15.64%. As the DPD is a one-to-one mapping, the extra computation required is negligible. Thus, we have demonstrated the effectiveness of the proposed DPD-OSPR method to improve reconstruction quality on the existing OSPR algorithm while still keeping its ability for real-time holography.

Acknowledgment

This work was supported by the Engineering and Physical Sciences Research Council (EPSRC) [EP/S022139/1].

References

1SeldowitzM. A.AllebachJ. P.SweeneyD. W.1987Synthesis of digital holograms by direct binary searchAppl. Opt.2610.1364/AO.26.002788

2KirkpatrickS.GelattC. D.VecchiM. P.1983Optimization by simulated annealingScience22010.1126/science.220.4598.671

3GerchbergR. W.1972A practical algorithm for the determination of phase from image and diffraction plane picturesOptik35237246237–46

4ZhangJ.PégardN.ZhongJ.AdesnikH.WallerL.20173d computer-generated holography by non-convex optimizationOpt.4130613131306–1310.1364/OPTICA.4.001306

5LiuS.TakakiY.2020Optimization of phase-only computer-generated holograms based on the gradient descent methodAppl. Sci. (Switzerland)1010.3390/app10124283

6ChoiS.KimJ.PengY.WetzsteinG.2021Optimizing image quality for holographic near-eye displays with michelson holographyOpt.8143146143–610.1364/OPTICA.410622

7ChenC.LeeB.LiN.-N.ChaeM.WangD.WangQ.-H.LeeB.2021Multi-depth hologram generation using stochastic gradient descent algorithm with complex loss functionOpt. Express291508910.1364/OE.425077

8KadisA.WetherfieldB.ShaJ.YangF.WangY.WilkinsonT. D.Effect of bit-depth in stochastic gradient descent performance for phase-only computer-generated holography displaysLondon Imaging Meeting 20222022IS&TSpringfield, VA364036–4010.2352/lim.2022.1.1.09

9CableA. J.BuckleyE.MashP.LawrenceN. A.WilkinsonT. D.CrosslandW. A.53.1: Real-time binary hologram generation for high-quality video projection applicationsSID Symposium Digest of Technical Papers2004Vol. 35John Wiley & SonsHoboken, NJ143110.1889/1.1825772

10GuanX.JianS.HongdaP.ZhiguoZ.HaibinG.An image enhancement method based on gamma correction2009 Second Int’l. Symposium on Computational Intelligence and Design2009Vol. 1IEEEPiscataway, NJ606360–310.1109/ISCID.2009.22

11KangS. J.ChienS. I.2009Apl-adaptive inverse gamma correction for improving gray-level linearity of pdp-tvMol. Cryst. Liq. Cryst.499185/[507]–192/[514]10.1080/15421400802619800

12LeeP.-M.ChenH.-Y.Adjustable gamma correction circuit for tft lcd2005 IEEE Int’l. Symposium on Circuits and Systems (ISCAS)2005Vol. 1IEEEPiscataway, NJ780783780–310.1109/ISCAS.2005.1464704

13PárragaC. A.Roca-VilaJ.KaratzasD.WuergerS. M.2014Limitations of visual gamma corrections in lcd displaysDisplays35227239227–3910.1016/j.displa.2014.07.001

14PoyntonC.in Digital Video and HD: Algorithms and Interfaces20122nd ed.Morgan KaufmannSan Francisco, CA

15CableA. J.“Real-time High-quality Two and Three Dimensional Holographic Video Projection using the One-step Phase Retrieval (OSPR) Approach”. Ph.D. thesis (Dept. Eng., Cambridge University, Cambridge, United Kingdom, 2006)

16DaintithJ.A Dictionary of Physics2009Oxford University PressOxford

17FreemanJ.‘Visor Projected Helmet Mounted Display for Fast Jet Aviators using a Fourier Video Projector”. Ph.D. thesis (Dept. Eng., Cambridge University, Cambridge, United Kingdom, 2009)

18T. M. Inc., “Matlab version: 9.13.0 (r2022b),” 2022

19ZhaoX.“Suzhou Center Mall” (Suzhou, Jiangsu, China, 2017)