Non-convex optimization for inverse problem solving in computer-generated holography

Sui et al. Light: Science & Applications (2024)13:158 https://doi.org/10.1038/s41377-024-01446-w Official journal of the CIOMP 2047-7538 www.nature.com/lsa REVIEW ARTICLE Open Access Non-convex optimization for inverse problem solving in computer-generated holography Xiaomeng Sui1,2, Zehao He1, Daping Chu 2,3 ✉ and Liangcai Cao 1✉ 1234567890():,; 1234567890():,; 1234567890():,; 1234567890():,; Abstract Computer-generated holography is a promising technique that modulates user-defined wavefronts with digital holograms. Computing appropriate holograms with faithful reconstructions is not only a problem closely related to the fundamental basis of holography but also a long-standing challenge for researchers in general fields of optics. Finding the exact solution of a desired hologram to reconstruct an accurate target object constitutes an ill-posed inverse problem. The general practice of single-diffraction computation for synthesizing holograms can only provide an approximate answer, which is subject to limitations in numerical implementation. Various non-convex optimization algorithms are thus designed to seek an optimal solution by introducing different constraints, frameworks, and initializations. Herein, we overview the optimization algorithms applied to computer-generated holography, incorporating principles of hologram synthesis based on alternative projections and gradient descent methods. This is aimed to provide an underlying basis for optimized hologram generation, as well as insights into the cuttingedge developments of this rapidly evolving field for potential applications in virtual reality, augmented reality, head-up display, data encryption, laser fabrication, and metasurface design. Introduction Holography is a long-existing concept first raised by Dennis Gabor in the late 1940s, which aimed at improving resolution in electron microscopy1. In the 1960s, the development of laser technology enabled practical optical holography2,3. Early demonstration of optical holography can be described with two steps: interferential recording of an object wavefront and diffractive reconstruction from a hologram. Recent advancements in digital devices have enabled both the recording and the reconstruction processes to be performed computationally. One branch of holography involves optically recording an object wavefront and computationally reconstructing it from a digital hologram4,5, commonly referred to as digital holography6. This approach enables promising applications such as imaging, measurement, and detection. Another branch of holography involves computationally generating a hologram and optically Correspondence: Daping Chu () or Liangcai Cao () 1 Department of Precision Instruments, Tsinghua University, Beijing 100084, China 2 Department of Engineering, Centre for Photonic Devices and Sensors, University of Cambridge, 9 JJ Thomson Avenue, Cambridge CB3 0FA, UK Full list of author information is available at the end of the article reconstructing an object’s wavefront, commonly referred to as computer-generated holography (CGH), which provides an approach to digitally modulate a volumetric wavefront7. This technology, half inherited from optical holography and half advanced by computer science, has become an emerging focus of academia and industry8–10. Computer-generated holograms, encoded on various types of holographic media, enable a wide range of applications. Holograms fabricated as diffractive optical elements (DOEs)11 or metasurfaces12–14 can reproduce specific spatial light fields, achieving structured light projection15–17, data storage18,19, and optical encryption20–24. With refreshable devices like spatial light modulators (SLMs)25–27, as is shown in Fig. 1, CGH is able to assist many fields of investigations, including three-dimensional display, holographic lithography28, optical trapping29, and optogenetics30–32. In recent years, CGH also boosts the birth and growth of potential markets of virtual reality (VR)33–38, augmented reality (AR)39–42, head-up display43–45, holographic printing46, optical communication47, and optical computing48. Although these applications and fields of investigation involve the encoding of holograms with various © The Author(s) 2024 Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. Sui et al. Light: Science & Applications (2024)13:158 2022 Tunable liquid crystal grating display 2022 End-to-end learning Page 2 of 23 2022 Modular coarse integral display 2022 Diffraction-engineered calculation 2022 Complex opto-magnetic modulation 2023 Continuous-depth display 1.4 500 mm mm 0.35 31 mm mm 1 Voff nam ic ho lo- bri write k tic ho lobri reset k write k+1 reset k+1 y x 5 mm z 500 mm diffraction image Tiling of two holo-bricks Holo-brick 2 1.25 mm 31 mm y x z 5 mm 1.25 mm 4 mm 1.25 mm recording track of movie Six holo-bricks 12 mm offset Tiling of six holo-bricks CGH simulation k XOR (k+1) write k write k+1 1.00 contrast 0 Eyepiece Fourier filter Broad spectrum Light Sci. Appl. 11(1): 57 Near focus (2D) kg 1 l kout kg 8 1.4 z 0.35 mm l Nat. Commun. 13(1): 6012 k0 1SLM CITL mm –1 1 Vectorial hologram QWP LP HWP y BS QWP l l 2 2020 3D vectorial learning –2 OAM-preserved hologram –1 1 2 PBS l = –1 OAM-selective hologram l = –2 0.8 I 50 Pm l = –2 l = –1 K l Ix l=1 ¦ l=2 x l=2 y x z y 1SLM CITL Iy 2SLM CITL n = 1k = 1 Far focus (0D) z 0.64 0.48 0.32 0.16 Iz l = –2 l = –1 l=1 l=2 OAM-multiplexing hologram Photograph Target 31 Nat. Photonics, 17(5): 427 l = –1 N ¦ Partially coherent model mm ϕ k0 –2 2020 Camera-in-the-loop training 0 l kin ϕ l=1 Finite-sized emitting area mm 500 10 1 l kout y 0 kin Target LED or SLED Laser x 6 2020 Orbital angular momentum encryption Captured image Polarizer 4 Nat. Commun. 13(1): 7286 2021 Dual-SLM display Optics Simulated partially coherent model Intermediate image SLM 2 Object sequence 2021 Partially-coherent source SLM 3D scene 0 Diffraction efficiency (%) Camera Rear focus Light Sci. Appl. 11(1): 247 Light Sci. Appl. 11(1): 188 y magnetic reset optical res (...truncated)