Full-colour 3D holographic augmented-reality displays with metasurface waveguides.
Journal
Nature
ISSN: 1476-4687
Titre abrégé: Nature
Pays: England
ID NLM: 0410462
Informations de publication
Date de publication:
08 May 2024
08 May 2024
Historique:
received:
02
07
2023
accepted:
04
04
2024
medline:
9
5
2024
pubmed:
9
5
2024
entrez:
8
5
2024
Statut:
aheadofprint
Résumé
Emerging spatial computing systems seamlessly superimpose digital information on the physical environment observed by a user, enabling transformative experiences across various domains, such as entertainment, education, communication and training
Identifiants
pubmed: 38720077
doi: 10.1038/s41586-024-07386-0
pii: 10.1038/s41586-024-07386-0
doi:
Types de publication
Journal Article
Langues
eng
Sous-ensembles de citation
IM
Informations de copyright
© 2024. The Author(s).
Références
Azuma, R. T. A survey of augmented reality. Presence: Teleoperators Virtual Environ. 6, 355–385 (1997).
doi: 10.1162/pres.1997.6.4.355
Xiong, J., Hsiang, E.-L., He, Z., Zhan, T. & Wu, S.-T. Augmented reality and virtual reality displays: emerging technologies and future perspectives. Light: Sci. Appl. 10, 216 (2021).
doi: 10.1038/s41377-021-00658-8
pubmed: 34697292
Chang, C., Bang, K., Wetzstein, G., Lee, B. & Gao, L. Toward the next-generation VR/AR optics: a review of holographic near-eye displays from a human-centric perspective. Optica 7, 1563–1578 (2020).
doi: 10.1364/OPTICA.406004
pubmed: 34141829
pmcid: 8208705
Kooi, F. L. & Toet, A. Visual comfort of binocular and 3D displays. Displays 25, 99–108 (2004).
doi: 10.1016/j.displa.2004.07.004
Shibata, T., Kim, J., Hoffman, D. M. & Banks, M. S. The zone of comfort: predicting visual discomfort with stereo displays. J. Vis. 11, 11 (2011).
doi: 10.1167/11.8.11
pubmed: 21778252
Cakmakci, O. & Rolland, J. Head-worn displays: a review. J. Disp. Technol. 2, 199–216 (2006).
doi: 10.1109/JDT.2006.879846
Kress, B. C. & Chatterjee, I. Waveguide combiners for mixed reality headsets: a nanophotonics design perspective. Nanophotonics 10, 41–74 (2021).
doi: 10.1515/nanoph-2020-0410
Gabor, D. A new microscopic principle. Nature 161, 777–778 (1949).
doi: 10.1038/161777a0
Sutherland, I. E. The ultimate display. In Proc. of the IFIP Congress (ed. Kalenich, W. A.) 2, 506–508 (Spartan, 1965).
Tay, S. et al. An updatable holographic three-dimensional display. Nature 451, 694–698 (2008).
doi: 10.1038/nature06596
pubmed: 18256667
Blanche, P.-A. et al. Holographic three-dimensional telepresence using large-area photorefractive polymer. Nature 468, 80–83 (2010).
doi: 10.1038/nature09521
pubmed: 21048763
Smalley, D. E., Smithwick, Q., Bove, V., Barabas, J. & Jolly, S. Anisotropic leaky-mode modulator for holographic video displays. Nature 498, 313–317 (2013).
doi: 10.1038/nature12217
pubmed: 23783627
Maimone, A., Georgiou, A. & Kollin, J. S. Holographic near-eye displays for virtual and augmented reality. ACM Trans. Graph. 36, 85 (2017).
doi: 10.1145/3072959.3073624
Molesky, S. et al. Inverse design in nanophotonics. Nat. Photon. 12, 659–670 (2018).
doi: 10.1038/s41566-018-0246-9
Li, Z., Pestourie, R., Lin, Z., Johnson, S. G. & Capasso, F. Empowering metasurfaces with inverse design: principles and applications. ACS Photonics 9, 2178–2192 (2022).
doi: 10.1021/acsphotonics.1c01850
Jiang, J., Chen, M. & Fan, J. A. Deep neural networks for the evaluation and design of photonic devices. Nat. Rev. Mater. 6, 679–700 (2021).
doi: 10.1038/s41578-020-00260-1
Genevet, P., Capasso, F., Aieta, F., Khorasaninejad, M. & Devlin, R. Recent advances in planar optics: from plasmonic to dielectric metasurfaces. Optica 4, 139–152 (2017).
doi: 10.1364/OPTICA.4.000139
Lee, G.-Y., Sung, J. & Lee, B. Metasurface optics for imaging applications. MRS Bull. 45, 202–209 (2020).
doi: 10.1557/mrs.2020.64
Lin, D. et al. Optical metasurfaces for high angle steering at visible wavelengths. Sci. Rep. 7, 2286 (2017).
doi: 10.1038/s41598-017-02167-4
pubmed: 28536465
pmcid: 5442109
Song, J.-H., van de Groep, J., Kim, S. J. & Brongersma, M. L. Non-local metasurfaces for spectrally decoupled wavefront manipulation and eye tracking. Nat. Nanotechnol. 16, 1224–1230 (2021).
doi: 10.1038/s41565-021-00967-4
pubmed: 34594006
Lawrence, M. et al. High quality factor phase gradient metasurfaces. Nat. Nanotechnol. 15, 956–961 (2020).
doi: 10.1038/s41565-020-0754-x
pubmed: 32807879
Cordaro, A. et al. Solving integral equations in free space with inverse-designed ultrathin optical metagratings. Nat. Nanotechnol. 18, 365–372 (2023).
Lee, G.-Y. et al. Metasurface eyepiece for augmented reality. Nat. Commun. 9, 4562 (2018).
Joo, W.-J. & Brongersma, M. L. Creating the ultimate virtual reality display. Science 377, 1376–1378 (2022).
doi: 10.1126/science.abq7011
pubmed: 36137048
Kim, J. et al. Holographic glasses for virtual reality. In ACM SIGGRAPH 2022 Conference Proc. (eds Nandigjav, M. et al.) 33 (ACM, 2022).
Peng, Y., Choi, S., Padmanaban, N. & Wetzstein, G. Neural holography with camera-in-the-loop training. ACM Trans. Graph. 39, 185 (2020).
doi: 10.1145/3414685.3417802
Shi, L., Li, B., Kim, C., Kellnhofer, P. & Matusik, W. Towards real-time photorealistic 3D holography with deep neural networks. Nature 591, 234–239 (2021).
doi: 10.1038/s41586-020-03152-0
pubmed: 33692557
Peng, Y., Choi, S., Kim, J. & Wetzstein, G. Speckle-free holography with partially coherent light sources and camera-in-the-loop calibration. Sci. Adv. 7, eabg5040 (2021).
Shi, L., Li, B. & Matusik, W. End-to-end learning of 3D phase-only holograms for holographic display. Light Sci. Appl. 11, 247 (2022).
Yeom, H.-J. et al. 3d holographic head mounted display using holographic optical elements with astigmatism aberration compensation. Opt, Express 23, 32025–32034 (2015).
doi: 10.1364/OE.23.032025
pubmed: 26698993
Jeong, J. et al. Holographically customized optical combiner for eye-box extended near-eye display. Opt. Express 27, 38006–38018 (2019).
doi: 10.1364/OE.382190
pubmed: 31878572
Yeom, J., Son, Y. & Choi, K. Crosstalk reduction in voxels for a see-through holographic waveguide by using integral imaging with compensated elemental images. Photonics 8, 217 (2021).
Choi, M.-H., Shin, K.-S., Jang, J., Han, W. & Park, J.-H. Waveguide-type Maxwellian near-eye display using a pin-mirror holographic optical element array. Opt. Lett. 47, 405–408 (2022).
doi: 10.1364/OL.443004
pubmed: 35030617
Chen, W. T. et al. A broadband achromatic metalens for focusing and imaging in the visible. Nat. Nanotechnol. 13, 220–226 (2018).
doi: 10.1038/s41565-017-0034-6
pubmed: 29292382
Li, Z. et al. Meta-optics achieves RGB-achromatic focusing for virtual reality. Sci. Adv. 7, eabe4458 (2021).
doi: 10.1126/sciadv.abe4458
pubmed: 33571130
pmcid: 7840120
Kim, C. & Lee, B. Torcwa: GPU-accelerated Fourier modal method and gradient-based optimization for metasurface design. Comput. Phys. Comm. 282, 108552 (2023).
doi: 10.1016/j.cpc.2022.108552
Kingma, D. P. & Ba, J. Adam: A method for stochastic optimization. In Proceedings of the 3rd International Conference on Learning Representations (2015).
Park, J.-S. et al. All-glass, large metalens at visible wavelength using deep-ultraviolet projection lithography. Nano Lett. 19, 8673–8682 (2019).
doi: 10.1021/acs.nanolett.9b03333
pubmed: 31726010
Kim, J. et al. Scalable manufacturing of high-index atomic layer–polymer hybrid metasurfaces for metaphotonics in the visible. Nat. Mater. 22, 474–481 (2023).
doi: 10.1038/s41563-023-01485-5
pubmed: 36959502
Chakravarthula, P., Tseng, E., Srivastava, T., Fuchs, H. & Heide, F. Learned hardware-in-the-loop phase retrieval for holographic near-eye displays. ACM Trans. Graph. 39, 186 (2020).
doi: 10.1145/3414685.3417846
Choi, S., Gopakumar, M., Peng, Y., Kim, J. & Wetzstein, G. Neural 3D holography: learning accurate wave propagation models for 3D holographic virtual and augmented reality displays. ACM Trans. Graph. 40, 240 (2021).
Choi, S. et al. Time-multiplexed neural holography: a flexible framework for holographic near-eye displays with fast heavily-quantized spatial light modulators. In ACM SIGGRAPH 2022 Conference Proc. (eds Nandigjav, M. et al.) 32 (2022).
Jang, C., Bang, K., Chae, M., Lee, B. & Lanman, D. Waveguide holography for 3D augmented reality glasses. Nat. Commun. 15, 66 (2024).
Hwang, C.-S. et al. 21-2: Invited paper: 1µm pixel pitch spatial light modulator panel for digital holography. Dig. Tech. Pap. SID Int. Symp. 51, 297–300 (2020).
doi: 10.1002/sdtp.13862
Park, J., Lee, K. & Park, Y. Ultrathin wide-angle large-area digital 3D holographic display using a non-periodic photon sieve. Nat. Commun. 10, 1304 (2019).
doi: 10.1038/s41467-019-09126-9
pubmed: 30898998
pmcid: 6428928
Kuo, G., Waller, L., Ng, R. & Maimone, A. High resolution étendue expansion for holographic displays. ACM Trans. Graph. 39, 66 (2020).
doi: 10.1145/3386569.3392414
Jang, C., Bang, K., Li, G. & Lee, B. Holographic near-eye display with expanded eye-box. ACM Trans. Graph. 37, 195 (2018).
doi: 10.1145/3272127.3275069
Horisaki, R., Takagi, R. & Tanida, J. Deep-learning-generated holography. Appl. Optics 57, 3859–3863 (2018).
doi: 10.1364/AO.57.003859
Kim, C., Zimmer, H., Pritch, Y., Sorkine-Hornung, A. & Gross, M. Scene reconstruction from high spatio-angular resolution light fields. ACM Trans. Graph. 32, 73 (2013).
doi: 10.1145/2461912.2461926
Ronneberger, O., Fischer, P. & Brox, T. U-net: convolutional networks for biomedical image segmentation. In Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015 (eds Navab, N., Hornegger, J., Wells, W. & Frangi, A.) 234–241 (Springer, 2015).
Ulyanov, D., Vedaldi, A. & Lempitsky, V. Improved texture networks: maximizing quality and diversity in feed-forward stylization and texture synthesis. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition 6924–6932 (2017).