Real-time skin chromophore estimation from hyperspectral images using a neural network.
hyperspectral
imaging
in vivo
machine learning
non-invasive
Journal
Skin research and technology : official journal of International Society for Bioengineering and the Skin (ISBS) [and] International Society for Digital Imaging of Skin (ISDIS) [and] International Society for Skin Imaging (ISSI)
ISSN: 1600-0846
Titre abrégé: Skin Res Technol
Pays: England
ID NLM: 9504453
Informations de publication
Date de publication:
Mar 2021
Mar 2021
Historique:
received:
14
05
2020
accepted:
20
06
2020
pubmed:
18
7
2020
medline:
19
8
2021
entrez:
18
7
2020
Statut:
ppublish
Résumé
Hyperspectral imaging for in vivo human skin study has shown great potential by providing non-invasive measurement from which information usually invisible to the human eye can be revealed. In particular, maps of skin parameters including oxygen rate, blood volume fraction, and melanin concentration can be estimated from a hyperspectral image by using an optical model and an optimization algorithm. These applications, relying on hyperspectral images acquired with a high-resolution camera especially dedicated to skin measurement, have yielded promising results. However, the data analysis process is relatively expensive in terms of computation cost, with calculation of full-face skin property maps requiring up to 5 hours for 3-megapixels hyperspectral images. Such a computation time prevents punctual previewing and quality assessment of the maps immediately after acquisition. To address this issue, we have implemented a neural network that models the optimization-based analysis algorithm. This neural network has been trained on a set of hyperspectral images, acquired from 204 patients and their corresponding skin parameter maps, which were calculated by optimization. The neural network is able to generate skin parameter maps that are visually very faithful to the reference maps much more quickly than the optimization-based algorithm, with computation times as short as 2 seconds for a 3-megapixel image representing a full face and 0.5 seconds for a 1-megapixel image representing a smaller area of skin. The average deviation calculated on selected areas shows the network's promising generalization ability, even on wide-field full-face images. Currently, the network is adequate for preview purposes, providing relatively accurate results in a few seconds.
Sections du résumé
BACKGROUND
BACKGROUND
Hyperspectral imaging for in vivo human skin study has shown great potential by providing non-invasive measurement from which information usually invisible to the human eye can be revealed. In particular, maps of skin parameters including oxygen rate, blood volume fraction, and melanin concentration can be estimated from a hyperspectral image by using an optical model and an optimization algorithm. These applications, relying on hyperspectral images acquired with a high-resolution camera especially dedicated to skin measurement, have yielded promising results. However, the data analysis process is relatively expensive in terms of computation cost, with calculation of full-face skin property maps requiring up to 5 hours for 3-megapixels hyperspectral images. Such a computation time prevents punctual previewing and quality assessment of the maps immediately after acquisition.
METHODS
METHODS
To address this issue, we have implemented a neural network that models the optimization-based analysis algorithm. This neural network has been trained on a set of hyperspectral images, acquired from 204 patients and their corresponding skin parameter maps, which were calculated by optimization.
RESULTS
RESULTS
The neural network is able to generate skin parameter maps that are visually very faithful to the reference maps much more quickly than the optimization-based algorithm, with computation times as short as 2 seconds for a 3-megapixel image representing a full face and 0.5 seconds for a 1-megapixel image representing a smaller area of skin. The average deviation calculated on selected areas shows the network's promising generalization ability, even on wide-field full-face images.
CONCLUSION
CONCLUSIONS
Currently, the network is adequate for preview purposes, providing relatively accurate results in a few seconds.
Substances chimiques
Melanins
0
Types de publication
Journal Article
Langues
eng
Sous-ensembles de citation
IM
Pagination
163-177Subventions
Organisme : Région Auvergne-Rhône-Alpes
ID : ARC 6
Organisme : Agence Nationale de la Recherche
ID : ANR-10-LABX-0075
Organisme : Agence Nationale de la Recherche
ID : ANR-11-IDEX-0007
Informations de copyright
© 2020 John Wiley & Sons A/S. Published by John Wiley & Sons Ltd.
Références
Matts PJ, Fink B, Grammer K, Burquest M. Color homogeneity and visual perception of age, health, and attractiveness of female facial skin. J Am Acad Dermatol. 2007;57:977-984.
Lu G, Fei B. Medical hyperspectral imaging: a review. J Biomed Optics. 2014;19:10901.
Seroul P, Hébert M, Cherel M, Vernet R, Clerc R, Jomier M. Model-based skin pigment cartography by high-resolution hyperspectral imaging. J Imag Sci Technol. 2017;60(6):604041-604047.
Gevaux L, Adnet C, Séroul P, et al. Three-dimensional maps of human skin properties on full face with shadows using 3-D hyperspectral imaging. J Biomed Optics. 2019;24:1.
Nkengne J, Robic P, Seroul S, Gueheunneux MJ, Vie K. SpectraCam®: A new polarized hyperspectral imaging system for repeatable and reproducible in vivo skin quantification of melanin, total hemoglobin, and oxygen saturation. Skin Res Technol. 2018;24:99-107.
Wang L, Zhao X, Jacques SL. "Computation of the optical properties of tissues from light reflectance using a neural network," Laser-Tissue Interaction V; and Ultraviolet Radiation Hazards. Int Society Optics Photonics. 1994;2134:391-400.
Jäger M, Foschum F, Kienle A. Application of multiple artificial neural networks for the determination of the optical properties of turbid media. J Biomed Optics. 2013;18:57005.
Wirkert SJ, Kenngott H, Mayer B, et al. Robust near real-time estimation of physiological parameters from megapixel multispectral images with inverse Monte Carlo and random forest regression. Int J Comp Ass Radiol Surg. 2016;11:909-917.
Panigrahi S, Gioux S. Machine learning approach for rapid and accurate estimation of optical properties using spatial frequency domain imaging. J Biomed Optics. 2018;24:1.
Zherebtsov E, Dremin V, Popov A, et al. Hyperspectral imaging of human skin aided by artificial neural networks. Biomed Opt Express. 2019;10:3545.
Garini Y, Young IT, McNamara G. Spectral imaging: Principles and applications. Cytometry Part A. 2006;69A:735-747.
Kubelka P, Munk F. An article on optics of paint layers. Z Tech Phys. 1931;12:593-601.
Jacques SL. Optical properties of biological tissues: a review. Phys Med Biol. 2013;58:R37-R61.
Jacques SL. Optical absorption of melanin. https://omlc.org/spectra/melanin/. Accessed March 16, 2020.
Prahl S. Optical absorption of hemoglobin. https://omlc.org/spectra/hemoglobin/. Accessed March 16, 2020
Kubelka P. New Contributions to the Optics of Intensely Light-Scattering Materials Part II: Nonhomogeneous Layers*. J Opt Soc Am. 1954;44:330.
Saunderson JL. Calculation of the color of pigmented plastics. J Opt Soc Am. 1942;32:727-736.
Yuhas R, Goetz A, Boardman J. “Discrimination among semiarid landscape endmembers using the spectral angle mapper (sam) algorithm,” JPL Publication. 147-149 (1992).
Jain K, Mao J, Mohiuddin KM. Artificial neural networks: a tutorial. Computer. 1996;3:31-44.
Ioffe S, Szegedy C. "Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift," arXiv:1502.03167 [cs] (2015).
Gardner MW, Dorling SR. Artificial neural networks (the multilayer perceptron)-a review of applications in the atmospheric sciences. Atmos Environ. 1998;32:2627-2636.
Werbos P“Beyond regression: new tools for prediction and analysis in the behavioral sciences”, PhD Diss (1974).
Forsyth D, Zisserman A. “Mutual illumination”, Proceedings CVPR'89. IEEE Comput Soc Conf Comput Vis Pattern Recognit. 1989;466-473.
Deeb R, Van de Weijer J, Muselet D, Hebert M, Tremeau A. Deep spectral reflectance and illuminant estimation from self-interreflections. J Opt Soc Am. 2019;36:105.