Multi-channel feature extraction for virtual histological staining of photon absorption remote sensing images.
Journal
Scientific reports
ISSN: 2045-2322
Titre abrégé: Sci Rep
Pays: England
ID NLM: 101563288
Informations de publication
Date de publication:
23 Jan 2024
23 Jan 2024
Historique:
received:
09
08
2023
accepted:
20
01
2024
medline:
24
1
2024
pubmed:
24
1
2024
entrez:
23
1
2024
Statut:
epublish
Résumé
Accurate and fast histological staining is crucial in histopathology, impacting diagnostic precision and reliability. Traditional staining methods are time-consuming and subjective, causing delays in diagnosis. Digital pathology plays a vital role in advancing and optimizing histology processes to improve efficiency and reduce turnaround times. This study introduces a novel deep learning-based framework for virtual histological staining using photon absorption remote sensing (PARS) images. By extracting features from PARS time-resolved signals using a variant of the K-means method, valuable multi-modal information is captured. The proposed multi-channel cycleGAN model expands on the traditional cycleGAN framework, allowing the inclusion of additional features. Experimental results reveal that specific combinations of features outperform the conventional channels by improving the labeling of tissue structures prior to model training. Applied to human skin and mouse brain tissue, the results underscore the significance of choosing the optimal combination of features, as it reveals a substantial visual and quantitative concurrence between the virtually stained and the gold standard chemically stained hematoxylin and eosin images, surpassing the performance of other feature combinations. Accurate virtual staining is valuable for reliable diagnostic information, aiding pathologists in disease classification, grading, and treatment planning. This study aims to advance label-free histological imaging and opens doors for intraoperative microscopy applications.
Identifiants
pubmed: 38263394
doi: 10.1038/s41598-024-52588-1
pii: 10.1038/s41598-024-52588-1
doi:
Types de publication
Journal Article
Langues
eng
Sous-ensembles de citation
IM
Pagination
2009Subventions
Organisme : Canadian Network for Research and Innovation in Machining Technology, Natural Sciences and Engineering Research Council of Canada
ID : DGECR-2019-00143
Organisme : Canadian Network for Research and Innovation in Machining Technology, Natural Sciences and Engineering Research Council of Canada
ID : DGECR-2019-00143
Organisme : Canadian Network for Research and Innovation in Machining Technology, Natural Sciences and Engineering Research Council of Canada
ID : DGECR-2019-00143
Organisme : Canadian Network for Research and Innovation in Machining Technology, Natural Sciences and Engineering Research Council of Canada
ID : DGECR-2019-00143
Organisme : Canadian Network for Research and Innovation in Machining Technology, Natural Sciences and Engineering Research Council of Canada
ID : DGECR-2019-00143
Organisme : Canadian Network for Research and Innovation in Machining Technology, Natural Sciences and Engineering Research Council of Canada
ID : DGECR-2019-00143
Organisme : Canada Foundation for Innovation
ID : JELF #38000
Organisme : Canada Foundation for Innovation
ID : JELF #38000
Organisme : Canada Foundation for Innovation
ID : JELF #38000
Organisme : Canada Foundation for Innovation
ID : JELF #38000
Organisme : Canada Foundation for Innovation
ID : JELF #38000
Organisme : Canada Foundation for Innovation
ID : JELF #38000
Organisme : Mitacs Accelerate
ID : IT13594
Organisme : Mitacs Accelerate
ID : IT13594
Organisme : Mitacs Accelerate
ID : IT13594
Organisme : Mitacs Accelerate
ID : IT13594
Organisme : Mitacs Accelerate
ID : IT13594
Organisme : Mitacs Accelerate
ID : IT13594
Organisme : illumiSonics Inc
ID : SRA #083181
Organisme : illumiSonics Inc
ID : SRA #083181
Organisme : illumiSonics Inc
ID : SRA #083181
Organisme : illumiSonics Inc
ID : SRA #083181
Organisme : illumiSonics Inc
ID : SRA #083181
Organisme : illumiSonics Inc
ID : SRA #083181
Organisme : New frontiers in research fund - exploration
ID : NFRFE-2019-01012
Organisme : New frontiers in research fund - exploration
ID : NFRFE-2019-01012
Organisme : New frontiers in research fund - exploration
ID : NFRFE-2019-01012
Organisme : New frontiers in research fund - exploration
ID : NFRFE-2019-01012
Organisme : New frontiers in research fund - exploration
ID : NFRFE-2019-01012
Organisme : New frontiers in research fund - exploration
ID : NFRFE-2019-01012
Organisme : The Canadian Institutes of Health Research
ID : CIHR PJT 185984
Organisme : The Canadian Institutes of Health Research
ID : CIHR PJT 185984
Organisme : The Canadian Institutes of Health Research
ID : CIHR PJT 185984
Organisme : The Canadian Institutes of Health Research
ID : CIHR PJT 185984
Organisme : The Canadian Institutes of Health Research
ID : CIHR PJT 185984
Organisme : The Canadian Institutes of Health Research
ID : CIHR PJT 185984
Informations de copyright
© 2024. The Author(s).
Références
Brown, L. Improving histopathology turnaround time: A process management approach. Curr. Diagn. Pathol. 10(6), 444–452. https://doi.org/10.1016/j.cdip.2004.07.008 (2004).
doi: 10.1016/j.cdip.2004.07.008
Oberauner-Wappis, L. et al. Protocol for HER2 FISH determination on PAXgene-fixed and paraffin-embedded tissue in breast cancer. Int. J. Exp. Pathol. 97(2), 202–206. https://doi.org/10.1111/iep.12185 (2016).
doi: 10.1111/iep.12185
pubmed: 27273709
pmcid: 4926048
Li, J. et al. Biopsy-free in vivo virtual histology of skin using deep learning. Light Sci. Appl. 10(1), 233. https://doi.org/10.1038/s41377-021-00674-8 (2021).
doi: 10.1038/s41377-021-00674-8
pubmed: 34795202
pmcid: 8602311
Aumann, S., Donner, S., Fischer, J., and Müller, F., “Optical Coherence Tomography (OCT): Principle and Technical Realization,” in High Resolution Imaging in Microscopy and Ophthalmology: New Frontiers in Biomedical Optics, J. F. Bille, Ed., Cham: Springer International Publishing, 2019, pp. 59–85. doi: https://doi.org/10.1007/978-3-030-16638-0_3 .
Eichberg, D. G. et al. Stimulated Raman histology for rapid and accurate intraoperative diagnosis of CNS tumors: Prospective blinded study. J. Neurosurg. 134(1), 137–143. https://doi.org/10.3171/2019.9.JNS192075 (2021).
doi: 10.3171/2019.9.JNS192075
Sarri, B. et al. Stimulated Raman histology: One to one comparison with standard hematoxylin and eosin staining. Biomed. Opt. Express 10(10), 5378. https://doi.org/10.1364/BOE.10.005378 (2019).
doi: 10.1364/BOE.10.005378
pubmed: 31646052
pmcid: 6788596
Rivenson, Y. et al. Virtual histological staining of unlabelled tissue-autofluorescence images via deep learning. Nat. Biomed. Eng. 3(6), 466–477. https://doi.org/10.1038/s41551-019-0362-y (2019).
doi: 10.1038/s41551-019-0362-y
pubmed: 31142829
Rivenson, Y. et al. PhaseStain: The digital staining of label-free quantitative phase microscopy images using deep learning. Light Sci. Appl. 8(1), 23. https://doi.org/10.1038/s41377-019-0129-y (2019).
doi: 10.1038/s41377-019-0129-y
pubmed: 30728961
pmcid: 6363787
Kang, L., Li, X., Zhang, Y. & Wong, T. T. W. Deep learning enables ultraviolet photoacoustic microscopy based histological imaging with near real-time virtual staining. Photoacoustics 25, 100308. https://doi.org/10.1016/j.pacs.2021.100308 (2022).
doi: 10.1016/j.pacs.2021.100308
pubmed: 34703763
Boktor, M. et al. Virtual histological staining of label-free total absorption photoacoustic remote sensing (TA-PARS). Sci. Rep. 12(1), 10296. https://doi.org/10.1038/s41598-022-14042-y (2022).
doi: 10.1038/s41598-022-14042-y
pubmed: 35717539
pmcid: 9206643
Kretschmer, S. et al. Autofluorescence multiphoton microscopy for visualization of tissue morphology and cellular dynamics in murine and human airways. Lab. Invest. 96(8), 918–931. https://doi.org/10.1038/labinvest.2016.69 (2016).
doi: 10.1038/labinvest.2016.69
pubmed: 27400364
pmcid: 4972900
Ecclestone, B. R., Bell, K. L., Sparkes, S., Dinakaran, D., Mackey, J., and Reza, P. H., Label-free virtual Hematoxylin and Eosin (H&E) staining using second generation Photoacoustic Remote Sensing (PARS),” (2021).
Hajireza, P., Shi, W., Bell, K., Paproski, R. J. & Zemp, R. J. Non-interferometric photoacoustic remote sensing microscopy. Light Sci. Appl. 6(6), e16278–e16278. https://doi.org/10.1038/lsa.2016.278 (2017).
doi: 10.1038/lsa.2016.278
pubmed: 30167263
pmcid: 6062239
Bell, K. et al. Reflection-mode virtual histology using photoacoustic remote sensing microscopy. Sci. Rep. https://doi.org/10.1038/s41598-020-76155-6 (2020).
doi: 10.1038/s41598-020-76155-6
pubmed: 33318517
pmcid: 7736588
Haven, N. J. M., Bell, K. L., Kedarisetti, P., Lewis, J. D. & Zemp, R. J. Ultraviolet photoacoustic remote sensing microscopy. Opt. Lett. 44(14), 3586. https://doi.org/10.1364/OL.44.003586 (2019).
doi: 10.1364/OL.44.003586
pubmed: 31305578
Kedarisetti, P., Haven, N. J. M., Restall, B. S., Martell, M. T. & Zemp, R. J. Label-free lipid contrast imaging using non-contact near-infrared photoacoustic remote sensing microscopy. Opt. Lett. 45(16), 4559. https://doi.org/10.1364/OL.397614 (2020).
doi: 10.1364/OL.397614
pubmed: 32797009
Tweel, J. E. D., Ecclestone, B. R., Boktor, M., Simmons, J. A. T., Fieguth, P., and Reza, P. H., Virtual Histology with Photon Absorption Remote Sensing using a Cycle-Consistent Generative Adversarial Network with Weakly Registered Pairs,” (2023), https://doi.org/10.48550/ARXIV.2306.08583 .
Zhu, J., Park, T., Isola, P., and Efros, A. A. Unpaired image-to-image translation using cycle-consistent adversarial networks,” in 2017 IEEE International Conference on Computer Vision (ICCV), Oct. 2017, pp. 2242–2251. https://doi.org/10.1109/ICCV.2017.244 .
Tweel, J. E. D., Ecclestone, B. R., Boktor, M., Dinakaran, D., Mackey, J. R., and Reza, P. H. Automated Whole Slide Imaging for Label-Free Histology using Photon Absorption Remote Sensing Microscopy .
Pellegrino, N., Fieguth, P. W. & HajiReza, P. “K-Means for noise-insensitive multi-dimensional feature learning. Pattern Recognit. Lett. 170, 113–120. https://doi.org/10.1016/j.patrec.2023.04.009 (2023).
doi: 10.1016/j.patrec.2023.04.009
Pellegrino, N. et al. Time-domain feature extraction for target specificity in photoacoustic remote sensing microscopy. Opt. Lett. 47(15), 3952. https://doi.org/10.1364/OL.457142 (2022).
doi: 10.1364/OL.457142
pubmed: 35913356
Goshtasby, A. Image registration by local approximation methods. Image Vis. Comput. 6(4), 255–261. https://doi.org/10.1016/0262-8856(88)90016-9 (1988).
doi: 10.1016/0262-8856(88)90016-9
P. Fieguth, Statistical image processing and multidimensional modeling. in Information Science and Statistics. New York, NY: Springer New York, (2011). https://doi.org/10.1007/978-1-4419-7294-1 .
Liang, Y., Lee, D., Li, Y. & Shin, B.-S. Unpaired medical image colorization using generative adversarial network. Multimed. Tools Appl. https://doi.org/10.1007/s11042-020-10468-6 (2021).
doi: 10.1007/s11042-020-10468-6
Wang, Z., Bovik, A. C., Sheikh, H. R. & Simoncelli, E. P. Image quality assessment: From error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612. https://doi.org/10.1109/TIP.2003.819861 (2004).
doi: 10.1109/TIP.2003.819861
pubmed: 15376593
Huynh-Thu, Q. & Ghanbari, M. Scope of validity of PSNR in image/video quality assessment. Electron. Lett. 44(13), 800. https://doi.org/10.1049/el:20080522 (2008).
doi: 10.1049/el:20080522
Bai, B. et al. Deep learning-enabled virtual histological staining of biological samples. Light Sci. Appl. 12(1), 57. https://doi.org/10.1038/s41377-023-01104-7 (2023).
doi: 10.1038/s41377-023-01104-7
pubmed: 36864032
pmcid: 9981740
R. Zhang, P. Isola, A. A. Efros, E. Shechtman, and O. Wang, “The unreasonable effectiveness of deep features as a perceptual metric,” (2018), https://doi.org/10.48550/ARXIV.1801.03924 .
V. Jolly, M. Dwivedi, Y. Patel, T. Yadav, and S. Dhage, Bringing monochrome to life: A GAN-based approach to colorizing black and white images, in 2023 IEEE 8th International Conference for Convergence in Technology (I2CT), Lonavla, India: IEEE, Apr. 2023, pp. 1–6. doi: https://doi.org/10.1109/I2CT57861.2023.10126485 .
Tweel, J. E. D. et al. Photon absorption remote sensing imaging of breast needle core biopsies is diagnostically equivalent to gold standard H&E histologic assessment. Curr. Oncol. 30(11), 11. https://doi.org/10.3390/curroncol30110708 (2023).
doi: 10.3390/curroncol30110708