Virtual birefringence imaging and histological staining of amyloid deposits in label-free tissue using autofluorescence microscopy and deep learning.
Journal
Nature communications
ISSN: 2041-1723
Titre abrégé: Nat Commun
Pays: England
ID NLM: 101528555
Informations de publication
Date de publication:
12 Sep 2024
12 Sep 2024
Historique:
received:
02
03
2024
accepted:
02
09
2024
medline:
13
9
2024
pubmed:
13
9
2024
entrez:
12
9
2024
Statut:
epublish
Résumé
Systemic amyloidosis involves the deposition of misfolded proteins in organs/tissues, leading to progressive organ dysfunction and failure. Congo red is the gold-standard chemical stain for visualizing amyloid deposits in tissue, showing birefringence under polarization microscopy. However, Congo red staining is tedious and costly to perform, and prone to false diagnoses due to variations in amyloid amount, staining quality and manual examination of tissue under a polarization microscope. We report virtual birefringence imaging and virtual Congo red staining of label-free human tissue to show that a single neural network can transform autofluorescence images of label-free tissue into brightfield and polarized microscopy images, matching their histochemically stained versions. Blind testing with quantitative metrics and pathologist evaluations on cardiac tissue showed that our virtually stained polarization and brightfield images highlight amyloid patterns in a consistent manner, mitigating challenges due to variations in chemical staining quality and manual imaging processes in the clinical workflow.
Identifiants
pubmed: 39266547
doi: 10.1038/s41467-024-52263-z
pii: 10.1038/s41467-024-52263-z
doi:
Substances chimiques
Amyloid
0
Congo Red
3U05FHG59S
Types de publication
Journal Article
Langues
eng
Sous-ensembles de citation
IM
Pagination
7978Informations de copyright
© 2024. The Author(s).
Références
Baker, K. R. & Rice, L. The amyloidoses: clinical features, diagnosis and treatment. Methodist DeBakey Cardiovasc. J. 8, 3–7 (2012).
pubmed: 23227278
pmcid: 3487569
doi: 10.14797/mdcj-8-3-3
Picken, M. M. The pathology of amyloidosis in classification: a review. Acta Haematol. 143, 322–334 (2020).
pubmed: 32392555
doi: 10.1159/000506696
Amyloidosis Statistics. Cancer.Net https://www.cancer.net/cancer-types/amyloidosis/statistics (2012).
Maixnerova, D. et al. Nationwide biopsy survey of renal diseases in the Czech Republic during the years 1994-2011. J. Nephrol. 28, 39–49 (2015).
pubmed: 24756969
doi: 10.1007/s40620-014-0090-z
Desikan, K. R. et al. Incidence and impact of light chain associated (AL) amyloidosis on the prognosis of patients with multiple myeloma treated with autologous transplantation. Leuk. Lymphoma 27, 315–319 (1997).
pubmed: 9402328
doi: 10.3109/10428199709059685
Gertz, M. A., Lacy, M. Q. & Dispenzieri, A. Amyloidosis: recognition, confirmation, prognosis, and therapy. Mayo Clin. Proc. 74, 490–494 (1999).
pubmed: 10319082
doi: 10.1016/S0025-6196(11)65130-6
Gilstrap, L. G. et al. Epidemiology of cardiac amyloidosis–associated heart failure hospitalizations among fee-for-service medicare beneficiaries in the United States. Circ. Heart Fail. 12, e005407 (2019).
pubmed: 31170802
pmcid: 6557425
doi: 10.1161/CIRCHEARTFAILURE.118.005407
Benson, M. D. et al. Tissue biopsy for the diagnosis of amyloidosis: experience from some centres. Amyloid 29, 8–13 (2022).
pubmed: 34766859
doi: 10.1080/13506129.2021.1994386
Martinez-Naharro, A., Hawkins, P. N. & Fontana, M. Cardiac amyloidosis. Clin. Med. Lond. Engl. 18, s30–s35 (2018).
doi: 10.7861/clinmedicine.18-2-s30
Klunk, W. E., Pettegrew, J. W. & Abraham, D. J. Quantitative evaluation of congo red binding to amyloid-like proteins with a beta-pleated sheet conformation. J. Histochem. Cytochem. 37, 1273–1281 (1989).
pubmed: 2666510
doi: 10.1177/37.8.2666510
El-Meanawy, A., Mueller, C. & Iczkowski, K. A. Improving sensitivity of amyloid detection by Congo red stain by using polarizing microscope and avoiding pitfalls. Diagn. Pathol. 14, 57 (2019).
pubmed: 31200733
pmcid: 6567537
doi: 10.1186/s13000-019-0822-4
Maurer, M. S., Elliott, P., Comenzo, R., Semigran, M. & Rapezzi, C. Addressing common questions encountered in the diagnosis and management of cardiac amyloidosis. Circulation 135, 1357–1377 (2017).
pubmed: 28373528
pmcid: 5392416
doi: 10.1161/CIRCULATIONAHA.116.024438
Al-Janabi, S., Huisman, A. & Van Diest, P. J. Digital pathology: current status and future perspectives. Histopathology 61, 1–9 (2012).
pubmed: 21477260
doi: 10.1111/j.1365-2559.2011.03814.x
Song, A. H. et al. Artificial intelligence for digital and computational pathology. Nat. Rev. Bioeng. 1, 930–949 (2023).
doi: 10.1038/s44222-023-00096-8
Patel, A. et al. Contemporary whole slide imaging devices and their applications within the modern pathology department: a selected hardware review. J. Pathol. Inform. 12, 50 (2021).
pubmed: 35070479
pmcid: 8721869
doi: 10.4103/jpi.jpi_66_21
Madabhushi, A. & Lee, G. Image analysis and machine learning in digital pathology: challenges and opportunities. Med. Image Anal. 33, 170–175 (2016).
pubmed: 27423409
pmcid: 5556681
doi: 10.1016/j.media.2016.06.037
LeCun, Y., Bengio, Y. & Hinton, G. Deep learning. nature 521, 436–444 (2015).
pubmed: 26017442
doi: 10.1038/nature14539
Rivenson, Y. et al. Virtual histological staining of unlabelled tissue-autofluorescence images via deep learning. Nat. Biomed. Eng. 3, 466–477 (2019).
pubmed: 31142829
doi: 10.1038/s41551-019-0362-y
Zhang, Y. et al. Digital synthesis of histological stains using micro-structured and multiplexed virtual staining of label-free tissue. Light Sci. Appl. 9, 78 (2020).
pubmed: 32411363
pmcid: 7203145
doi: 10.1038/s41377-020-0315-y
de Haan, K. et al. Deep learning-based transformation of H&E stained tissues into special stains. Nat. Commun. 12, 4884 (2021).
pubmed: 34385460
pmcid: 8361203
doi: 10.1038/s41467-021-25221-2
Bai, B. et al. Label-free virtual HER2 immunohistochemical staining of breast tissue using deep learning. BME Front. 2022 (2022).
Zhang, Y. et al. Virtual staining of defocused autofluorescence images of unlabeled tissue using deep neural networks. Intell. Comput. 2022, 2022/9818965 (2022).
doi: 10.34133/2022/9818965
Bai, B. et al. Deep learning-enabled virtual histological staining of biological samples. Light Sci. Appl. 12, 57 (2023).
pubmed: 36864032
pmcid: 9981740
doi: 10.1038/s41377-023-01104-7
Kreiss, L. et al. Digital staining in optical microscopy using deep learning—a review. PhotoniX 4, 34 (2023).
doi: 10.1186/s43074-023-00113-4
Cao, R. et al. Label-free intraoperative histology of bone tissue via deep-learning-assisted ultraviolet photoacoustic microscopy. Nat. Biomed. Eng. 7, 124–134 (2023).
pubmed: 36123403
doi: 10.1038/s41551-022-00940-z
Abraham, T. M. et al. Label- and slide-free tissue histology using 3D epi-mode quantitative phase imaging and virtual hematoxylin and eosin staining. Optica 10, 1605–1618 (2023).
doi: 10.1364/OPTICA.502859
Pillar, N., Li, Y., Zhang, Y. & Ozcan, A. Virtual staining of non-fixed tissue histology. Mod. Pathol. 37, 100444 (2024).
Mayerich, D. et al. Stain-less staining for computed histopathology. Technology 3, 27–31 (2015).
pubmed: 26029735
pmcid: 4445956
doi: 10.1142/S2339547815200010
Rivenson, Y. et al. PhaseStain: the digital staining of label-free quantitative phase microscopy images using deep learning. Light Sci. Appl. 8, 23 (2019).
pubmed: 30728961
pmcid: 6363787
doi: 10.1038/s41377-019-0129-y
Li, Y. et al. Virtual histological staining of unlabeled autopsy tissue. Nat. Commun. 15, 1684 (2024).
pubmed: 38396004
pmcid: 10891155
doi: 10.1038/s41467-024-46077-2
Cone, B. et al. 72 Virtual stain overlay: a novel approach for evaluating tumor microenvironment (TME) specific expression of PD-L1 in whole slide images of lung cancer. J. Immunother. Cancer 11, https://doi.org/10.1136/jitc-2023-SITC2023.0072 (2023).
Patrick, D., Moghtader, J., Wang, H., de Haan, K. & Rivenson, Y. Deep Learning-Enabled Virtual H&E and Fluoro- Jade B Tissue Staining for Neuronal Degeneration. Society of Toxicologic Pathology 40th Annual Symposium, https://www.toxpath.org/AM2021/docs/2021-Poster-Abstracts.pdf (Baltimore, MD, 2021).
Liu, S. et al. Unpaired stain transfer using pathology-consistent constrained generative adversarial networks. IEEE Trans. Med. Imaging 40, 1977–1989 (2021).
pubmed: 33784619
doi: 10.1109/TMI.2021.3069874
Ghahremani, P. et al. Deep learning-inferred multiplex immunofluorescence for immunohistochemical image quantification. Nat. Mach. Intell. 4, 401–412 (2022).
pubmed: 36118303
pmcid: 9477216
doi: 10.1038/s42256-022-00471-x
Yang, X. et al. Virtual stain transfer in histology via cascaded deep neural networks. ACS Photonics 9, 3134–3143 (2022).
doi: 10.1021/acsphotonics.2c00932
Mirza, M. & Osindero, S. Conditional generative adversarial nets. Preprint at https://doi.org/10.48550/arXiv.1411.1784 (2014).
Kong, L., Lian, C., Huang, D., Hu, Y. & Zhou, Q. Breaking the dilemma of medical image-to-image translation. Adv. Neural Inf. Process. Syst. 34, 1964–1978 (2021).
Wang, H. et al. Deep learning enables cross-modality super-resolution in fluorescence microscopy. Nat. Methods 16, 103–110 (2019).
pubmed: 30559434
doi: 10.1038/s41592-018-0239-0
Howie, A. J., Brewer, D. B., Howell, D. & Jones, A. P. Physical basis of colors seen in Congo red-stained amyloid in polarized light. Lab. Invest. 88, 232–242 (2008).
pubmed: 18166974
doi: 10.1038/labinvest.3700714
Yakupova, E. I., Bobyleva, L. G., Vikhlyantsev, I. M. & Bobylev, A. G. Congo Red and amyloids: history and relationship. Biosci. Rep. 39, BSR20181415 (2019).
pubmed: 30567726
pmcid: 6331669
doi: 10.1042/BSR20181415
Wisniowski, B. & Wechalekar, A. Confirming the diagnosis of amyloidosis. Acta Haematol. 143, 312–321 (2020).
pubmed: 32544917
doi: 10.1159/000508022
Croce, A. C. & Bottiroli, G. Autofluorescence spectroscopy and imaging: a tool for biomedical research and diagnosis. Eur. J. Histochem. 58, 2461 (2014).
pubmed: 25578980
pmcid: 4289852
Edelstein, A. D. et al. Advanced methods of microscope control using μManager software. J. Biol. Methods 1, e10 (2014).
pubmed: 25606571
doi: 10.14440/jbm.2014.36
Redondo, R. et al. Autofocus evaluation for brightfield microscopy pathology. J. Biomed. Opt. 17, 036008–036008 (2012).
pubmed: 22502566
doi: 10.1117/1.JBO.17.3.036008
Zhu, J.-Y., Park, T., Isola, P. & Efros, A. A. Unpaired image-to-image translation using cycle-consistent adversarial networks. In: Proc. IEEE International Conference on Computer Vision. (Ikeuchi, K., Medioni, G. & Pelillo, M.) 2223–2232 (IEEE, 2017).
Bay, H., Tuytelaars, T. & Van Gool, L. SURF: speeded up robust features. In: Computer Vision—ECCV 2006 (eds Leonardis, A., Bischof, H. & Pinz, A.) Vol. 3951 404–417 (Springer Berlin Heidelberg, 2006).
Lowe, D. G. Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60, 91–110 (2004).
doi: 10.1023/B:VISI.0000029664.99615.94
Torr, P. H. & Zisserman, A. MLESAC: a new robust estimator with application to estimating image geometry. Comput. Vis. Image Underst. 78, 138–156 (2000).
doi: 10.1006/cviu.1999.0832
Saalfeld, S., Fetter, R., Cardona, A. & Tomancak, P. Elastic volume reconstruction from series of ultra-thin microscopy sections. Nat. Methods 9, 717–720 (2012).
pubmed: 22688414
doi: 10.1038/nmeth.2072
Schindelin, J. et al. Fiji: an open-source platform for biological-image analysis. Nat. Methods 9, 676–682 (2012).
pubmed: 22743772
doi: 10.1038/nmeth.2019
Paszke, A. et al. Pytorch: An imperative style, high-performance deep learning library. Adv. Neural Inf. Process. Syst. 32 (2019).
Zingman, I., Frayle, S., Tankoyeu, I., Sukhanov, S. & Heinemann, F. A comparative evaluation of image-to-image translation methods for stain transfer in histopathology. In: Medical Imaging with Deep Learning (eds Burgos, N., Petitjean, C. & Vakalopoulou, M.) 1509–1525 (PMLR, 2024).
Wang, Z., Simoncelli, E. P. & Bovik, A. C. Multiscale structural similarity for image quality assessment. In: Proc. 37th Asilomar Conference on Signals, Systems and Computers (ed. Jullien, G.) Vol. 2, 1398–1402 (IEEE, 2003).
Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B. & Hochreiter, S. Gans trained by a two time-scale update rule converge to a local Nash equilibrium. Adv. Neural Inf. Process. Syst. 30 (2017).
Landini, G., Martinelli, G. & Piccinini, F. Colour deconvolution: stain unmixing in histological imaging. Bioinformatics 37, 1485–1487 (2021).
pubmed: 32997742
doi: 10.1093/bioinformatics/btaa847
Ostu, N. A threshold selection method from gray-level histograms. IEEE Trans. SMC 9, 62 (1979).
Ahmadzadeh, A., Kempton, D. J., Chen, Y. & Angryk, R. A. Multiscale iou: A metric for evaluation of salient object detection with fine structures. In: 2021 IEEE International Conference on Image Processing (ICIP) (eds alZahir, S., Labeau, F. & Mock, K.) 684–688 (IEEE, 2021).
Goodfellow, I. et al. Generative adversarial nets. Adv. Neural Inf. Process. Syst. 27 (2014).
Arar, M., Ginger, Y., Danon, D., Bermano, A. H. & Cohen-Or, D. Unsupervised multi-modal image registration via geometry preserving image-to-image translation. In: Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (eds Boult, T., Medioni, G. & Zabih, R.) 13410–13419 (IEEE, 2020).
Balakrishnan, G., Zhao, A., Sabuncu, M. R., Guttag, J. & Dalca, A. V. VoxelMorph: a learning framework for deformable medical image registration. IEEE Trans. Med. Imaging 38, 1788–1800 (2019).
doi: 10.1109/TMI.2019.2897538
McNeil, C. et al. An end-to-end platform for digital pathology using hyperspectral autofluorescence microscopy and deep learning-based virtual histology. Mod. Pathol. 37, 100377 (2024).
pubmed: 37926422
doi: 10.1016/j.modpat.2023.100377
Huber, P. J. Robust estimation of a location parameter. Ann. Math. Stat. 35, 73–101 (1964).
doi: 10.1214/aoms/1177703732
Oktay, O. et al. Attention U-Net: Learning Where to Look for the Pancreas. Medical Imaging with Deep Learning. https://openreview.net/forum?id=Skft7cijM (2018).
Maas, A. L., Hannun, A. Y. & Ng, A. Y. Rectifier nonlinearities improve neural network acoustic models. In: Proc. ICML (ed. Littman, M.) Vol. 30, 3 (PMLR, Atlanta, GA, 2013).
He, K., Zhang, X., Ren, S. & Sun, J. Deep residual learning for image recognition. In: Proc. IEEE Conference on Computer Vision and Pattern Recognition (eds Bajcsy, R., Li, F. -F. & Tuytelaars, T.) 770–778 (IEEE, 2016).
Kingma, D. P. & Ba, J. Adam: A Method for stochastic optimization. International Conference on Learning Representations (ICLR) (San Diega, CA, USA, 2015).
Chiu, C.-L., Clack, N. & Community, T. N. napari: a Python multi-dimensional image viewer platform for the research community. Microsc. Microanal. 28, 1576–1577 (2022).
doi: 10.1017/S1431927622006328
Yang, X., Bai, B., Zhang, Y., Pillar, N. & Ozcan, A. Virtual birefringence imaging and histological staining of amyloid deposits in label-free tissue using autofluorescence microscopy and deep learning. Zenodo https://doi.org/10.5281/zenodo.13328896 (2024).