Overcoming photon and spatiotemporal sparsity in fluorescence lifetime imaging with SparseFLIM.
Journal
Communications biology
ISSN: 2399-3642
Titre abrégé: Commun Biol
Pays: England
ID NLM: 101719179
Informations de publication
Date de publication:
21 Oct 2024
21 Oct 2024
Historique:
received:
20
03
2024
accepted:
15
10
2024
medline:
22
10
2024
pubmed:
22
10
2024
entrez:
21
10
2024
Statut:
epublish
Résumé
Fluorescence lifetime imaging microscopy (FLIM) provides quantitative readouts of biochemical microenvironments, holding great promise for biomedical imaging. However, conventional FLIM relies on slow photon counting routines to accumulate sufficient photon statistics, restricting acquisition speeds. Here we demonstrate SparseFLIM, an intelligent paradigm for achieving high-fidelity FLIM reconstruction from sparse photon measurements. We develop a coupled bidirectional propagation network that enriches photon counts and recovers hidden spatial-temporal information. Quantitative analysis shows over tenfold photon enrichment, dramatically improving signal-to-noise ratio, lifetime accuracy, and correlation compared to the original sparse data. SparseFLIM enables reconstructing spatially and temporally undersampled FLIM at full resolution and channel count. The model exhibits strong generalization across experimental modalities including multispectral FLIM and in vivo endoscopic FLIM. This work establishes deep learning as a promising approach to enhance fluorescence lifetime imaging and transcend limitations imposed by the inherent codependence between measurement duration and information content.
Identifiants
pubmed: 39433929
doi: 10.1038/s42003-024-07080-x
pii: 10.1038/s42003-024-07080-x
doi:
Types de publication
Journal Article
Langues
eng
Sous-ensembles de citation
IM
Pagination
1359Subventions
Organisme : National Natural Science Foundation of China (National Science Foundation of China)
ID : 62225505/61935012/ 62175163/61835009/62127819
Organisme : Natural Science Foundation of Guangdong Province (Guangdong Natural Science Foundation)
ID : 2024A1515010009
Informations de copyright
© 2024. The Author(s).
Références
Walsh, A. J. et al. Optical metabolic imaging identifies glycolytic levels, subtypes, and early-treatment response in breast cancer. Cancer Res. 73, 6164–6174 (2013).
doi: 10.1158/0008-5472.CAN-13-0527
pubmed: 24130112
pmcid: 3801432
Kantelhardt, S. R. et al. In vivo multiphoton tomography and fluorescence lifetime imaging of human brain tumor tissue. J. Neuro-Oncol. 127, 473–482 (2016).
doi: 10.1007/s11060-016-2062-8
Luo, T., Lu, Y., Liu, S., Lin, D. & Qu, J. J. A. C. Phasor-FLIM as a screening tool for the differential diagnosis of actinic keratosis, Bowen’s disease and basal cell carcinoma. Anal. Chem. 89, 8104–8111 (2017).
doi: 10.1021/acs.analchem.7b01681
pubmed: 28661125
Wang, M. Y. et al. Rapid diagnosis and intraoperative margin assessment of human lung cancer with fluorescence lifetime imaging microscopy. BBA Clin. 8, 7–13 (2017).
doi: 10.1016/j.bbacli.2017.04.002
pubmed: 28567338
pmcid: 5447569
Bower, A. J. et al. High-speed imaging of transient metabolic dynamics using two-photon fluorescence lifetime imaging microscopy. Optica 5, 1290–1296 (2018).
doi: 10.1364/OPTICA.5.001290
pubmed: 30984802
pmcid: 6457362
Shen, B. L. et al. Label-free whole-colony imaging and metabolic analysis of metastatic pancreatic cancer by an autoregulating flexible optical system. Theranostics 10, 1849–1860 (2020).
doi: 10.7150/thno.40869
pubmed: 32042340
pmcid: 6993220
Becker, W., Bergmann, A., Koenig, K. & Tirlapur, U. Picosecond fluorescence lifetime microscopy by TCSPC imaging, Vol. 4262. (SPIE, 2001).
Becker, W. et al. Fluorescence lifetime imaging by time-correlated single-photon counting. Microsc. Res. Tech. 63, 58–66 (2004).
Skala, M. et al. In vivo multiphoton fluorescence lifetime imaging of protein-bound and free nicotinamide adenine dinucleotide in normal and precancerous epithelia. J. Biomed. Opt. 12, 024014 (2007).
doi: 10.1117/1.2717503
pubmed: 17477729
Bowman, A. J., Klopfer, B. B., Juffmann, T. & Kasevich, M. A. Electro-optic imaging enables efficient wide-field fluorescence lifetime microscopy. Nat. Commun. 10, 4561 (2019).
doi: 10.1038/s41467-019-12535-5
pubmed: 31594938
pmcid: 6783475
Ulku, A. et al. Wide-field time-gated SPAD imager for phasor-based FLIM applications. Methods Appl. Fluoresc. 8, 024002 (2020).
doi: 10.1088/2050-6120/ab6ed7
pubmed: 31968310
pmcid: 8827132
Samimi, K. et al. Light-sheet autofluorescence lifetime imaging with a single-photon avalanche diode array. J. Biomed. Opt. 28, 066502 (2023).
doi: 10.1117/1.JBO.28.6.066502
pubmed: 37351197
pmcid: 10284079
Hirvonen, L. M. et al. Lightsheet fluorescence lifetime imaging microscopy with wide-field time-correlated single photon counting. J. Biophoton. 13, e201960099 (2020).
doi: 10.1002/jbio.201960099
Zhang, Y. et al. Instant FLIM enables 4D in vivo lifetime imaging of intact and injured zebrafish and mouse brains. Optica 8, 885–897 (2021).
doi: 10.1364/OPTICA.426870
Raspe, M. et al. siFLIM: single-image frequency-domain FLIM provides fast and photon-efficient lifetime data. Nat. Methods 13, 501–504 (2016).
doi: 10.1038/nmeth.3836
pubmed: 27088314
Li, X. et al. Real-time denoising enables high-sensitivity fluorescence time-lapse imaging beyond the shot-noise limit. Nat. Biotechnol. 41, 282–292 (2023).
doi: 10.1038/s41587-022-01450-8
pubmed: 36163547
Mannam, V. et al. Real-time image denoising of mixed Poisson–Gaussian noise in fluorescence microscopy images using ImageJ. Optica 9, 335–345 (2022).
doi: 10.1364/OPTICA.448287
Jin, L. B. et al. Deep learning extended depth-of-field microscope for fast and slide-free histology. Proc. Natl Acad. Sci. USA 117, 33051–33060 (2020).
doi: 10.1073/pnas.2013571117
pubmed: 33318169
pmcid: 7776814
Weigert, M. et al. Content-aware image restoration: pushing the limits of fluorescence microscopy. Nat. Methods 15, 1090–1097 (2018).
doi: 10.1038/s41592-018-0216-7
pubmed: 30478326
Chen, J. J. et al. Three-dimensional residual channel attention networks denoise and sharpen fluorescence microscopy image volumes. Nat. Methods 18, 678–687 (2021).
doi: 10.1038/s41592-021-01155-x
pubmed: 34059829
Smith, J. T. et al. Fast fit-free analysis of fluorescence lifetime imaging via deep learning. Proc. Natl Acad. Sci. USA 116, 24019–24030 (2019).
doi: 10.1073/pnas.1912707116
pubmed: 31719196
pmcid: 6883809
Xiao, D., Chen, Y. & Li, D. D. U. One-dimensional deep learning architecture for fast fluorescence lifetime imaging. IEEE J. Sel. Top. Quantum Electron. 27, 1–10 (2021).
doi: 10.1109/JSTQE.2021.3049349
pubmed: 33154613
Chen, Y.-I. et al. Generative adversarial network enables rapid and robust fluorescence lifetime image analysis in live cells. Commun. Biol. 5, 18 (2022).
doi: 10.1038/s42003-021-02938-w
pubmed: 35017629
pmcid: 8752789
Ochoa, M. et al. High compression deep learning based single-pixel hyperspectral macroscopic fluorescence lifetime imaging in vivo. Biomed. Opt. Express 11, 5401–5424 (2020).
doi: 10.1364/BOE.396771
pubmed: 33149959
pmcid: 7587256
Mannam, V., Zhang, Y. D., Yuan, X. T., Ravasio, C. & Howard, S. S. Machine learning for faster and smarter fluorescence lifetime imaging microscopy. J. Phys. Photonics 2, 042005 (2020).
doi: 10.1088/2515-7647/abac1a
Xiao, D., Sapermsap, N., Chen, Y. & Li, D. D. U. Deep learning enhanced fast fluorescence lifetime imaging with a few photons. Optica 10, 944–951 (2023).
doi: 10.1364/OPTICA.491798
Skala, M. C. et al. In vivo multiphoton microscopy of NADH and FAD redox states, fluorescence lifetimes, and cellular morphology in precancerous epithelia. Proc. Natl Acad. Sci. USA 104, 19494–19499 (2007).
doi: 10.1073/pnas.0708425104
pubmed: 18042710
pmcid: 2148317
Ranjit, S. et al. Measuring the effect of a Western diet on liver tissue architecture by FLIM autofluorescence and harmonic generation microscopy. Biomed. Opt. Expr. 8, 3143–3154 (2017).
doi: 10.1364/BOE.8.003143
Chan, K. C. K., Wang, X., Yu, K., Dong, C. & Loy, C. C. BasicVSR: The Search for Essential Components in Video Super-Resolution and Beyond. in Proc. 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 4945–4954 (2021).
Wang, X. T. et al. EDVR: Video Restoration with Enhanced Deformable Convolutional Networks. in Proc. 2019 Ieee/Cvf Conference on Computer Vision and Pattern Recognition Workshops 1954-1963 (IEEE, Long Beach; 2019).
Gao, D. et al. FLIMJ: An open-source ImageJ toolkit for fluorescence lifetime image data analysis. Plos One 15, e0238327 (2021).
doi: 10.1371/journal.pone.0238327
Çiçek, Ö., Abdulkadir, A., Lienkamp, S. S., Brox, T. & Ronneberger, O. 3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation in Medical Image Computing and Computer-Assisted Intervention—MICCAI 2016. (eds. S. Ourselin, L. Joskowicz, M. R. Sabuncu, G. Unal & W. Wells) 424–432 (Springer International Publishing, Cham; 2016).
Lehtinen, J. et al. Noise2Noise: Learning Image Restoration without Clean Data. International Conference on Machine Learning. vol. 80 (PMLR, 2018).
Lin, H. N. et al. Microsecond fingerprint stimulated Raman spectroscopic imaging by ultrafast tuning and spatial-spectral learning. Nat. Commun. 12, 3052 (2021).
doi: 10.1038/s41467-021-23202-z
pubmed: 34031374
pmcid: 8144602
Zhang, Y. et al. Image Super-Resolution Using Very Deep Residual Channel Attention Networks in Computer Vision—ECCV 2018. (eds. V. Ferrari, M. Hebert, C. Sminchisescu & Y. Weiss) 294-310 (Springer International Publishing, Cham; 2018).
Shi, W. et al. Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network in 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 1874–1883 (2016).
Koho, S. et al. Fourier ring correlation simplifies image restoration in fluorescence microscopy. Nat. Commun. 10, 3103 (2019).
doi: 10.1038/s41467-019-11024-z
pubmed: 31308370
pmcid: 6629685
Williams, G. O. S. et al. Full spectrum fluorescence lifetime imaging with 0.5 nm spectral and 50 ps temporal resolution. Nat. Commun. 12, 6616 (2021).
doi: 10.1038/s41467-021-26837-0
pubmed: 34785666
pmcid: 8595732
Pian, Q., Yao, R., Sinsuebphon, N. & Intes, X. Compressive hyperspectral time-resolved wide-field fluorescence lifetime imaging. Nat. Photonics 11, 411–414 (2017).
doi: 10.1038/nphoton.2017.82
pubmed: 29242714
pmcid: 5726531
Popleteeva, M. et al. Fast and simple spectral FLIM for biochemical and medical imaging. Opt. Express 23, 23511–23525 (2015).
doi: 10.1364/OE.23.023511
pubmed: 26368450
Coda, S., Siersema, P. D., Stamp, G. W. H. & Thillainayagam, A. V. Biophotonic endoscopy: a review of clinical research techniques for optical imaging and sensing of early gastrointestinal cancer. Endosc. Int. Open 03, E380–E392 (2015).
doi: 10.1055/s-0034-1392513
Fruhwirth, G. O. et al. Fluorescence lifetime endoscopy using TCSPC for the measurement of FRET in live cells. Opt. Express 18, 11148–11158 (2010).
doi: 10.1364/OE.18.011148
pubmed: 20588974
pmcid: 3408954
Lin, F. et al. In vivo two-photon fluorescence lifetime imaging microendoscopy based on fiber-bundle. Opt. Lett. 47, 2137–2140 (2022).
doi: 10.1364/OL.453102
pubmed: 35486743
Chan, K. C. K., Zhou, S., Xu, X. & Loy, C. C. BasicVSR++: Improving video super-resolution with enhanced propagation and alignment in 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 5962–5971 (2022).
Cortinas-Lorenzo, B. & Perez-Gonzalez, F. Adam and the Ants: on the influence of the optimization algorithm on the detectability of DNN watermarks. Entropy 22, 1379 (2020).
doi: 10.3390/e22121379
pubmed: 33279925
pmcid: 7762180