DBlink: dynamic localization microscopy in super spatiotemporal resolution via deep learning.
Journal
Nature methods
ISSN: 1548-7105
Titre abrégé: Nat Methods
Pays: United States
ID NLM: 101215604
Informations de publication
Date de publication:
Dec 2023
Dec 2023
Historique:
received:
22
08
2022
accepted:
26
06
2023
pubmed:
28
7
2023
medline:
28
7
2023
entrez:
27
7
2023
Statut:
ppublish
Résumé
Single-molecule localization microscopy (SMLM) has revolutionized biological imaging, improving the spatial resolution of traditional microscopes by an order of magnitude. However, SMLM techniques require long acquisition times, typically a few minutes, to yield a single super-resolved image, because they depend on accumulation of many localizations over thousands of recorded frames. Hence, the capability of SMLM to observe dynamics at high temporal resolution has always been limited. In this work, we present DBlink, a deep-learning-based method for super spatiotemporal resolution reconstruction from SMLM data. The input to DBlink is a recorded video of SMLM data and the output is a super spatiotemporal resolution video reconstruction. We use a convolutional neural network combined with a bidirectional long short-term memory network architecture, designed for capturing long-term dependencies between different input frames. We demonstrate DBlink performance on simulated filaments and mitochondria-like structures, on experimental SMLM data under controlled motion conditions and on live-cell dynamic SMLM. DBlink's spatiotemporal interpolation constitutes an important advance in super-resolution imaging of dynamic processes in live cells.
Identifiants
pubmed: 37500760
doi: 10.1038/s41592-023-01966-0
pii: 10.1038/s41592-023-01966-0
doi:
Types de publication
Journal Article
Langues
eng
Sous-ensembles de citation
IM
Pagination
1939-1948Subventions
Organisme : EC | Horizon 2020 Framework Programme (EU Framework Programme for Research and Innovation H2020)
ID : 802567
Organisme : Deutsche Forschungsgemeinschaft (German Research Foundation)
ID : SFB1177; INST 161/926-1 FUGG
Informations de copyright
© 2023. The Author(s), under exclusive licence to Springer Nature America, Inc.
Références
Hell, S. W. & Wichmann, J. Breaking the diffraction resolution limit by stimulated-emission—stimulated-emission-depletion fluorescence microscopy. Opt. Lett. 19, 780–782 (1994).
pubmed: 19844443
Gustafsson, M. G. L. Surpassing the lateral resolution limit by a factor of two using structured illumination microscopy. J. Microsc. 198, 82–87 (2000).
pubmed: 10810003
Sauer, M. & Heilemann, M. Single-molecule localization microscopy in eukaryotes. Chem. Rev. 117, 7478–7509 (2017).
pubmed: 28287710
Betzig, E. et al. Imaging intracellular fluorescent proteins at nanometer resolution. Science 313, 1642–1645 (2006).
pubmed: 16902090
Rust, M. J., Bates, M. & Zhuang, X. Sub-diffraction-limit imaging by stochastic optical reconstruction microscopy (STORM). Nat. Methods 3, 793–795 (2006).
pubmed: 16896339
pmcid: 2700296
Sharonov, A. & Hochstrasser, R. Wide-field subdiffraction imaging by accumulated binding of diffusing probes. Proc. Natl Acad. Sci. USA 103, 18911–18916 (2006).
pubmed: 17142314
pmcid: 1748151
Schnitzbauer, J., Strauss, M. T., Schlichthaerle, T., Schueder, F. & Jungmann, R. Super-resolution microscopy with DNA-PAINT. Nat. Protoc. 12, 1198–1228 (2017).
pubmed: 28518172
Ouyang, W., Aristov, A., Lelek, M., Hao, X. & Zimmer, C. Deep learning massively accelerates super-resolution localization microscopy. Nat. Biotechnol. 36, 460–468 (2018).
pubmed: 29658943
Wang, Y. et al. Blind sparse inpainting reveals cytoskeletal filaments with sub-Nyquist localization. Optica 4, 1277–1284 (2017).
pubmed: 30320156
pmcid: 6179357
Barentine, A. E. S. et al. An integrated platform for high-throughput nanoscopy. Nat. Biotechnol. https://doi.org/10.1038/s41587-023-01702-1 (2023).
Liu, Z., Lavis, L. D. & Betzig, E. Imaging live-cell dynamics and structure at the single-molecule level. Mol. Cell 58, 644–659 (2015).
pubmed: 26000849
Nehme, E., Weiss, L. E., Michaeli, T. & Schechtman, Y. Deep-STORM: super-resolution single-molecule microscopy by deep learning.Optica 5, 458–464 (2018).
Nehme, E. et al. DeepSTORM3D: dense 3D localization microscopy and PSF design by deep learning. Nat. Methods 17, 734–740 (2020).
pubmed: 32541853
pmcid: 7610486
Speiser, A. et al. Deep learning enables fast and dense single-molecule localization with high accuracy. Nat. Methods 18, 1082–1090 (2021).
pubmed: 34480155
pmcid: 7611669
Nixon-Abell, J. et al. Increased spatiotemporal resolution reveals highly dynamic dense tubular matrices in the peripheral ER. Science 354, aaf3928 (2016).
pubmed: 27789813
pmcid: 6528812
Priessner, M. et al. Content-aware frame interpolation (CAFI): Deep Learning-based temporal super-resolution for fast bioimaging. Preprint at bioRxiv https://doi.org/10.1101/2021.11.02.466664 (2021).
Chen, R. et al. Single-frame deep-learning super-resolution microscopy for intracellular dynamics imaging. Nat. Commun. 14, 2854 (2023).
pubmed: 37202407
pmcid: 10195829
Dertinger, T., Colyer, R., Iyer, G., Weiss, S. & Enderlein, J. Fast, background-free, 3D super-resolution optical fluctuation imaging (SOFI). Proc. Natl Acad. Sci. USA 106, 22287–22292 (2009).
pubmed: 20018714
pmcid: 2799731
Gustafsson, N. et al. Fast live-cell conventional fluorophore nanoscopy with ImageJ through super-resolution radial fluctuations. Nat. Commun. 7, 12471 (2016).
pubmed: 27514992
pmcid: 4990649
Agarwal, K. & Macháň, R. Multiple signal classification algorithm for super-resolution fluorescence microscopy. Nat. Commun. 7, 13752 (2016).
pubmed: 27934858
pmcid: 5155148
Laine, R. F. et al. High-fidelity 3D live-cell nanoscopy through data-driven enhanced super-resolution radial fluctuation. Preprint at bioRxiv https://doi.org/10.1101/2022.04.07.487490 (2022).
Ovesný, M., Křížek, P., Borkovec, J., Švindrych, Z. & Hagen, G. M. ThunderSTORM: a comprehensive ImageJ plug-in for PALM and STORM data analysis and super-resolution imaging. Bioinformatics 30, 2389–2390 (2014).
pubmed: 24771516
pmcid: 4207427
Su, Y.-T., Lu, Y., Chen, M. & Liu, A.-A. Spatiotemporal joint mitosis detection using CNN-LSTM network in time-lapse phase contrast microscopy images. IEEE Access 5, 18033–18041 (2017).
Yu, Y., Si, X., Hu, C. & Zhang, J. A review of recurrent neural networks: LSTM cells and network architectures. Neural Comput. 31, 1235–1270 (2019).
pubmed: 31113301
Shariff, A., Murphy, R. F. & Rohde, G. K. A generative model of microtubule distributions, and indirect estimation of its parameters from fluorescence microscopy images. Cytometry A 77, 457–466 (2010).
pubmed: 20104579
pmcid: 2901542
Banterle, N., Bui, K. H., Lemke, E. A. & Beck, M. Fourier ring correlation as a resolution criterion for super-resolution microscopy. J. Struct. Biol. 183, 363–367 (2013).
pubmed: 23684965
Descloux, A., Grußmayer, K. S. & Radenovic, A. Parameter-free image resolution estimation based on decorrelation analysis. Nat. Methods 16, 918–924 (2019).
pubmed: 31451766
Niekamp, S., Coudray, N., Zhang, N., Vale, R. D. & Bhabha, G. Coupling of ATPase activity, microtubule binding, and mechanics in the dynein motor domain. EMBO J. 38, e101414 (2019).
pubmed: 31268607
pmcid: 6600642
Kompa, J. et al. Exchangeable HaloTag ligands for super-resolution fluorescence microscopy. J. Am. Chem. Soc. 145, 3075–3083 (2023).
pubmed: 36716211
pmcid: 9912333
Lefebvre, A. E. Y. T., Ma, D., Kessenbrock, K., Lawson, D. A. & Digman, M. A. Automated segmentation and tracking of mitochondria in live-cell time-lapse images. Nat. Methods 18, 1091–1102 (2021).
pubmed: 34413523
Yang, X. et al. Mitochondrial dynamics quantitatively revealed by STED nanoscopy with an enhanced squaraine variant probe. Nat. Commun. 11, 3699 (2020).
pubmed: 32709877
pmcid: 7382495
Friedman, J. R. & Nunnari, J. Mitochondrial form and function. Nature 505, 335–343 (2014).
pubmed: 24429632
pmcid: 4075653
Quiles, J. M. & Gustafsson, Å. B. The role of mitochondrial fission in cardiovascular health and disease. Nat. Rev. Cardiol. 19, 723–736 (2022).
pubmed: 35523864
pmcid: 10584015
Kleele, T. et al. Distinct fission signatures predict mitochondrial degradation or biogenesis. Nature 593, 435–439 (2021).
pubmed: 33953403
Tachibana, R. et al. Design of spontaneously blinking fluorophores for live-cell super-resolution imaging based on quantum-chemical calculations. Chem. Commun. 56, 13173–13176 (2020).
Möckl, L., Roy, A. R. & Moerner, W. E. Deep learning in single-molecule microscopy: fundamentals, caveats, and recent developments [Invited]. Biomed. Opt. Express 11, 1633 (2020).
pubmed: 32206433
pmcid: 7075610
Matlock, A., Zhu, J. & Tian, L. Multiple-scattering simulator-trained neural network for intensity diffraction tomography. Opt. Express 31, 4094–4107 (2023).
pubmed: 36785385
Belthangady, C. & Royer, L. A. Applications, promises, and pitfalls of deep learning for fluorescence image reconstruction. Nat. Methods 16, 1215–1225 (2019).
pubmed: 31285623
Spahn, C., Grimm, J. B., Lavis, L. D., Lampe, M. & Heilemann, M. Whole-Cell, 3D, and multicolor STED imaging with exchangeable fluorophores. Nano Lett. 19, 500–505 (2019).
pubmed: 30525682
Vaswani, A. et al. Attention is all you need. Adv. Neural Inf. Process. Syst. 2017, 5998–6008 (2017).
Dosovitskiy, A. et al. An image is worth 16x16 words: transformers for image recognition at scale. Preprint at https://doi.org/10.48550/arXiv.2010.11929 (2020).
Wensel, T. G., Potter, V. L., Moye, A., Zhang, Z. & Robichaux, M. A. Structure and dynamics of photoreceptor sensory cilia. Pflug. Arch. 473, 1517–1537 (2021).
Guggenheim, E. J. et al. Comparison of confocal and super-resolution reflectance imaging of metal oxide nanoparticles. PLoS ONE 11, e0159980 (2016).
pubmed: 27695038
pmcid: 5047631
van der Zwaag, D. et al. Super resolution imaging of nanoparticles cellular uptake and trafficking. ACS Appl. Mater. Interfaces 8, 6391–6399 (2016).
pubmed: 26905516
Pujals, S., Feiner-Gracia, N., Delcanale, P., Voets, I. & Albertazzi, L. Super-resolution microscopy as a powerful tool to study complex synthetic materials. Nat. Rev. Chem. 3, 68–84 (2019).
Valli, J. et al. Seeing beyond the limit: a guide to choosing the right super-resolution microscopy technique. J. Biol. Chem. 297, 100791 (2021).
pubmed: 34015334
pmcid: 8246591
Mahecic, D. et al. Mitochondrial membrane tension governs fission. Cell Rep. 35, 108947 (2021).
pubmed: 33852852
Mahecic, D. et al. Event-driven acquisition for content-enriched microscopy. Nat. Methods 19, 1262–1267 (2022).
pubmed: 36076039