Image processing tools for petabyte-scale light sheet microscopy data.
Journal
Nature methods
ISSN: 1548-7105
Titre abrégé: Nat Methods
Pays: United States
ID NLM: 101215604
Informations de publication
Date de publication:
17 Oct 2024
17 Oct 2024
Historique:
received:
15
02
2024
accepted:
16
09
2024
medline:
18
10
2024
pubmed:
18
10
2024
entrez:
17
10
2024
Statut:
aheadofprint
Résumé
Light sheet microscopy is a powerful technique for high-speed three-dimensional imaging of subcellular dynamics and large biological specimens. However, it often generates datasets ranging from hundreds of gigabytes to petabytes in size for a single experiment. Conventional computational tools process such images far slower than the time to acquire them and often fail outright due to memory limitations. To address these challenges, we present PetaKit5D, a scalable software solution for efficient petabyte-scale light sheet image processing. This software incorporates a suite of commonly used processing tools that are optimized for memory and performance. Notable advancements include rapid image readers and writers, fast and memory-efficient geometric transformations, high-performance Richardson-Lucy deconvolution and scalable Zarr-based stitching. These features outperform state-of-the-art methods by over one order of magnitude, enabling the processing of petabyte-scale image data at the full teravoxel rates of modern imaging cameras. The software opens new avenues for biological discoveries through large-scale imaging experiments.
Identifiants
pubmed: 39420143
doi: 10.1038/s41592-024-02475-4
pii: 10.1038/s41592-024-02475-4
doi:
Types de publication
Journal Article
Langues
eng
Sous-ensembles de citation
IM
Subventions
Organisme : DOE | LDRD | Lawrence Berkeley National Laboratory (Berkeley Lab)
ID : 7647437
Organisme : DOE | LDRD | Lawrence Berkeley National Laboratory (Berkeley Lab)
ID : 7721359
Organisme : Alexander von Humboldt-Stiftung (Alexander von Humboldt Foundation)
ID : Feodor Lynen Research Fellowship
Organisme : California Institute for Regenerative Medicine (CIRM)
ID : EDUC4-12790
Informations de copyright
© 2024. The Author(s).
Références
Stelzer, E. H. K. et al. Light sheet fluorescence microscopy. Nat. Rev. Methods Primers 1, 73 (2021).
doi: 10.1038/s43586-021-00069-4
Holekamp, T. F., Turaga, D. & Holy, T. E. Fast three-dimensional fluorescence imaging of activity in neural populations by objective-coupled planar illumination microscopy. Neuron 57, 661–672 (2008).
doi: 10.1016/j.neuron.2008.01.011
pubmed: 18341987
Krzic, U., Gunther, S., Saunders, T. E., Streichan, S. J. & Hufnagel, L. Multiview light-sheet microscope for rapid in toto imaging. Nat. Methods 9, 730–733 (2012).
doi: 10.1038/nmeth.2064
pubmed: 22660739
Wu, Y. et al. Spatially isotropic four-dimensional imaging with dual-view plane illumination microscopy. Nat. Biotechnol. 31, 1032–1038 (2013).
pmcid: 4105320
doi: 10.1038/nbt.2713
pubmed: 24108093
Chen, B.-C. et al. Lattice light-sheet microscopy: imaging molecules to embryos at high spatiotemporal resolution. Science 346, 1257998 (2014).
pmcid: 4336192
doi: 10.1126/science.1257998
pubmed: 25342811
Dean, K. M., Roudot, P., Welf, E. S., Danuser, G. & Fiolka, R. Deconvolution-free subcellular imaging with axially swept light sheet microscopy. Biophys. J. 108, 2807–2815 (2015).
pmcid: 4472079
doi: 10.1016/j.bpj.2015.05.013
pubmed: 26083920
Chakraborty, T. et al. Light-sheet microscopy of cleared tissues with isotropic, subcellular resolution. Nat. Methods 16, 1109–1113 (2019).
pmcid: 6924633
doi: 10.1038/s41592-019-0615-4
pubmed: 31673159
Dunsby, C. Optically sectioned imaging by oblique plane microscopy. Opt. Express 16, 20306–20316 (2008).
doi: 10.1364/OE.16.020306
pubmed: 19065169
Sapoznik, E. et al. A versatile oblique plane microscope for large-scale and high-resolution imaging of subcellular dynamics. Elife 9, e57681 (2020).
pmcid: 7707824
doi: 10.7554/eLife.57681
pubmed: 33179596
Yang, B. et al. DaXi—high-resolution, large imaging volume and multi-view single-objective light-sheet microscopy. Nat. Methods 19, 461–469 (2022).
pmcid: 9007742
doi: 10.1038/s41592-022-01417-2
pubmed: 35314838
Chen, F., Tillberg, P. W. & Boyden, E. S. Expansion microscopy. Science 347, 543–548 (2015).
pmcid: 4312537
doi: 10.1126/science.1260088
pubmed: 25592419
Gao, R. et al. Cortical column and whole-brain imaging with molecular contrast and nanoscale resolution. Science 363, eaau8302 (2019).
pmcid: 6481610
doi: 10.1126/science.aau8302
pubmed: 30655415
Glaser, A. et al. Expansion-assisted selective plane illumination microscopy for nanoscale imaging of centimeter-scale tissues. eLife https://doi.org/10.7554/eLife.91979.1 (2023).
Aguet, F. et al. Membrane dynamics of dividing cells imaged by lattice light-sheet microscopy. Mol. Biol. Cell 27, 3418–3435 (2016).
pmcid: 5221578
doi: 10.1091/mbc.e16-03-0164
pubmed: 27535432
Lamb, J. R., Ward, E. N. & Kaminski, C. F. Open-source software package for on-the-fly deskewing and live viewing of volumetric lightsheet microscopy data. Biomed. Opt. Express 14, 834–845 (2023).
pmcid: 9979666
doi: 10.1364/BOE.479977
pubmed: 36874505
Schmid, B. & Huisken, J. Real-time multi-view deconvolution. Bioinformatics 31, 3398–3400 (2015).
pmcid: 4595906
doi: 10.1093/bioinformatics/btv387
pubmed: 26112291
Guo, M. et al. Rapid image deconvolution and multiview fusion for optical microscopy. Nat. Biotechnol. 38, 1337–1346 (2020).
pmcid: 7642198
doi: 10.1038/s41587-020-0560-x
pubmed: 32601431
Bria, A. & Iannello, G. TeraStitcher—a tool for fast automatic 3D-stitching of teravoxel-sized microscopy images. BMC Bioinformatics 13, 316 (2012).
pmcid: 3582611
doi: 10.1186/1471-2105-13-316
pubmed: 23181553
Hörl, D. et al. BigStitcher: reconstructing high-resolution image datasets of cleared and expanded samples. Nat. Methods 16, 870–874 (2019).
doi: 10.1038/s41592-019-0501-0
pubmed: 31384047
Schmid, B., Schindelin, J., Cardona, A., Longair, M. & Heisenberg, M. A high-level 3D visualization API for Java and ImageJ. BMC Bioinformatics 11, 274 (2010).
pmcid: 2896381
doi: 10.1186/1471-2105-11-274
pubmed: 20492697
Campagnola, L., Klein, A., Larson, E., Rossant, C. & Rougier, N. P. VisPy: harnessing the GPU for fast, high-level visualization. in Proceedings of the 14th Python in Science Conference (2015).
Miles, A. et al. zarr-developers/zarr-python: v2.16.1. Zenodo https://doi.org/10.5281/zenodo.8263439 (2023).
Zaharia, M. et al. Apache spark: a unified engine for big data processing. Commun. ACM 59, 56–65 (2016).
doi: 10.1145/2934664
Dask Development Team. Dask: library for dynamic task scheduling. https://www.dask.org/ (2016).
Leigh, R. et al. OME Files—an open source reference library for the OME-XML metadata model and the OME-TIFF file format. Preprint at bioRxiv https://doi.org/10.1101/088740 (2016).
Gohlke, C. cgohlke/tifffile: v2023.7.10. Zenodo https://doi.org/10.5281/zenodo.8133352 (2023).
Dagum, L. & Menon, R. OpenMP: an industry standard API for shared-memory programming. IEEE Comput. Sci. Eng. 5, 46–55 (1998).
doi: 10.1109/99.660313
Saalfeld, S. et al. saalfeldlab/n5: n5-2.5.1. Zenodo https://doi.org/10.5281/zenodo.6578232 (2022).
Moore, J. et al. OME-Zarr: a cloud-optimized bioimaging file format with international community support. Histochem. Cell Biol. 160, 223–251 (2023).
pmcid: 10492740
doi: 10.1007/s00418-023-02209-1
pubmed: 37428210
TensorStore developers, TensorStore: library for reading and writing large multi-dimensional arrays, version 0.1.51. https://github.com/google/tensorstore/ (2023).
ZSTD developers, ZSTD: Zstandard—fast real-time compression algorithm, version 1.5.6. https://github.com/facebook/zstd/ . Accessed 7 July 2024.
LZ4 developers, LZ4: extremely fast compression algorithm, version 1.9.4. https://github.com/lz4/lz4 . Accessed 7 July 2024.
Schindelin, J. et al. Fiji: an open-source platform for biological-image analysis. Nat. Methods 9, 676–682 (2012).
doi: 10.1038/nmeth.2019
pubmed: 22743772
Maioli, V. A. High-speed 3-D fluorescence imaging by oblique plane microscopy: multi-well plate-reader development, biological applications and image analysis, PhD thesis, Imperial College London, 2017.
Haase, R. et al. clEsperanto/pyclesperanto_prototype: 0.24.1. Zenodo https://doi.org/10.5281/zenodo.7827755 (2023).
Djutanta, F. et al. Decoding the hydrodynamic properties of microscale helical propellers from Brownian fluctuations. Proc. Natl Acad. Sci. USA 120, e2220033120 (2023).
pmcid: 10235983
doi: 10.1073/pnas.2220033120
pubmed: 37235635
Liu, G. et al. Characterization, comparison, and optimization of lattice light sheets. Sci. Adv. 9, eade6623 (2023).
pmcid: 10065451
doi: 10.1126/sciadv.ade6623
pubmed: 37000868
Richardson, W. H. Bayesian-based iterative method of image restoration. J. Opt. Soc. Am. 62, 55–59 (1972).
doi: 10.1364/JOSA.62.000055
Lucy, L. B. An iterative technique for the rectification of observed distributions. Astron. J. 79, 745–754 (1974).
doi: 10.1086/111605
Biggs, D. S. C. & Andrews, M. Acceleration of iterative image restoration algorithms. Appl. Opt. 36, 1766–1775 (1997).
doi: 10.1364/AO.36.001766
pubmed: 18250863
Zeng, G. L. & Gullberg, G. T. Unmatched projector/backprojector pairs in an iterative reconstruction algorithm. IEEE Trans. Med. Imaging 19, 548–555 (2000).
pmcid: 5297459
doi: 10.1109/42.870265
pubmed: 11021698
Koho, S. et al. Fourier ring correlation simplifies image restoration in fluorescence microscopy. Nat. Commun. 10, 3103 (2019).
pmcid: 6629685
doi: 10.1038/s41467-019-11024-z
pubmed: 31308370
Briechle, K. & Hanebeck, U. D. Template matching using fast normalized cross correlation. in Proceedings of SPIE: Optical Pattern Recognition XII, Vol. 4387, pp. 95–102 (2001).
Szeliski, R. Image alignment and stitching: a tutorial. in Foundations and Trends in Computer Graphics and Vision 2, 1–104 (2007).
doi: 10.1561/0600000009
NVIDIA IndeX developers. NVIDIA IndeX: 3D scientific data visualization, version 0.20.2. https://developer.nvidia.com/index/ (2023).
Li, J., Wang, Z., Lai, S., Zhai, Y. & Zhang, M. Parallax-tolerant image stitching based on robust elastic warping. IEEE Trans. Multimedia 20, 1672–1687 (2017).
doi: 10.1109/TMM.2017.2777461
Liu, Z. et al. A survey on applications of deep learning in microscopy image analysis. Comput. Biol. Med. 134, 104523 (2021).
doi: 10.1016/j.compbiomed.2021.104523
pubmed: 34091383
Melanthota, S. K. et al. Deep learning-based image processing in optical microscopy. Biophys. Rev. 14, 463–481 (2022).
pmcid: 9043085
doi: 10.1007/s12551-022-00949-3
pubmed: 35528030
Volpe, G. et al. Roadmap on deep learning for microscopy. Preprint at https://arxiv.org/abs/2303.03793 (2023).
Li, Y. et al. Incorporating the image formation process into deep learning improves network performance. Nat. Methods 19, 1427–1437 (2022).
pmcid: 9636023
doi: 10.1038/s41592-022-01652-7
pubmed: 36316563
Laine, R. F., Jacquemet, G. & Krull, A. Imaging in focus: an introduction to denoising bioimages in the era of deep learning. Int. J. Biochem. Cell Biol. 140, 106077 (2021).
pmcid: 8552122
doi: 10.1016/j.biocel.2021.106077
pubmed: 34547502
Fu, Y. et al. Deep learning in medical image registration: a review. Physics Med. Biol. 65, 20TR01 (2020).
doi: 10.1088/1361-6560/ab843e
Wang, Y. & Jeon, H. 3D cell cultures toward quantitative high-throughput drug screening. Trends Pharmacol. Sci. 43, 569–581 (2022).
Ståhl, P. L. et al. Visualization and analysis of gene expression in tissue sections by spatial transcriptomics. Science 353, 78–82 (2016).
doi: 10.1126/science.aaf2403
pubmed: 27365449
Liu, T. -L. et al. Observing the cell in its native state: Imaging subcellular dynamics in multicellular organisms. Science 360, eaaq1392 (2018).
pmcid: 6040645
doi: 10.1126/science.aaq1392
pubmed: 29674564
Wan, Y., McDole, K. & Keller, P. J. Light-sheet microscopy and its potential for understanding developmental processes. Annu. Rev. Cell Dev. Biol. 35, 655–681 (2019).
doi: 10.1146/annurev-cellbio-100818-125311
pubmed: 31299171
Blosc Development Team. Blosc: a blocking, shuffling and lossless compression library, version 1.21.5. https://github.com/Blosc/c-blosc/ . Accessed 26 December 2023.
van Heel, M. & Schatz, M. Fourier shell correlation threshold criteria. J. Struct. Biol. 151, 250–262 (2005).
doi: 10.1016/j.jsb.2005.05.009
pubmed: 16125414
Peng, T. et al. A BaSiC tool for background and shading correction of optical microscopy images. Nat. Commun. 8, 14836 (2017).
pmcid: 5472168
doi: 10.1038/ncomms14836
pubmed: 28594001
Wang, K. et al. Rapid adaptive optical recovery of optimal resolution over large volumes. Nat. Methods 11, 625–628 (2014).
pmcid: 4069208
doi: 10.1038/nmeth.2925
pubmed: 24727653
Diao, F. et al. Plug-and-play genetic access to Drosophila cell types using exchangeable exon cassettes. Cell Rep. 10, 1410–1421 (2015).
pmcid: 4373654
doi: 10.1016/j.celrep.2015.01.059
pubmed: 25732830
Pfeiffer, B. D., Truman, J. W. & Rubin, G. M. Using translational enhancers to increase transgene expression in Drosophila. Proc. Natl Acad. Sci. USA 109, 6626–6631 (2012).
pmcid: 3340069
doi: 10.1073/pnas.1204520109
pubmed: 22493255
Lillvis, J. L. et al. Rapid reconstruction of neural circuits using tissue expansion and light sheet microscopy. Elife 11, e81248 (2022).
pmcid: 9651950
doi: 10.7554/eLife.81248
pubmed: 36286237
Hanisch, R. J., White, R. L. & Gilliland, R. L. Deconvolution of Hubble Space Telescope images and spectra. in Deconvolution of Images and Spectra, Vol. 2 (ed Jansson, P. A.) 310–360 (Academic Press, 1997).
Ruan, X. et al. Data for: Image processing tools for petabyte-scale light sheet microscopy data (part 1/2). Dryad https://doi.org/10.5061/dryad.kh18932g4 (2024).
Ruan, X. et al. Data for: Image processing tools for petabyte-scale light sheet microscopy data (part 2/2). Dryad https://doi.org/10.5061/dryad.jq2bvq8jd (2024).
Ruan, X., Mueller, M., Betzig, E., & Upadhyayula, S. abcucberkeley/PetaKit5D: v1.2.0. Zenodo https://doi.org/10.5281/zenodo.13686337 (2024).
Mueller, M., Ruan, X., Betzig, E., & Upadhyayula, S. abcucberkeley/Parallel_Fiji_Visualizer: v1.2.1. Zenodo https://doi.org/10.5281/zenodo.11516647 (2024).
Ruan, X., Mueller, M., Betzig, E. & Upadhyayula, S. Benchmark code for the paper "image processing tools for petabyte-scale light sheet microscopy data". Zenodo https://doi.org/10.5281/zenodo.13690716 (2024).
Ruan, X. et al. Example code and data for visualizing 3D time-series microscopy data with Nvidia IndeX. Zenodo https://doi.org/10.5281/zenodo.12539580 (2024).