Modular segmentation, spatial analysis and visualization of volume electron microscopy datasets.
Journal
Nature protocols
ISSN: 1750-2799
Titre abrégé: Nat Protoc
Pays: England
ID NLM: 101284307
Informations de publication
Date de publication:
29 Feb 2024
29 Feb 2024
Historique:
received:
07
03
2023
accepted:
24
11
2023
medline:
1
3
2024
pubmed:
1
3
2024
entrez:
29
2
2024
Statut:
aheadofprint
Résumé
Volume electron microscopy is the method of choice for the in situ interrogation of cellular ultrastructure at the nanometer scale, and with the increase in large raw image datasets generated, improving computational strategies for image segmentation and spatial analysis is necessary. Here we describe a practical and annotation-efficient pipeline for organelle-specific segmentation, spatial analysis and visualization of large volume electron microscopy datasets using freely available, user-friendly software tools that can be run on a single standard workstation. The procedures are aimed at researchers in the life sciences with modest computational expertise, who use volume electron microscopy and need to generate three-dimensional (3D) segmentation labels for different types of cell organelles while minimizing manual annotation efforts, to analyze the spatial interactions between organelle instances and to visualize the 3D segmentation results. We provide detailed guidelines for choosing well-suited segmentation tools for specific cell organelles, and to bridge compatibility issues between freely available open-source tools, we distribute the critical steps as easily installable Album solutions for deep learning segmentation, spatial analysis and 3D rendering. Our detailed description can serve as a reference for similar projects requiring particular strategies for single- or multiple-organelle analysis, which can be achieved with computational resources commonly available to single-user setups.
Identifiants
pubmed: 38424188
doi: 10.1038/s41596-024-00957-5
pii: 10.1038/s41596-024-00957-5
doi:
Types de publication
Journal Article
Review
Langues
eng
Sous-ensembles de citation
IM
Subventions
Organisme : EC | Horizon 2020 Framework Programme (EU Framework Programme for Research and Innovation H2020)
ID : 16.0097-2
Informations de copyright
© 2024. Springer Nature Limited.
Références
Peddie, C. J. & Collinson, L. M. Exploring the third dimension: volume electron microscopy comes of age. Micron 61, 9–19 (2014).
pubmed: 24792442
doi: 10.1016/j.micron.2014.01.009
Peddie, C. J. et al. Volume electron microscopy. Nat. Rev. Methods Prim. 2, 51 (2022).
doi: 10.1038/s43586-022-00131-9
Hua, Y., Laserstein, P. & Helmstaedter, M. Large-volume en-bloc staining for electron microscopy-based connectomics. Nat. Commun. 6, 7923 (2015).
pubmed: 26235643
doi: 10.1038/ncomms8923
Kievits, A. J., Lane, R., Carroll, E. C. & Hoogenboom, J. P. How innovations in methodology offer new prospects for volume electron microscopy. J. Microsc. 287, 114–137 (2022).
pubmed: 35810393
pmcid: 9546337
doi: 10.1111/jmi.13134
Graham, B. J. et al. High-throughput transmission electron microscopy with automated serial sectioning. Preprint at bioRxiv https://doi.org/10.1101/657346 (2019).
Yin, W. et al. A petascale automated imaging pipeline for mapping neuronal circuits with high-throughput transmission electron microscopy. Nat. Commun. 11, 4949 (2020).
pubmed: 33009388
pmcid: 7532165
doi: 10.1038/s41467-020-18659-3
Phelps, J. S. et al. Reconstruction of motor control circuits in adult Drosophila using automated transmission electron microscopy. Cell 184, 759–774.e18 (2021).
pubmed: 33400916
pmcid: 8312698
doi: 10.1016/j.cell.2020.12.013
Xu, C. S. et al. Enhanced FIB-SEM systems for large-volume 3D imaging. eLife 6, e25916 (2017).
pubmed: 28500755
pmcid: 5476429
doi: 10.7554/eLife.25916
Motta, A. et al. Dense connectomic reconstruction in layer 4 of the somatosensory cortex. Science 366, eaay3134 (2019).
pubmed: 31649140
doi: 10.1126/science.aay3134
Scheffer, L. K. et al. A connectome and analysis of the adult Drosophila central brain. eLife 9, e57443 (2020).
pubmed: 32880371
pmcid: 7546738
doi: 10.7554/eLife.57443
Müller, A. et al. 3D FIB-SEM reconstruction of microtubule-organelle interaction in whole primary mouse β. cells J. Cell Biol. 220, e202010039 (2021).
pubmed: 33326005
doi: 10.1083/jcb.202010039
Parlakgül, G. et al. Regulation of liver subcellular architecture controls metabolic homeostasis. Nature 603, 736–742 (2022).
pubmed: 35264794
pmcid: 9014868
doi: 10.1038/s41586-022-04488-5
Sheu, S.-H. et al. A serotonergic axon-cilium synapse drives nuclear signaling to alter chromatin accessibility. Cell 185, 3390–3407.e18 (2022).
pubmed: 36055200
pmcid: 9789380
doi: 10.1016/j.cell.2022.07.026
Weigel, A. V. et al. ER-to-Golgi protein delivery through an interwoven, tubular network extending from ER. Cell 184, 2412–2429.e16 (2021).
pubmed: 33852913
doi: 10.1016/j.cell.2021.03.035
Uwizeye, C. et al. Morphological bases of phytoplankton energy management and physiological responses unveiled by 3D subcellular imaging. Nat. Commun. 12, 1049 (2021).
pubmed: 33594064
pmcid: 7886885
doi: 10.1038/s41467-021-21314-0
Musser, J. M. et al. Profiling cellular diversity in sponges informs animal cell type and nervous system evolution. Science 374, 717–723 (2021).
pubmed: 34735222
pmcid: 9233960
doi: 10.1126/science.abj2949
Bharathan, N. K. et al. Architecture and dynamics of a desmosome–endoplasmic reticulum complex. Nat. Cell Biol. 25, 823–835 (2023).
pubmed: 37291267
doi: 10.1038/s41556-023-01154-4
Malong, L. et al. Characterization of the structure and control of the blood-nerve barrier identifies avenues for therapeutic delivery. Dev. Cell 58, 174–191.e8 (2023).
pubmed: 36706755
doi: 10.1016/j.devcel.2023.01.002
Cortese, M. et al. Integrative imaging reveals SARS-CoV-2-induced reshaping of subcellular morphologies. Cell Host Microbe 28, 853–866.e5 (2020).
pubmed: 33245857
pmcid: 7670925
doi: 10.1016/j.chom.2020.11.003
Vergara, H. M. et al. Whole-body integration of gene expression and single-cell morphology. Cell 184, 4819–4837.e22 (2021).
pubmed: 34380046
pmcid: 8445025
doi: 10.1016/j.cell.2021.07.017
Iudin, A., Korir, P. K., Salavert-Torres, J., Kleywegt, G. J. & Patwardhan, A. EMPIAR: a public archive for raw electron microscopy image data. Nat. Methods 13, 387–388 (2016).
pubmed: 27067018
doi: 10.1038/nmeth.3806
Conrad, R. & Narayan, K. CEM500K, a large-scale heterogeneous unlabeled cellular electron microscopy image dataset for deep learning. eLife 10, e65894 (2021).
pubmed: 33830015
pmcid: 8032397
doi: 10.7554/eLife.65894
Xu, C. S. et al. An open-access volume electron microscopy atlas of whole cells and tissues. Nature 599, 147–151 (2021).
pubmed: 34616045
pmcid: 9004664
doi: 10.1038/s41586-021-03992-4
Kremer, J. R., Mastronarde, D. N. & McIntosh, J. R. Computer visualization of three-dimensional image data using IMOD. J. Struct. Biol. 116, 71–76 (1996).
pubmed: 8742726
doi: 10.1006/jsbi.1996.0013
Noske, A. B., Costin, A. J., Morgan, G. P. & Marsh, B. J. Expedited approaches to whole cell electron tomography and organelle mark-up in situ in high-pressure frozen pancreatic islets. J. Struct. Biol. 161, 298–313 (2008).
pubmed: 18069000
doi: 10.1016/j.jsb.2007.09.015
Wu, Y. et al. Contacts between the endoplasmic reticulum and other membranes in neurons. Proc. Natl Acad. Sci. USA 114, E4859–E4867 (2017).
pubmed: 28559323
pmcid: 5474793
doi: 10.1073/pnas.1701078114
Kaynig, V. et al. Large-scale automatic reconstruction of neuronal processes from electron microscopy images. Med. Image Anal. 22, 77–88 (2015).
pubmed: 25791436
pmcid: 4406409
doi: 10.1016/j.media.2015.02.001
Dorkenwald, S. et al. Automated synaptic connectivity inference for volume electron microscopy. Nat. Methods 14, 435–442 (2017).
pubmed: 28250467
doi: 10.1038/nmeth.4206
Buhmann, J. et al. Automatic detection of synaptic partners in a whole-brain Drosophila electron microscopy data set. Nat. Methods 18, 771–774 (2021).
pubmed: 34168373
pmcid: 7611460
doi: 10.1038/s41592-021-01183-7
Heinrich, L. et al. Whole-cell organelle segmentation in volume electron microscopy. Nature 599, 141–146 (2021).
pubmed: 34616042
doi: 10.1038/s41586-021-03977-3
Spiers, H. et al. Deep learning for automatic segmentation of the nuclear envelope in electron microscopy data, trained with volunteer segmentations. Traffic 22, 240–253 (2021).
pubmed: 33914396
doi: 10.1111/tra.12789
Gallusser, B. et al. Deep neural network automated segmentation of cellular structures in volume electron microscopy. J. Cell Biol. 222, e202208005 (2022).
pubmed: 36469001
pmcid: 9728137
doi: 10.1083/jcb.202208005
Breiman, L. Random forests. Mach. Learn. 45, 5–32 (2001).
doi: 10.1023/A:1010933404324
Falk, T. et al. U-Net: deep learning for cell counting, detection, and morphometry. Nat. Methods 16, 67–70 (2019).
pubmed: 30559429
doi: 10.1038/s41592-018-0261-2
Belevich, I., Joensuu, M., Kumar, D., Vihinen, H. & Jokitalo, E. Microscopy image browser: a platform for segmentation and analysis of multidimensional datasets. PLoS Biol. 14, e1002340 (2016).
pubmed: 26727152
pmcid: 4699692
doi: 10.1371/journal.pbio.1002340
Tu, Z. & Bai, X. Auto-context and its application to high-level vision tasks and 3D brain image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 32, 1744–1757 (2010).
pubmed: 20724753
doi: 10.1109/TPAMI.2009.186
Berg, S. et al. ilastik: interactive machine learning for (bio)image analysis. Nat. Methods 16, 1226–1232 (2019).
pubmed: 31570887
doi: 10.1038/s41592-019-0582-9
Kreshuk, A. & Zhang, C. in Computer Optimized Microscopy: Methods and Protocols (eds Rebollo, E. & Bosch, M.) 449–463 (Springer, 2019).
Weigert, M. et al. Content-aware image restoration: pushing the limits of fluorescence microscopy. Nat. Methods 15, 1090–1097 (2018).
pubmed: 30478326
doi: 10.1038/s41592-018-0216-7
Weigert, M., Schmidt, U., Haase, R., Sugawara, K. & Myers, G. Star-convex polyhedra for 3D object detection and segmentation in microscopy. In 2020 IEEE Winter Conference on Applications of Computer Vision (WACV) https://doi.org/10.1109/WACV45572.2020.9093435 (2020).
Helmstaedter, M., Briggman, K. L. & Denk, W. High-accuracy neurite reconstruction for high-throughput neuroanatomy. Nat. Neurosci. 14, 1081–1088 (2011).
pubmed: 21743472
doi: 10.1038/nn.2868
Arzt, M. et al. LABKIT: labeling and segmentation toolkit for big image data. Front. Comput. Sci. https://doi.org/10.3389/fcomp.2022.777728 (2022).
Schindelin, J. et al. Fiji: an open-source platform for biological-image analysis. Nat. Methods 9, 676–682 (2012).
pubmed: 22743772
doi: 10.1038/nmeth.2019
Bolte, S. & Cordelières, F. P. A guided tour into subcellular colocalization analysis in light microscopy. J. Microsc. 224, 213–232 (2006).
pubmed: 17210054
doi: 10.1111/j.1365-2818.2006.01706.x
Shrestha, N. et al. Integration of ER protein quality control mechanisms defines β-cell function and ER architecture. J. Clin. Invest. https://doi.org/10.1172/JCI163584 (2022).
Schmid, B. et al. 3Dscript: animating 3D/4D microscopy data using a natural-language-based syntax. Nat. Methods 16, 278–280 (2019).
pubmed: 30886414
doi: 10.1038/s41592-019-0359-1
Albrecht, J. P., Schmidt, D. & Harrington, K. Album: a framework for scientific data processing with software solutions of heterogeneous tools. Preprint at https://doi.org/10.48550/arXiv.2110.00601 (2021).
Conrad, R. & Narayan, K. Instance segmentation of mitochondria in electron microscopy images with a generalist deep learning model trained on a diverse dataset. Cell Syst. 14, 58–71.e5 (2023).
pubmed: 36657391
pmcid: 9883049
doi: 10.1016/j.cels.2022.12.006
von Chamier, L. et al. Democratising deep learning for microscopy with ZeroCostDL4Mic. Nat. Commun. 12, 2276 (2021).
doi: 10.1038/s41467-021-22518-0
Ouyang, W. et al. BioImage Model Zoo: a community-driven resource for accessible deep learning in bioimage analysis. Preprint at bioRxiv https://doi.org/10.1101/2022.06.07.495102 (2022).
Weber, B. et al. Automated tracing of microtubules in electron tomograms of plastic embedded samples of Caenorhabditis elegans embryos. J. Struct. Biol. 178, 129–138 (2012).
pubmed: 22182731
doi: 10.1016/j.jsb.2011.12.004
Eckstein, N., Buhmann, J., Cook, M. & Funke, J. Microtubule tracking in electron microscopy volumes. In Medical Image Computing and Computer Assisted Intervention (MICCAI) Part V 99–108 (Springer, 2020).
Kaltdorf, K. V. et al. Automated classification of synaptic vesicles in electron tomograms of C. elegans using machine learning. PLoS ONE 13, e0205348 (2018).
pubmed: 30296290
pmcid: 6175533
doi: 10.1371/journal.pone.0205348
Haberl, M. G. et al. CDeep3M—plug-and-play cloud-based deep learning for image segmentation. Nat. Methods 15, 677–680 (2018).
pubmed: 30171236
pmcid: 6548193
doi: 10.1038/s41592-018-0106-z
Koranne, S. in Handbook of Open Source Tools (ed. Koranne, S.) 191–200 (Springer, 2011).
Saalfeld, S. et al. saalfeldlab/n5: n5-2.5.1 https://doi.org/10.5281/zenodo.6578232 (2022).
Miles, A. et al. zarr-developers/zarr-python: v2.4.0 https://doi.org/10.5281/zenodo.3773450 (2020).
Luengo, I. et al. SuRVoS: super-region volume segmentation workbench. J. Struct. Biol. 198, 43–53 (2017).
pubmed: 28246039
pmcid: 5405849
doi: 10.1016/j.jsb.2017.02.007
Pennington, A. et al. SuRVoS 2: accelerating annotation and segmentation for large volumetric bioimage workflows across modalities and scales. Front. Cell Dev. Biol. 10, 842342 (2022).
pubmed: 35433703
pmcid: 9011330
doi: 10.3389/fcell.2022.842342
Belevich, I. & Jokitalo, E. DeepMIB: user-friendly and open-source software for training of deep learning network for biological image segmentation. PLoS Comput. Biol. 17, e1008374 (2021).
pubmed: 33651804
pmcid: 7954287
doi: 10.1371/journal.pcbi.1008374
Hennies, J. et al. CebraEM: a practical workflow to segment cellular organelles in volume SEM datasets using a transferable CNN-based membrane prediction. Preprint at bioRxiv https://doi.org/10.1101/2023.04.06.535829 (2023).
Smith, P. et al. Online citizen science with the Zooniverse for analysis of biological volumetric data. Histochem. Cell Biol. https://doi.org/10.1007/s00418-023-02204-6 (2023).
Jorstad, A. et al. NeuroMorph: a toolset for the morphometric analysis and visualization of 3D models derived from electron microscopy image stacks. Neuroinformatics 13, 83–92 (2015).
pubmed: 25240318
doi: 10.1007/s12021-014-9242-5
Jorstad, A., Blanc, J. & Knott, G. NeuroMorph: a software toolset for 3D analysis of neurite morphology and connectivity. Front. Neuroanat. 12, 59 (2018).
pubmed: 30083094
pmcid: 6064741
doi: 10.3389/fnana.2018.00059
Troidl, J. et al. Barrio: customizable spatial neighborhood analysis and comparison for nanoscale brain structures. Comput. Graph. Forum 41, 183–194 (2022).
doi: 10.1111/cgf.14532
Schroff, F., Criminisi, A. & Zisserman, A. in Procedings of the British Machine Vision Conference 2008 54.1–54.10 (British Machine Vision Association, 2008).
Arganda-Carreras, I. et al. Trainable weka segmentation: a machine learning tool for microscopy pixel classification. Bioinformatics 33, 2424–2426 (2017).
pubmed: 28369169
doi: 10.1093/bioinformatics/btx180
Hallou, A., Yevick, H. G., Dumitrascu, B. & Uhlmann, V. Deep learning for bioimage analysis in developmental biology. Development 148, dev199616 (2021).
pubmed: 34490888
pmcid: 8451066
doi: 10.1242/dev.199616
Shaga Devan, K., Kestler, H. A., Read, C. & Walther, P. Weighted average ensemble-based semantic segmentation in biological electron microscopy images. Histochem. Cell Biol. 158, 447–462 (2022).
pubmed: 35988009
pmcid: 9630254
doi: 10.1007/s00418-022-02148-3
Mandal, S. & Uhlmann, V. Splinedist: automated cell segmentation with spline curves. In 2021 IEEE 18th International Symposium on Biomedical Imaging (ISBI) https://doi.org/10.1109/ISBI48211.2021.9433928 (IEEE, 2021).
Stringer, C., Wang, T., Michaelos, M. & Pachitariu, M. Cellpose: a generalist algorithm for cellular segmentation. Nat. Methods 18, 100–106 (2021).
pubmed: 33318659
doi: 10.1038/s41592-020-01018-x
Sheridan, A. et al. Local shape descriptors for neuron segmentation. Nat. Methods 20, 295–303 (2023).
pubmed: 36585455
doi: 10.1038/s41592-022-01711-z
McDonald, K. L., O’Toole, E. T., Mastronarde, D. N. & McIntosh, J. R. Kinetochore microtubules in PTK cells. J. Cell Biol. 118, 369–383 (1992).
pubmed: 1629239
doi: 10.1083/jcb.118.2.369
Marsh, B. J., Mastronarde, D. N., Buttle, K. F., Howell, K. E. & McIntosh, J. R. Organellar relationships in the Golgi region of the pancreatic beta cell line, HIT-T15, visualized by high resolution electron tomography. Proc. Natl Acad. Sci. USA 98, 2399–2406 (2001).
pubmed: 11226251
pmcid: 30150
doi: 10.1073/pnas.051631998
Rueden, C. T. et al. ImageJ2: ImageJ for the next generation of scientific image data. BMC Bioinforma. 18, 529 (2017).
doi: 10.1186/s12859-017-1934-z
Pietzsch, T., Preibisch, S., Tomančák, P. & Saalfeld, S. ImgLib2—generic image processing in Java. Bioinformatics 28, 3009–3011 (2012).
pubmed: 22962343
pmcid: 3496339
doi: 10.1093/bioinformatics/bts543
Harris, C. R. et al. Array programming with NumPy. Nature 585, 357–362 (2020).
pubmed: 32939066
pmcid: 7759461
doi: 10.1038/s41586-020-2649-2
Hunter, J. D. Matplotlib: a 2D graphics environment. Comput. Sci. Eng. 9, 90–95 (2007).
doi: 10.1109/MCSE.2007.55
Virtanen, P. et al. SciPy 1.0: fundamental algorithms for scientific computing in Python. Nat. Methods 17, 261–272 (2020).
pubmed: 32015543
pmcid: 7056644
doi: 10.1038/s41592-019-0686-2
McKinney, W. Data structures for statistical computing in Python. In Proc. of the 9th Python in Science Conference. https://doi.org/10.25080/Majora-92bf1922-00a (2010).
Schmid, B., Schindelin, J., Cardona, A., Longair, M. & Heisenberg, M. A high-level 3D visualization API for Java and ImageJ. BMC Bioinforma. 11, 274 (2010).
doi: 10.1186/1471-2105-11-274
Cardona, A. et al. TrakEM2 software for neural circuit reconstruction. PLoS ONE 7, e38011 (2012).
pubmed: 22723842
pmcid: 3378562
doi: 10.1371/journal.pone.0038011
Hennies, J. et al. AMST: alignment to median smoothed template for focused ion beam scanning electron microscopy image stacks. Sci. Rep. 10, 2004 (2020).
pubmed: 32029771
pmcid: 7004979
doi: 10.1038/s41598-020-58736-7
Hanslovsky, P., Bogovic, J. A. & Saalfeld, S. Image-based correction of continuous and discontinuous non-planar axial distortion in serial section microscopy. Bioinformatics 33, 1379–1386 (2017).
pubmed: 28453669
doi: 10.1093/bioinformatics/btw794
Roels, J. et al. An interactive ImageJ plugin for semi-automated image denoising in electron microscopy. Nat. Commun. 11, 771 (2020).
pubmed: 32034132
pmcid: 7005902
doi: 10.1038/s41467-020-14529-0
Krull, A., Buchholz, T.-O. & Jug, F. Noise2Void—learning denoising from single noisy images. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2129–2137 (2019).
Perez, A. J. et al. A workflow for the automatic segmentation of organelles in electron microscopy image stacks. Front. Neuroanat. 8, 126 (2014).
pubmed: 25426032
pmcid: 4224098
doi: 10.3389/fnana.2014.00126
Hoffman, D. P. et al. Correlative three-dimensional super-resolution and block-face electron microscopy of whole vitreously frozen cells. Science 367, eaaz5357 (2020).
pubmed: 31949053
pmcid: 7339343
doi: 10.1126/science.aaz5357
Müller, A. et al. Structure, interaction, and nervous connectivity of beta cell primary cilia. Preprint at bioRxriv https://doi.org/10.1101/2023.12.01.568979 (2024).
Park, G. et al. Amira annotation protocol. protocols.io https://www.protocols.io/view/amira-annotation-protocol-b834ryqw (2022).