The multimodality cell segmentation challenge: toward universal solutions.

Journal

Nature methods

ISSN: 1548-7105

Titre abrégé: Nat Methods

Pays: United States

ID NLM: 101215604

Informations de publication

Date de publication:
26 Mar 2024

Historique:

received: 27 07 2023

accepted: 04 03 2024

medline: 27 3 2024

pubmed: 27 3 2024

entrez: 27 3 2024

Statut: aheadofprint

Résumé

Cell segmentation is a critical step for quantitative single-cell analysis in microscopy images. Existing cell segmentation methods are often tailored to specific modalities or require manual interventions to specify hyper-parameters in different experimental settings. Here, we present a multimodality cell segmentation benchmark, comprising more than 1,500 labeled images derived from more than 50 diverse biological experiments. The top participants developed a Transformer-based deep-learning algorithm that not only exceeds existing methods but can also be applied to diverse microscopy images across imaging platforms and tissue types without manual parameter adjustments. This benchmark and the improved algorithm offer promising avenues for more accurate and versatile cell analysis in microscopy imaging.

Identifiants

DOI: 10.1038/s41592-024-02233-6 PMID: 38532015

pubmed: 38532015

doi: 10.1038/s41592-024-02233-6

pii: 10.1038/s41592-024-02233-6

doi:

Types de publication

Journal Article

Langues

eng

Sous-ensembles de citation

Subventions

Organisme : Canadian Network for Research and Innovation in Machining Technology, Natural Sciences and Engineering Research Council of Canada (NSERC Canadian Network for Research and Innovation in Machining Technology)

ID : NSERC, RGPIN-2020-06189 and DGECR-2020-00294

Informations de copyright

Références

Jackson, H. W. et al. The single-cell pathology landscape of breast cancer. Nature 578, 615–620 (2020).

doi: 10.1038/s41586-019-1876-x pubmed: 31959985

Capolupo, L. et al. Sphingolipids control dermal fibroblast heterogeneity. Science 376, eabh1623 (2022).

doi: 10.1126/science.abh1623 pubmed: 35420948

Lin, J.-R. et al. Multiplexed 3d atlas of state transitions and immune interaction in colorectal cancer. Cell 186, 363–381 (2023).

doi: 10.1016/j.cell.2022.12.028 pubmed: 36669472 pmcid: 10019067

Hollandi, R. et al. Nucleus segmentation: towards automated solutions. Trends Cell Biol. 32, 295–310 (2022).

Greenwald, N. F. et al. Whole-cell segmentation of tissue images with human-level performance using large-scale data annotation and deep learning. Nat. Biotechnol. 40, 555–565 (2021).

Lee, M. Y. et al. Cellseg: a robust, pre-trained nucleus segmentation and pixel quantification software for highly multiplexed fluorescence images. BMC Bioinform. 23, 1–17 (2022).

doi: 10.1186/s12859-022-04570-9

Kempster, C. et al. Fully automated platelet differential interference contrast image analysis via deep learning. Sci. Rep. 12, 1–13 (2022).

doi: 10.1038/s41598-022-08613-2

Cutler, K. J. et al. Omnipose: a high-precision morphology-independent solution for bacterial cell segmentation. Nat. Meth. 19, 1438–1448 (2022).

doi: 10.1038/s41592-022-01639-4

Bunk, D. et al. Yeastmate: neural network-assisted segmentation of mating and budding events in Saccharomyces cerevisiae. Bioinformatics 38, 2667–2669 (2022).

doi: 10.1093/bioinformatics/btac107 pubmed: 35179572 pmcid: 9048668

Dietler, N. et al. A convolutional neural network segments yeast microscopy images with high accuracy. Nat. Commun. 11, 1–8 (2020).

doi: 10.1038/s41467-020-19557-4

Stringer, C., Wang, T., Michaelos, M. & Pachitariu, M. Cellpose: a generalist algorithm for cellular segmentation. Nat. Meth. 18, 100–106 (2021).

doi: 10.1038/s41592-020-01018-x

Pachitariu, M. & Stringer, C. Cellpose 2.0: how to train your own model. Nat. Meth. 19, 1634–1641 (2022).

doi: 10.1038/s41592-022-01663-4

Ulman, V. et al. An objective comparison of cell-tracking algorithms. Nat. Meth. 14, 1141–1152 (2017).

doi: 10.1038/nmeth.4473

Maška, M. et al. The cell tracking challenge: 10 years of objective benchmarking. Nat. Meth. 20, 1010–1020 (2023).

Caicedo, J. C. et al. Nucleus segmentation across imaging experiments: the 2018 data science bowl. Nat. Meth. 16, 1247–1253 (2019).

doi: 10.1038/s41592-019-0612-7

Graham, S. et al. CoNIC challenge: pushing the frontiers of nuclear detection, segmentation, classification and counting. Med. Image Anal. 92, 103047 (2024).

doi: 10.1016/j.media.2023.103047 pubmed: 38157647

Tajbakhsh, N. et al. Embracing imperfect datasets: a review of deep learning solutions for medical image segmentation. Med. Image Anal. 63, 101693 (2020).

doi: 10.1016/j.media.2020.101693 pubmed: 32289663

Ma, J. & Wang, B. Towards foundation models of biological image segmentation. Nat. Meth. 20, 953–955 (2023).

doi: 10.1038/s41592-023-01885-0

Gupta, A. et al. Segpc-2021: a challenge & dataset on segmentation of multiple myeloma plasma cells from microscopic images. Med. Image Anal. 83, 102677 (2023).

doi: 10.1016/j.media.2022.102677 pubmed: 36403309

Falk, T. et al. U-net: deep learning for cell counting, detection, and morphometry. Nat. Meth. 16, 67–70 (2019).

doi: 10.1038/s41592-018-0261-2

Chen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K. & Yuille, A. L. Deeplab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs. IEEE Trans. Pattern Anal. Mach. Intell. 40, 834–848 (2017).

doi: 10.1109/TPAMI.2017.2699184 pubmed: 28463186

Vaswani, A. et al. Attention is all you need. in Advances in Neural Information Processing Systems, vol. 30 (NeurIPS, 2017).

Dosovitskiy, A. et al. An image is worth 16 × 16 words: Transformers for image recognition at scale. in International Conference on Learning Representations (ICLR, 2021).

Ma, J. et al. Segment anything in medical images. Nat. Commun. 15, 654 (2024).

doi: 10.1038/s41467-024-44824-z pubmed: 38253604 pmcid: 10803759

Lee, G., Kim, S., Kim, J. & Yun, S.-Y. Mediar: harmony of data-centric and model-centric for multi-modality microscopy. in Proceedings of The Cell Segmentation Challenge in Multi-modality High-Resolution Microscopy Images, vol. 212, pages 1–16 (2023).

Xie, E. et al. Segformer: simple and efficient design for semantic segmentation with transformers. in Advances in Neural Information Processing Systems, vol. 34 (NeurIPS, 2021).

Fan, T., Wang, G., Li, Y. & Wang, H. Ma-net: a multi-scale attention network for liver and tumor segmentation. IEEE Access 8, 179656–179665 (2020).

doi: 10.1109/ACCESS.2020.3025372

Chaudhry, A., Gordo, A., Dokania, P., Torr, P. & Lopez-Paz, D. Using hindsight to anchor past knowledge in continual learning. in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, pages 6993–7001 (AAAI, 2021).

Lou, W. et al. Multi-stream cell segmentation with low-level cues for multi-modality images. Proc. Mach. Learn. Res. 212, 1–10 (2023).

Liu, Z. et al. A convnet for the 2020s. in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 11976–11986 (IEEE, 2022).

Upschulte, E., Harmeling, S., Amunts, K. & Dickscheid, T. Uncertainty-aware contour proposal networks for cell segmentation in multi-modality high-resolution microscopy images. Proc. Mach. Learn Res. 212, 1–12 (2023).

Xie, S., Girshick, R., Dollár, P., Tu, Z. & He, K. Aggregated residual transformations for deep neural networks. in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 1492–1500 (IEEE, 2017).

Scherr, T., Löffler, K., Böhland, M. & Mikut, R. Cell segmentation and tracking using cnn-based distance predictions and a graph-based matching strategy. PLoS ONE 15, e0243219 (2020).

doi: 10.1371/journal.pone.0243219 pubmed: 33290432 pmcid: 7723299

Parisi, G. I., Kemker, R., Part, J. L., Kanan, C. & Wermter, S. Continual lifelong learning with neural networks: a review. Neural Netw. 113, 54–71 (2019).

doi: 10.1016/j.neunet.2019.01.012 pubmed: 30780045

De Lange, M. et al. A continual learning survey: defying forgetting in classification tasks. IEEE Trans. Pattern Anal. Mach. Intell. 44, 3366–3385 (2021).

Pena, F. A. G. et al. J regularization improves imbalanced multiclass segmentation. in IEEE 17th International Symposium on Biomedical Imaging, 1–5 (IEEE, 2020).

Isensee, F., Jaeger, P. F., Kohl, S. A. A., Petersen, J. & Maier-Hein, K. H. nnu-net: a self-configuring method for deep learning-based biomedical image segmentation. Nat. Meth. 18, 203–211 (2021).

doi: 10.1038/s41592-020-01008-z

He, K., Gkioxari, G., Dollár, P. & Girshick, R. Mask r-cnn. in Proceedings of the IEEE International Conference on Computer Vision, 2961–2969 (IEEE, 2017).

Wangkai, L. et al. Maunet: modality-aware anti-ambiguity u-net for multi-modality cell segmentation. Proc. Mach. Learn Res. 212, 1–12 (2023).

Bochkovskiy, A., Wang, C.-Y. & Liao, H.-Y. M. Yolov4: optimal speed and accuracy of object detection. Preprint at arXiv https://doi.org/10.48550/arXiv.2004.10934 (2020).

Jeong, J., Lee, S., Kim, J. & Kwak, N. Consistency-based semi-supervised learning for object detection. in Advances in Neural Information Processing Systems, vol. 32 (NeurIPS, 2019).

Chen, S., Bortsova, G., Juárez, A.G.-U., Van Tulder, G. & De Bruijne, M. Multi-task attention-based semi-supervised learning for medical image segmentation. in Medical Image Computing and Computer Assisted Intervention, 457–465 (MICCAI, 2019).

Liu, Y.-C., Ma, C.-Y. & Kira, Z. Unbiased teacher v2: semi-supervised object detection for anchor-free and anchor-based detectors. in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 9819–9828 (IEEE, 2022).

Sofroniew, N. et al. napari: a multi-dimensional image viewer for Python. Zenodo https://zenodo.org/10.5281/zenodo.3555620 (2022).

Risom, T. et al. Transition to invasive breast cancer is associated with progressive changes in the structure and composition of tumor stroma. Cell 185, 299–310 (2022).

doi: 10.1016/j.cell.2021.12.023 pubmed: 35063072 pmcid: 8792442

Fu, S. et al. Field-dependent deep learning enables high-throughput whole-cell 3D super-resolution imaging. Nat. Meth. 20, 459–468 (2023).

doi: 10.1038/s41592-023-01775-5

Misra, D. Mish: a self regularized non-monotonic activation function. in British Machine Vision Conference (2020).

Edlund, C. et al. Livecell—a large-scale dataset for label-free live cell segmentation. Nat. Meth. 18, 1038–1045 (2021).

doi: 10.1038/s41592-021-01249-6

He, K., Zhang, X., Ren, S. & Sun, J. Deep residual learning for image recognition. in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 770–778 (IEEE, 2016).

Schmidt, U., Weigert, M., Broaddus, C. & Myers, G. Cell detection with star-convex polygons. in Medical Image Computing and Computer Assisted Intervention – MICCAI 2018, 265–273 (MICCAI, 2018).

Ren, S., He, K., Girshick, R. & Sun, J. Faster r-cnn: towards real-time object detection with region proposal networks. in Advances in Neural Information Processing Systems, vol. 28 (NeurIPS, 2015).

Graham, S. et al. Hover-net: simultaneous segmentation and classification of nuclei in multi-tissue histology images. Med. Image Anal. 58, 101563 (2019).

doi: 10.1016/j.media.2019.101563 pubmed: 31561183

Upschulte, E., Harmeling, S., Amunts, K. & Dickscheid, T. Contour proposal networks for biomedical instance segmentation. Med. Image Anal. 77, 102371 (2022).

doi: 10.1016/j.media.2022.102371 pubmed: 35180674

Kuhl, F. P. & Giardina, C. R. Elliptic fourier features of a closed contour. Comput.Graph. Image Process. 18, 236–258 (1982).

doi: 10.1016/0146-664X(82)90034-X

Rezatofighi, H. et al. Generalized intersection over union: a metric and a loss for bounding box regression. in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (IEEE, 2019).

Lee, Y. et al. Localization uncertainty estimation for anchor-free object detection. in Computer Vision – ECCV 2022 Workshops, 27–42 (ECCV, 2023).

Adams, R. & Bischof, L. Seeded region growing. IEEE Trans. Pattern Anal. Mach. Intell. 16, 641–647 (1994).

doi: 10.1109/34.295913

Ronneberger, O., Fischer, P. & Brox, T. U-net: convolutional networks for biomedical image segmentation. in International Conference on Medical Image Computing and Computer-assisted Intervention, 234–241 (MICCAI, 2015).

Maier-Hein, L. et al. Metrics reloaded: recommendations for image analysis validation. Nat. Meth. https://doi.org/10.1038/s41592-023-02151-z (2024).

Hirling, D. et al. Segmentation metric misinterpretations in bioimage analysis. Nat. Meth. https://doi.org/10.1038/s41592-023-01942-8 (2023).

Maier-Hein, L. et al. Why rankings of biomedical image analysis competitions should be interpreted with care. Nat. Commun. 9, 5217 (2018).

doi: 10.1038/s41467-018-07619-7 pubmed: 30523263 pmcid: 6284017

Wiesenfarth, M. et al. Methods and open-source toolkit for analyzing and visualizing challenge results. Sci. Rep. 11, 1–15 (2021).

Kendall, M. G. A new measure of rank correlation. Biometrika 30, 81–93 (1938).

doi: 10.1093/biomet/30.1-2.81

Ma, J. et al. NeurIPS 2022 Cell Segmentation Competition Dataset. in Thirty-sixth Conference on Neural Information Processing Systems (NeurIPS) Zenodo https://doi.org/10.5281/zenodo.10719375 (2024).