VOLTA: an enVironment-aware cOntrastive ceLl represenTation leArning for histopathology.
Journal
Nature communications
ISSN: 2041-1723
Titre abrégé: Nat Commun
Pays: England
ID NLM: 101528555
Informations de publication
Date de publication:
10 May 2024
10 May 2024
Historique:
received:
29
03
2023
accepted:
19
04
2024
medline:
11
5
2024
pubmed:
11
5
2024
entrez:
10
5
2024
Statut:
epublish
Résumé
In clinical oncology, many diagnostic tasks rely on the identification of cells in histopathology images. While supervised machine learning techniques necessitate the need for labels, providing manual cell annotations is time-consuming. In this paper, we propose a self-supervised framework (enVironment-aware cOntrastive cell represenTation learning: VOLTA) for cell representation learning in histopathology images using a technique that accounts for the cell's mutual relationship with its environment. We subject our model to extensive experiments on data collected from multiple institutions comprising over 800,000 cells and six cancer types. To showcase the potential of our proposed framework, we apply VOLTA to ovarian and endometrial cancers and demonstrate that our cell representations can be utilized to identify the known histotypes of ovarian cancer and provide insights that link histopathology and molecular subtypes of endometrial cancer. Unlike supervised models, we provide a framework that can empower discoveries without any annotation data, even in situations where sample sizes are limited.
Identifiants
pubmed: 38729933
doi: 10.1038/s41467-024-48062-1
pii: 10.1038/s41467-024-48062-1
doi:
Types de publication
Journal Article
Research Support, Non-U.S. Gov't
Research Support, N.I.H., Extramural
Langues
eng
Sous-ensembles de citation
IM
Pagination
3942Subventions
Organisme : Gouvernement du Canada | Natural Sciences and Engineering Research Council of Canada (Conseil de Recherches en Sciences Naturelles et en Génie du Canada)
ID : Discovery Grant
Organisme : Gouvernement du Canada | Canadian Institutes of Health Research (Instituts de Recherche en Santé du Canada)
ID : Project Grant
Organisme : Michael Smith Foundation for Health Research (MSFHR)
ID : Scholar Program
Organisme : Terry Fox Research Institute (Institut de Recherche Terry Fox)
ID : Program Project
Informations de copyright
© 2024. The Author(s).
Références
Farc, O. & Cristea, V. An overview of the tumor microenvironment, from cells to complex networks. Exp. Therapeutic Med. 21, 1–1 (2021).
Yang, Y., Yang, Y., Yang, J., Zhao, X. & Wei, X. Tumor microenvironment in ovarian cancer: function and therapeutic strategy. Front. Cell Dev. Biol. 8, 758 (2020).
doi: 10.3389/fcell.2020.00758
pubmed: 32850861
pmcid: 7431690
Liu, W. et al. Transcriptome-derived stromal and immune scores infer clinical outcomes of patients with cancer. Oncol. Lett. 15, 4351–4357 (2018).
pubmed: 29541203
pmcid: 5835954
Bremnes, R. M. et al. The role of tumor-infiltrating lymphocytes in development, progression, and prognosis of non–small cell lung cancer. J. Thorac. Oncol. 11, 789–800 (2016).
doi: 10.1016/j.jtho.2016.01.015
pubmed: 26845192
Schalper, K. A. et al. Objective measurement and clinical significance of tils in non–small cell lung cancer. J. Natl Cancer Inst. 107, dju435 (2015).
doi: 10.1093/jnci/dju435
pubmed: 25650315
pmcid: 4565530
Pogrebniak, K. L. & Curtis, C. Harnessing tumor evolution to circumvent resistance. Trends Genet. 34, 639–651 (2018).
doi: 10.1016/j.tig.2018.05.007
pubmed: 29903534
pmcid: 6368975
Zhang, A., Miao, K., Sun, H. & Deng, C.-X. Tumor heterogeneity reshapes the tumor microenvironment to influence drug resistance. Int. J. Biol. Sci. 18, 3019 (2022).
doi: 10.7150/ijbs.72534
pubmed: 35541919
pmcid: 9066118
Zhang, J. et al. Intratumor heterogeneity in localized lung adenocarcinomas delineated by multiregion sequencing. Science 346, 256–259 (2014).
doi: 10.1126/science.1256930
pubmed: 25301631
pmcid: 4354858
Schwarz, R. F. et al. Spatial and temporal heterogeneity in high-grade serous ovarian cancer: a phylogenetic analysis. PLoS Med. 12, e1001789 (2015).
doi: 10.1371/journal.pmed.1001789
pubmed: 25710373
pmcid: 4339382
Andor, N. et al. Pan-cancer analysis of the extent and consequences of intratumor heterogeneity. Nat. Med. 22, 105–113 (2016).
doi: 10.1038/nm.3984
pubmed: 26618723
Steinhart, B. et al. The spatial context of tumor-infiltrating immune cells associates with improved ovarian cancer survivalspatial interactions in the tumor immune microenvironment. Mol. Cancer Res. 19, 1973–1979 (2021).
doi: 10.1158/1541-7786.MCR-21-0411
pubmed: 34615692
Fu, T. et al. Spatial architecture of the immune microenvironment orchestrates tumor immunity and therapeutic response. J. Hematol. Oncol. 14, 98 (2021).
doi: 10.1186/s13045-021-01103-4
pubmed: 34172088
pmcid: 8234625
Brambilla, E. et al. Prognostic effect of tumor lymphocytic infiltration in resectable non–small-cell lung cancer. J. Clin. Oncol. 34, 1223 (2016).
doi: 10.1200/JCO.2015.63.0970
pubmed: 26834066
pmcid: 4872323
Corredor, G. et al. Spatial architecture and arrangement of tumor-infiltrating lymphocytes for predicting likelihood of recurrence in early-stage non–small cell lung cancer. Clin. Cancer Res. 25, 1526–1534 (2019).
doi: 10.1158/1078-0432.CCR-18-2013
pubmed: 30201760
Javed, S. et al. Cellular community detection for tissue phenotyping in colorectal cancer histology images. Med. Image Anal. 63, 101696 (2020).
doi: 10.1016/j.media.2020.101696
pubmed: 32330851
Zhou, Y. et al. Cgc-net: Cell graph convolutional network for grading of colorectal cancer histology images. IEEE/CVF International Conference on Computer Vision workshop 0–0 (IEEE, 2019).
Martin-Gonzalez, P., Crispin-Ortuzar, M. & Markowetz, F. Predictive modelling of highly multiplexed tumour tissue images by graph neural networks. In: Interpretability of Machine Intelligence in Medical Image Computing, and Topological Data Analysis and Its Applications for Medical Data. (eds. Reyes, M. et al.) vol 12929, 98–107 (Springer, Cham, 2021). https://doi.org/10.1007/978-3-030-87444-5_10 .
Lu, W., Graham, S., Bilal, M., Rajpoot, N. & Minhas, F. Capturing cellular topology in multi-gigapixel pathology images. In IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops 260–261 (IEEE, 2020).
Elmore, J. G. et al. Diagnostic concordance among pathologists interpreting breast biopsy specimens. JAMA 313, 1122–1132 (2015).
doi: 10.1001/jama.2015.1405
pubmed: 25781441
pmcid: 4516388
Sirinukunwattana, K. et al. Locality sensitive deep learning for detection and classification of nuclei in routine colon cancer histology images. IEEE Trans. Med. Imaging 35, 1196–1206 (2016).
doi: 10.1109/TMI.2016.2525803
pubmed: 26863654
Graham, S. et al. Hover-net: Simultaneous segmentation and classification of nuclei in multi-tissue histology images. Med. Image Anal. 58, 101563 (2019).
doi: 10.1016/j.media.2019.101563
pubmed: 31561183
Amgad, M.et al. Nucls: A scalable crowdsourcing, deep learning approach and dataset for nucleus classification, localization and segmentation. Gigascience 11, giac037 (2022).
Dalle, J.-R. et al. Nuclear pleomorphism scoring by selective cell nuclei detection. IEEE/CVF Winter Conference of Computer Vision (IEEE, 2009).
Nguyen, K., Jain, A. K. & Sabata, B. Prostate cancer detection: Fusion of cytological and textural features. J. Pathol. Informatics 2, S3 (2011).
Cruz-Roa, A. A., Ovalle, J. E. A., Madabhushi, A. & Osorio, F. A. G. A deep learning architecture for image representation, visual interpretability and automated basal-cell carcinoma cancer detection. Med. Image Comput. Comput. Assist. Interv. 16, 403–410 (2013).
Han, W. et al. Identification of molecular cell type of breast cancer on digital histopathology images using deep learning and multiplexed fluorescence imaging. Digital Comput. Pathol. 12471, 26–31 (2023).
Hu, B. et al. Unsupervised learning for cell-level visual representation in histopathology images with generative adversarial networks. IEEE J. Biomed. Health Inform. 23, 1316–1328 (2018).
doi: 10.1109/JBHI.2018.2852639
pubmed: 29994411
Chen, X. et al. Infogan: Interpretable representation learning by information maximizing generative adversarial nets. In Proceedings of the 30th International Conference on Neural Information Processing Systems 2180–2188 (NIPS, 2016).
Vununu, C., Lee, S.-H. & Kwon, K.-R. A strictly unsupervised deep learning method for hep-2 cell image classification. Sensors 20, 2717 (2020).
doi: 10.3390/s20092717
pubmed: 32397567
pmcid: 7249201
Keren, L. et al. A structured tumor-immune microenvironment in triple negative breast cancer revealed by multiplexed ion beam imaging. Cell 174, 1373–1387 (2018).
doi: 10.1016/j.cell.2018.08.039
pubmed: 30193111
pmcid: 6132072
Schürch, C. M. et al. Coordinated cellular neighborhoods orchestrate antitumoral immunity at the colorectal cancer invasive front. Cell 182, 1341–1359 (2020).
doi: 10.1016/j.cell.2020.07.005
pubmed: 32763154
pmcid: 7479520
Yuan, Y. Spatial heterogeneity in the tumor microenvironment. Cold Spring Harb. Perspect. Med. 6, a026583 (2016).
doi: 10.1101/cshperspect.a026583
pubmed: 27481837
pmcid: 4968167
Caron, M. et al. Emerging properties in self-supervised vision transformers. In: Proceedings of the IEEE/CVF international conference on computer vision. 9650–9660 (2021).
Sohn, K. et al. A simple semi-supervised learning framework for object detection. Preprint at https://arxiv.org/abs/2005.04757 (2020).
Ciga, O., Xu, T. & Martel, A. L. Self supervised contrastive learning for digital histopathology. Mach. Learn. Appl. 7, 100198 (2022).
Zhang, L., Amgad, M. & Cooper, L. A. A histopathology study comparing contrastive semi-supervised and fully supervised learning. Preprint at https://arxiv.org/abs/2111.05882 (2021).
Chen, X., Fan, H., Girshick, R. & He, K. Improved baselines with momentum contrastive learning. Preprint at https://arxiv.org/abs/2003.04297 (2020).
LeCun, Y., Chopra, S., Hadsell, R., Ranzato, M. & Huang, F. A tutorial on energy-based learning. Predicting structured data 1 (2006).
Oord, A. v. d., Li, Y. & Vinyals, O. Representation learning with contrastive predictive coding. Preprint at https://arxiv.org/abs/1807.03748 (2018).
Gamper, J. et al. Pannuke dataset extension, insights and baselines. Preprint at https://arxiv.org/abs/2003.10778 (2020).
Graham, S. et al. Lizard: A large-scale dataset for colonic nuclear instance segmentation and classification. In Proceedings of the IEEE/CVF International Conference on Computer Vision 684–693 (IEEE, 2021).
Aubreville, M. et al. Mitosis domain generalization in histopathology images-the midog challenge. Med. Image Anal. 84, 102699 (2023).
doi: 10.1016/j.media.2022.102699
pubmed: 36463832
Vinh, N. X., Epps, J. & Bailey, J. Information theoretic measures for clusterings comparison: Variants, properties, normalization and correction for chance. J. Mach. Learn. Res. 11, 2837–2854 (2010).
Hubert, L. & Arabie, P. Comparing partitions. J. Classification 2, 193–218 (1985).
doi: 10.1007/BF01908075
Rand, W. M. Objective criteria for the evaluation of clustering methods. J. Am. Stat. Assoc. 66, 846–850 (1971).
doi: 10.1080/01621459.1971.10482356
Boschman, J. et al. The utility of color normalization for ai-based diagnosis of hematoxylin and eosin-stained pathology images. J. Pathol. 256, 15–24 (2022).
doi: 10.1002/path.5797
pubmed: 34543435
Vahadane, A. et al. Structure-preserving color normalization and sparse stain separation for histological images. IEEE Trans. Med. imaging 35, 1962–1971 (2016).
doi: 10.1109/TMI.2016.2529665
pubmed: 27164577
Farahani, H. et al. Deep learning-based histotype diagnosis of ovarian carcinoma whole-slide pathology images. Mod. Pathol. 35, 1983–1990 (2022).
doi: 10.1038/s41379-022-01146-z
pubmed: 36065012
Moch, H. Female genital tumours: Who classification of tumours, vol. 4 (WHO, 2020).
Pasanen, A., Loukovaara, M. & Bützow, R. Clinicopathological significance of deficient dna mismatch repair and mlh1 promoter methylation in endometrioid endometrial carcinoma. Mod. Pathol. 33, 1443–1452 (2020).
doi: 10.1038/s41379-020-0501-8
pubmed: 32060377
Ramchander, N. C. et al. Distinct immunological landscapes characterize inherited and sporadic mismatch repair deficient endometrial cancer. Front. Immunol. 10, 3023 (2020).
doi: 10.3389/fimmu.2019.03023
pubmed: 31998307
pmcid: 6970202
Dong, D. et al. Pole and mismatch repair status, checkpoint proteins and tumor-infiltrating lymphocytes in combination, and tumor differentiation: identify endometrial cancers for immunotherapy. Front. Oncol. 11, 640018 (2021).
doi: 10.3389/fonc.2021.640018
pubmed: 33816285
pmcid: 8017289
Scalon, J. D., Avelar, M. B. L., Alves, Gd. F. & Zacarias, M. S. Spatial and temporal dynamics of coffee-leaf-miner and predatory wasps in organic coffee field in formation. Ciência Rural 41, 646–652 (2011).
doi: 10.1590/S0103-84782011005000037
Ripley, B. D. The second-order analysis of stationary point processes. J. Appl. Probab. 13, 255–266 (1976).
doi: 10.2307/3212829
Yuan, Y. Modelling the spatial heterogeneity and molecular correlates of lymphocytic infiltration in triple-negative breast cancer. J. R. Soc. Interface 12, 20141153 (2015).
doi: 10.1098/rsif.2014.1153
pubmed: 25505134
pmcid: 4305416
Denkert, C. et al. Tumor-associated lymphocytes as an independent predictor of response to neoadjuvant chemotherapy in breast cancer. J. Clin. Oncol. 28, 105–113 (2010).
doi: 10.1200/JCO.2009.23.7370
pubmed: 19917869
Blise, K. E., Sivagnanam, S., Banik, G. L., Coussens, L. M. & Goecks, J. Single-cell spatial architectures associated with clinical outcome in head and neck squamous cell carcinoma. NPJ Precis. Oncol. 6, 1–14 (2022).
Nakhli, R., Darbandsari, A., Farahani, H. & Bashashati, A. Ccrl: Contrastive cell representation learning. European conference on computer vision. 397–407 (Springer, 2022).
Wu, M., Zhuang, C., Mosse, M., Yamins, D. & Goodman, N. On mutual information in contrastive learning for visual representations. Preprint at https://arxiv.org/abs/2005.13149 (2020).
Minderer, M., Bachem, O., Houlsby, N. & Tschannen, M. Automatic shortcut removal for self-supervised representation learning. In International Conference on Machine Learning 6927–6937 (ACM, 2020).
He, K., Zhang, X., Ren, S. & Sun, J. Identity mappings in deep residual networks. In Computer Vision–ECCV 2016: 14th European Conference. (eds. Leibe, B., Matas, J., Sebe, N. & Welling, M.) vol. 9908, 630–645 (Springer, Amsterdam, The Netherlands, 2016).
Bello, I. Lambdanetworks: Modeling long-range interactions without attention. Preprint at https://arxiv.org/abs/2102.08602 (2021).
Van der Maaten, L. & Hinton, G. Visualizing data using t-sne. J. Mach. Learn. Res. 9, 2579−2605 (2008).