Leveraging weak complementary labels enhances semantic segmentation of hepatocellular carcinoma and intrahepatic cholangiocarcinoma.
Journal
Scientific reports
ISSN: 2045-2322
Titre abrégé: Sci Rep
Pays: England
ID NLM: 101563288
Informations de publication
Date de publication:
23 10 2024
23 10 2024
Historique:
received:
25
09
2024
accepted:
03
10
2024
medline:
24
10
2024
pubmed:
24
10
2024
entrez:
24
10
2024
Statut:
epublish
Résumé
In this paper we present a deep learning segmentation approach to classify and quantify the two most prevalent primary liver cancers - hepatocellular carcinoma and intrahepatic cholangiocarcinoma - from hematoxylin and eosin (H&E) stained whole slide images. While semantic segmentation of medical images typically requires costly pixel-level annotations by domain experts, there often exists additional information which is routinely obtained in clinical diagnostics but rarely utilized for model training. We propose to leverage such weak information from patient diagnoses by deriving complementary labels that indicate to which class a sample cannot belong to. To integrate these labels, we formulate a complementary loss for segmentation. Motivated by the medical application, we demonstrate for general segmentation tasks that including additional patches with solely weak complementary labels during model training can significantly improve the predictive performance and robustness of a model. On the task of diagnostic differentiation between hepatocellular carcinoma and intrahepatic cholangiocarcinoma, we achieve a balanced accuracy of 0.91 (CI 95%: 0.86-0.95) at case level for 165 hold-out patients. Furthermore, we also show that leveraging complementary labels improves the robustness of segmentation and increases performance at case level.
Identifiants
pubmed: 39443575
doi: 10.1038/s41598-024-75256-w
pii: 10.1038/s41598-024-75256-w
doi:
Types de publication
Journal Article
Langues
eng
Sous-ensembles de citation
IM
Pagination
24988Informations de copyright
© 2024. The Author(s).
Références
Sung, H. et al. Global cancer statistics 2020: Globocan estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA Cancer J Clin. 71, 209–249. https://doi.org/10.3322/caac.21660 (2021).
doi: 10.3322/caac.21660
pubmed: 33538338
Altekruse, S. F., Devesa, S. S., Dickie, L. A., McGlynn, K. A. & Kleiner, D. E. Histological classification of liver and intrahepatic bile duct cancers in seer registries. J Registry Manag. 38, 201–205 (2011).
pubmed: 23270094
Hass, H. G., Vogel, U., Scheurlen, M. & Jobst, J. Subclassification and detection of new markers for the discrimination of primary liver tumors by gene expression analysis using oligonucleotide arrays. Gut Liver. 12, 306–315 (2018).
doi: 10.5009/gnl17277
pubmed: 29271183
Wu, G. et al. Importance of tumor size at diagnosis as a prognostic factor for hepatocellular carcinoma survival: a population-based study. Cancer Manag Res. 10, 4401–4410 (2018).
doi: 10.2147/CMAR.S177663
pubmed: 30349373
pmcid: 6188157
Yan, C. et al. Spatial distribution of tumor-infiltrating t cells indicated immune response status under chemoradiotherapy plus pd-1 blockade in esophageal cancer. Front Immunol. 14, 1138054. https://doi.org/10.3389/fimmu.2023.1138054 (2023).
doi: 10.3389/fimmu.2023.1138054
pubmed: 37275884
pmcid: 10235618
Yin, Y. et al. High density and proximity of cd8(+) t cells to tumor cells are correlated with better response to nivolumab treatment in metastatic pleural mesothelioma. Thorac Cancer. 14, 1991–2000. https://doi.org/10.1111/1759-7714.14981 (2023).
doi: 10.1111/1759-7714.14981
pubmed: 37253418
pmcid: 10344741
Gide, T. N. et al. Close proximity of immune and tumor cells underlies response to anti-pd-1 based therapies in metastatic melanoma patients. Oncoimmunology.9, 1659093. https://doi.org/10.1080/2162402X.2019.1659093 (2020).
doi: 10.1080/2162402X.2019.1659093
pubmed: 32002281
Asgari Taghanaki, S., Abhishek, K., Cohen, J. P., Cohen-Adad, J. & Hamarneh, G. Deep semantic segmentation of natural and medical images: a review. Artificial Intelligence Review. 54, 137–178 (2021).
Marini, N. et al. Unleashing the potential of digital pathology data by training computer-aided diagnosis models without human annotations. npj Digital Medicine. 5, 1–18 (2022).
Ilse, M., Tomczak, J. & Welling, M. Attention-based deep multiple instance learning. Proceedings of the 35th International Conference on Machine Learning (2018).
Ronneberger, O., Fischer, P. & Brox, T. U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical image computing and computer-assisted intervention, 234–241 (Springer, 2015).
Bulten, W. et al. Epithelium segmentation using deep learning in h &e-stained prostate specimens with immunohistochemistry as reference standard. Sci rep. 9, 1–10 (2019).
doi: 10.1038/s41598-018-37257-4
Van Rijthoven, M., Balkenhol, M., Siliņa, K., Van Der Laak, J. & Ciompi, F. Hooknet: Multi-resolution convolutional neural networks for semantic segmentation in histopathology whole-slide images. Med Image Anal 68, 101890 (2021).
doi: 10.1016/j.media.2020.101890
pubmed: 33260110
Burlutskiy, N., Gu, F., Wilen, L. K., Backman, M. & Micke, P. A deep learning framework for automatic diagnosis in lung cancer. International Conference on Medical Imaging with Deep Learning (2018).
Bowles, C. et al. Gan augmentation: Augmenting training data using generative adversarial networks. arXiv preprint arXiv:1810.10863 (2018).
Hou, L. et al. Unsupervised histopathology image synthesis. arXiv preprint arXiv:1712.05021 (2017).
Mahmood, F. et al. Deep adversarial training for multi-organ nuclei segmentation in histopathology images. IEEE transactions on medical imaging 39, 3257–3267 (2019).
doi: 10.1109/TMI.2019.2927182
Ishida, T., Niu, G., Hu, W. & Sugiyama, M. Learning from complementary labels. Advances in neural information processing systems 30 (2017).
Ishida, T., Niu, G., Menon, A. & Sugiyama, M. Complementary-label learning for arbitrary losses and models. In International Conference on Machine Learning, 2971–2980 (PMLR, 2019).
Yu, X., Liu, T., Gong, M. & Tao, D. Learning with biased complementary labels. In Proceedings of the European conference on computer vision (ECCV), 68–83 (2018).
Rezaei, M., Yang, H. & Meinel, C. Recurrent generative adversarial network for learning imbalanced medical image semantic segmentation. Multim. Tools Appl. 79, 15329–15348 (2020).
doi: 10.1007/s11042-019-7305-1
Chen, M. et al. Classification and mutation prediction based on histopathology h &e images in liver cancer using deep learning. NPJ precision oncology. 4, 1–7 (2020).
doi: 10.1038/s41698-020-0120-3
Aziz, M. A. et al. Enhancing automatic classification of hepatocellular carcinoma images through image masking, tissue changes and trabecular features. J. Pathol. Inform. 6, 26 (2015).
doi: 10.4103/2153-3539.158044
pubmed: 26110093
pmcid: 4470016
Huang, W.-C. et al. Automatic hcc detection using convolutional network with multi-magnification input images. In 2019 IEEE International Conference on Artificial Intelligence Circuits and Systems (AICAS), 194–198 (IEEE, 2019).
Atupelage, C. et al. Computational hepatocellular carcinoma tumor grading based on cell nuclei classification. J. of Medi. Imaging. 1, 034501 (2014).
doi: 10.1117/1.JMI.1.3.034501
pubmed: 26158066
pmcid: 4479247
Kiani, A. et al. Impact of a deep learning assistant on the histopathologic classification of liver cancer. NPJ Digital Medicine 3, 1–8 (2020).
doi: 10.1038/s41746-020-0232-8
Lin, T.-Y., Goyal, P., Girshick, R., He, K. & Dollár, P. Focal loss for dense object detection. In Proceedings of the IEEE international conference on computer vision, 2980–2988 (2017).
Loshchilov, I. & Hutter, F. Decoupled weight decay regularization. In International Conference on Learning Representations (2018).
Bokhorst, J.-M. et al. Learning from sparsely annotated data for semantic segmentation in histopathology images. In International Conference on Medical Imaging with Deep Learning (2018).
Ruderman, D. L., Cronin, T. W. & Chiao, C.-C. Statistics of cone responses to natural images: implications for visual coding. JOSA A. 15, 2036–2045 (1998).
doi: 10.1364/JOSAA.15.002036
Reinhard, E., Adhikhmin, M., Gooch, B. & Shirley, P. Color transfer between images. IEEE Computer graphics and applications 21, 34–41 (2001).
doi: 10.1109/38.946629
Laine, S. & Aila, T. Temporal ensembling for semi-supervised learning. arXiv preprint arXiv:1610.02242 (2016).
Ouali, Y., Hudelot, C. & Tami, M. Semi-supervised semantic segmentation with cross-consistency training. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 12674–12684 (2020).
Haegele, M. Miriamhaegele/complementary-loss-segmentation:v1.0, https://doi.org/10.5281/zenodo.13772874 (2024).