Integration of deep learning-based image analysis and genomic data in cancer pathology: A systematic review.
Biomarker identification
Cancer
Convolutional neural networks
Multimodal fusion
Omics
Journal
European journal of cancer (Oxford, England : 1990)
ISSN: 1879-0852
Titre abrégé: Eur J Cancer
Pays: England
ID NLM: 9005373
Informations de publication
Date de publication:
01 2022
01 2022
Historique:
received:
10
09
2021
accepted:
11
10
2021
pubmed:
24
11
2021
medline:
29
1
2022
entrez:
23
11
2021
Statut:
ppublish
Résumé
Over the past decade, the development of molecular high-throughput methods (omics) increased rapidly and provided new insights for cancer research. In parallel, deep learning approaches revealed the enormous potential for medical image analysis, especially in digital pathology. Combining image and omics data with deep learning tools may enable the discovery of new cancer biomarkers and a more precise prediction of patient prognosis. This systematic review addresses different multimodal fusion methods of convolutional neural network-based image analyses with omics data, focussing on the impact of data combination on the classification performance. PubMed was screened for peer-reviewed articles published in English between January 2015 and June 2021 by two independent researchers. Search terms related to deep learning, digital pathology, omics, and multimodal fusion were combined. We identified a total of 11 studies meeting the inclusion criteria, namely studies that used convolutional neural networks for haematoxylin and eosin image analysis of patients with cancer in combination with integrated omics data. Publications were categorised according to their endpoints: 7 studies focused on survival analysis and 4 studies on prediction of cancer subtypes, malignancy or microsatellite instability with spatial analysis. Image-based classifiers already show high performances in prognostic and predictive cancer diagnostics. The integration of omics data led to improved performance in all studies described here. However, these are very early studies that still require external validation to demonstrate their generalisability and robustness. Further and more comprehensive studies with larger sample sizes are needed to evaluate performance and determine clinical benefits.
Sections du résumé
BACKGROUND
Over the past decade, the development of molecular high-throughput methods (omics) increased rapidly and provided new insights for cancer research. In parallel, deep learning approaches revealed the enormous potential for medical image analysis, especially in digital pathology. Combining image and omics data with deep learning tools may enable the discovery of new cancer biomarkers and a more precise prediction of patient prognosis. This systematic review addresses different multimodal fusion methods of convolutional neural network-based image analyses with omics data, focussing on the impact of data combination on the classification performance.
METHODS
PubMed was screened for peer-reviewed articles published in English between January 2015 and June 2021 by two independent researchers. Search terms related to deep learning, digital pathology, omics, and multimodal fusion were combined.
RESULTS
We identified a total of 11 studies meeting the inclusion criteria, namely studies that used convolutional neural networks for haematoxylin and eosin image analysis of patients with cancer in combination with integrated omics data. Publications were categorised according to their endpoints: 7 studies focused on survival analysis and 4 studies on prediction of cancer subtypes, malignancy or microsatellite instability with spatial analysis.
CONCLUSIONS
Image-based classifiers already show high performances in prognostic and predictive cancer diagnostics. The integration of omics data led to improved performance in all studies described here. However, these are very early studies that still require external validation to demonstrate their generalisability and robustness. Further and more comprehensive studies with larger sample sizes are needed to evaluate performance and determine clinical benefits.
Identifiants
pubmed: 34810047
pii: S0959-8049(21)01160-6
doi: 10.1016/j.ejca.2021.10.007
pii:
doi:
Types de publication
Journal Article
Research Support, Non-U.S. Gov't
Systematic Review
Langues
eng
Sous-ensembles de citation
IM
Pagination
80-91Commentaires et corrections
Type : CommentIn
Type : CommentIn
Informations de copyright
Copyright © 2021 The Authors. Published by Elsevier Ltd.. All rights reserved.
Déclaration de conflit d'intérêts
Conflict of interest statement The authors declare the following financial interests/personal relationships which may be considered as potential competing interests: T.J.B. would like to disclose that he is the owner of Smart Health Heidelberg GmbH (Handschuhsheimer Landstr. 9/1, 69120 Heidelberg, Germany; https://smarthealth.de) which developed the online teledermatology apps AppDoc (https://online-hautarzt.net) and Intimarzt (https://intimarzt.de) and the online doctor service doc2go (https://doc2go.de), outside of the submitted work. JNK declares consulting services for Owkin, France and Panakeia, UK. No other potential conflicts of interest are reported by any of the authors.