Artificial intelligence for multimodal data integration in oncology.

Artificial Intelligence Electronic Health Records Genomics Humans Medical Oncology Radiology
AI in oncology deep learning deep learning in oncology multimodal AI multimodal fusion multimodal integration

Journal

Cancer cell
ISSN: 1878-3686
Titre abrégé: Cancer Cell
Pays: United States
ID NLM: 101130617

Informations de publication

Date de publication:
10 10 2022
Historique:
received: 09 03 2022
revised: 12 07 2022
accepted: 15 09 2022
entrez: 11 10 2022
pubmed: 12 10 2022
medline: 14 10 2022
Statut: ppublish

Résumé

In oncology, the patient state is characterized by a whole spectrum of modalities, ranging from radiology, histology, and genomics to electronic health records. Current artificial intelligence (AI) models operate mainly in the realm of a single modality, neglecting the broader clinical context, which inevitably diminishes their potential. Integration of different data modalities provides opportunities to increase robustness and accuracy of diagnostic and prognostic models, bringing AI closer to clinical practice. AI models are also capable of discovering novel patterns within and across modalities suitable for explaining differences in patient outcomes or treatment resistance. The insights gleaned from such models can guide exploration studies and contribute to the discovery of novel biomarkers and therapeutic targets. To support these advances, here we present a synopsis of AI methods and strategies for multimodal data fusion and association discovery. We outline approaches for AI interpretability and directions for AI-driven exploration through multimodal data interconnections. We examine challenges in clinical adoption and discuss emerging solutions.

Identifiants

DOI: 10.1016/j.ccell.2022.09.012 PMID: 36220072 PMC: PMC10655164
pubmed: 36220072
pii: S1535-6108(22)00441-X
doi: 10.1016/j.ccell.2022.09.012
pmc: PMC10655164
mid: NIHMS1940936
pii:
doi:

Types de publication

Journal Article Review Research Support, N.I.H., Extramural Research Support, Non-U.S. Gov't Research Support, U.S. Gov't, Non-P.H.S.

Langues

eng

Sous-ensembles de citation

IM

Pagination

1095-1110

Subventions

Organisme : NIGMS NIH HHS
ID : R35 GM138216
Pays : United States
Organisme : NCI NIH HHS
ID : T32 CA251062
Pays : United States
Organisme : NHGRI NIH HHS
ID : T32 HG002295
Pays : United States

Informations de copyright

Copyright © 2022 The Author(s). Published by Elsevier Inc. All rights reserved.

Déclaration de conflit d'intérêts

Declaration of interests F.M. and R.J.C. are inventors on a patent related to multimodal learning.

Références

IEEE/ACM Trans Comput Biol Bioinform. 2015 Jul-Aug;12(4):928-37
pubmed: 26357333
Neuro Oncol. 2020 Mar 5;22(3):402-411
pubmed: 31637430
Nat Biomed Eng. 2021 Jun;5(6):522-532
pubmed: 33875840
IEEE Trans Pattern Anal Mach Intell. 2019 Feb;41(2):423-443
pubmed: 29994351
J Clin Med. 2019 Oct 14;8(10):
pubmed: 31614962
Nat Med. 2018 Oct;24(10):1559-1567
pubmed: 30224757
Med Image Anal. 2023 Aug;88:102802
pubmed: 37315483
Eur Urol Focus. 2022 Mar;8(2):472-479
pubmed: 33895087
Nature. 2019 Jul;571(7765):319-321
pubmed: 31316200
N Engl J Med. 2004 Dec 30;351(27):2817-26
pubmed: 15591335
Phys Med Biol. 2017 Jul 24;62(16):6497-6514
pubmed: 28582269
Br J Clin Pharmacol. 2020 Nov;86(11):2114-2122
pubmed: 32358800
Med Image Comput Comput Assist Interv. 2016 Oct;9901:212-220
pubmed: 28149967
Nat Cancer. 2020 Aug;1(8):789-799
pubmed: 33763651
Methods Protoc. 2019 Nov 15;2(4):
pubmed: 31731599
IEEE Trans Med Imaging. 2019 Aug;38(8):1875-1884
pubmed: 30835219
Sci Rep. 2020 Mar 13;10(1):4679
pubmed: 32170141
Nat Commun. 2021 Jan 4;12(1):31
pubmed: 33397893
Nat Commun. 2020 Aug 3;11(1):3877
pubmed: 32747659
Cell Oncol (Dordr). 2019 Jun;42(3):331-341
pubmed: 30825182
Neurooncol Adv. 2020 Jul 17;2(1):vdaa066
pubmed: 32705083
IEEE Trans Med Imaging. 2020 Jan;39(1):99-110
pubmed: 31170067
NPJ Precis Oncol. 2021 Jul 26;5(1):72
pubmed: 34312469
Exp Dermatol. 2018 Nov;27(11):1261-1267
pubmed: 30187575
Sci Rep. 2019 Jan 31;9(1):1103
pubmed: 30705340
Cancer Cell. 2022 Aug 8;40(8):865-878.e6
pubmed: 35944502
Nat Commun. 2020 Nov 16;11(1):5727
pubmed: 33199723
Theranostics. 2020 Sep 2;10(24):11080-11091
pubmed: 33042271
Eur Respir J. 2019 Mar 28;53(3):
pubmed: 30635290
Nat Commun. 2021 Aug 12;12(1):4884
pubmed: 34385460
IEEE Trans Med Imaging. 2022 Apr;41(4):757-770
pubmed: 32881682
NPJ Digit Med. 2020 Oct 16;3:136
pubmed: 33083571
Sci Rep. 2021 Jun 29;11(1):13505
pubmed: 34188098
Int J Gen Med. 2020 Sep 17;13:627-640
pubmed: 32982380
Nat Med. 2020 Sep;26(9):1351-1363
pubmed: 32908284
Radiology. 2019 Jul;292(1):60-66
pubmed: 31063083
Gastroenterology. 2020 Oct;159(4):1406-1416.e11
pubmed: 32562722
J Magn Reson Imaging. 2021 Aug;54(2):462-471
pubmed: 33719168
J Pathol Inform. 2020 Jul 24;11:19
pubmed: 33033656
Transl Cancer Res. 2019 Sep;8(5):1741-1749
pubmed: 35116924
Nat Commun. 2021 Mar 12;12(1):1637
pubmed: 33712598
Annu Int Conf IEEE Eng Med Biol Soc. 2018 Jul;2018:5549-5553
pubmed: 30441594
AJR Am J Roentgenol. 2017 Oct;209(4):920-928
pubmed: 28796549
Technol Cancer Res Treat. 2018 Jan 1;17:1533034618775530
pubmed: 29804518
NPJ Precis Oncol. 2020 Jun 8;4:14
pubmed: 32550270
Entropy (Basel). 2020 Dec 25;23(1):
pubmed: 33375658
Proc Natl Acad Sci U S A. 2018 Mar 27;115(13):E2970-E2979
pubmed: 29531073
Nature. 2021 Jun;594(7861):106-110
pubmed: 33953404
Nat Med. 2020 Sep;26(9):1318-1320
pubmed: 32908274
Nat Med. 2023 May;29(5):1113-1122
pubmed: 37156936
Nat Commun. 2019 Dec 18;10(1):5642
pubmed: 31852890
Nature. 2022 Jan;601(7894):623-629
pubmed: 34875674
AJNR Am J Neuroradiol. 2015 Jan;36(1):77-83
pubmed: 25104288
IEEE Trans Pattern Anal Mach Intell. 2021 Nov;43(11):4037-4058
pubmed: 32386141
Sci Rep. 2020 Dec 17;10(1):22147
pubmed: 33335111
BMC Med Imaging. 2020 Jun 1;20(1):59
pubmed: 32487083
Med Image Anal. 2023 Apr;85:102762
pubmed: 36738650
Int J Radiat Oncol Biol Phys. 2019 Oct 1;105(2):440-447
pubmed: 31201897
Nat Rev Drug Discov. 2005 Nov;4(11):881-2
pubmed: 16299918
Science. 2006 Jul 28;313(5786):504-7
pubmed: 16873662
PLoS One. 2021 Dec 20;16(12):e0261401
pubmed: 34928978
JAMA Oncol. 2016 Nov 01;2(11):1490-1495
pubmed: 27491050
World J Gastroenterol. 2020 Oct 28;26(40):6207-6223
pubmed: 33177794
N Engl J Med. 2009 Apr 2;360(14):1408-17
pubmed: 19339720
NPJ Digit Med. 2020 Oct 27;3:140
pubmed: 33134557
Nat Commun. 2021 Jul 20;12(1):4423
pubmed: 34285218
Clin Cancer Res. 2019 Jul 1;25(13):3753-3758
pubmed: 30787022
Comput Med Imaging Graph. 2022 Jan;95:102027
pubmed: 34959100
Nat Biomed Eng. 2017 Dec;1(12):957-966
pubmed: 31015706
J Oral Maxillofac Pathol. 2018 May-Aug;22(2):279
pubmed: 30158787
JCO Clin Cancer Inform. 2020 Oct;4:885-894
pubmed: 33058693
Diagnostics (Basel). 2021 Aug 03;11(8):
pubmed: 34441338
BMC Med Imaging. 2020 Jul 29;20(1):86
pubmed: 32727387
Comput Soc Netw. 2019;6(1):11
pubmed: 37915858
IEEE Trans Med Imaging. 2020 Oct;39(10):3042-3052
pubmed: 32275587
NPJ Breast Cancer. 2021 Aug 6;7(1):104
pubmed: 34362928
Nat Med. 2020 Jan;26(1):52-58
pubmed: 31907460
Nat Cancer. 2020 Aug;1(8):800-810
pubmed: 35122049
Lancet Digit Health. 2022 Jan;4(1):e8-e17
pubmed: 34952679
Int J Comput Assist Radiol Surg. 2020 Jul;15(7):1215-1223
pubmed: 32372384
Med Image Anal. 2020 Aug;64:101753
pubmed: 32574986
South Asian J Cancer. 2015 Oct-Dec;4(4):163-73
pubmed: 26981507
Med Image Anal. 2019 May;54:280-296
pubmed: 30959445
Radiol Artif Intell. 2020 May 27;2(3):e190043
pubmed: 32510054
BMC Med Inform Decis Mak. 2020 Dec 14;20(Suppl 4):139
pubmed: 33317502
Nat Med. 2021 Apr;27(4):582-584
pubmed: 33820998
Acta Neuropathol. 2016 Jun;131(6):803-20
pubmed: 27157931
Nat Med. 2022 Jan;28(1):154-163
pubmed: 35027755
Clin Nucl Med. 2019 Dec;44(12):956-960
pubmed: 31689276
Sci Rep. 2021 Sep 22;11(1):18800
pubmed: 34552163
Pharmacol Rev. 2018 Jul;70(3):412-445
pubmed: 29669750
Diagnostics (Basel). 2022 Apr 26;12(5):
pubmed: 35626241
Nature. 2021 Oct;598(7879):65-71
pubmed: 34616057
Nat Rev Genet. 2015 Jan;16(1):57-66
pubmed: 25446315
Sci Rep. 2022 Jan 12;12(1):607
pubmed: 35022512
NPJ Digit Med. 2020 Jun 1;3:81
pubmed: 32529043
CA Cancer J Clin. 2020 Mar;70(2):86-104
pubmed: 31944278
Nature. 2012 Apr 18;486(7403):346-52
pubmed: 22522925
Int J Comput Assist Radiol Surg. 2020 Jan;15(1):163-172
pubmed: 31722085
IEEE Trans Med Imaging. 2019 Jun 17;:
pubmed: 31217099

Auteurs

Jana Lipkova (J)

Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA; Department of Pathology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA; Cancer Program, Broad Institute of Harvard and MIT, Cambridge, MA, USA; Data Science Program, Dana-Farber Cancer Institute, Boston, MA, USA.

Richard J Chen (RJ)

Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA; Department of Pathology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA; Cancer Program, Broad Institute of Harvard and MIT, Cambridge, MA, USA; Data Science Program, Dana-Farber Cancer Institute, Boston, MA, USA; Department of Biomedical Informatics, Harvard Medical School, Boston, MA, USA.

Bowen Chen (B)

Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA; Department of Pathology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA; Department of Computer Science, Harvard University, Cambridge, MA, USA.

Ming Y Lu (MY)

Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA; Department of Pathology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA; Cancer Program, Broad Institute of Harvard and MIT, Cambridge, MA, USA; Data Science Program, Dana-Farber Cancer Institute, Boston, MA, USA; Department of Electrical Engineering and Computer Science, Massachusetts Institute of Technology (MIT), Cambridge, MA, USA.

Matteo Barbieri (M)

Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA.

Daniel Shao (D)

Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA; Department of Pathology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA; Harvard-MIT Health Sciences and Technology (HST), Cambridge, MA, USA.

Anurag J Vaidya (AJ)

Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA; Department of Pathology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA; Harvard-MIT Health Sciences and Technology (HST), Cambridge, MA, USA.

Chengkuan Chen (C)

Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA; Department of Pathology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA; Cancer Program, Broad Institute of Harvard and MIT, Cambridge, MA, USA; Data Science Program, Dana-Farber Cancer Institute, Boston, MA, USA.

Luoting Zhuang (L)

Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA; Cancer Program, Broad Institute of Harvard and MIT, Cambridge, MA, USA.

Drew F K Williamson (DFK)

Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA; Department of Pathology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA; Cancer Program, Broad Institute of Harvard and MIT, Cambridge, MA, USA; Data Science Program, Dana-Farber Cancer Institute, Boston, MA, USA.

Muhammad Shaban (M)

Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA; Department of Pathology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA; Cancer Program, Broad Institute of Harvard and MIT, Cambridge, MA, USA; Data Science Program, Dana-Farber Cancer Institute, Boston, MA, USA.

Tiffany Y Chen (TY)

Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA; Department of Pathology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA; Cancer Program, Broad Institute of Harvard and MIT, Cambridge, MA, USA; Data Science Program, Dana-Farber Cancer Institute, Boston, MA, USA.

Faisal Mahmood (F)

Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA; Department of Pathology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA; Cancer Program, Broad Institute of Harvard and MIT, Cambridge, MA, USA; Data Science Program, Dana-Farber Cancer Institute, Boston, MA, USA; Harvard Data Science Initiative, Harvard University, Cambridge, MA, USA. Electronic address: faisalmahmood@bwh.harvard.edu.

Articles similaires

[Redispensing of expensive oral anticancer medicines: a practical application].

Lisanne N van Merendonk, Kübra Akgöl, Bastiaan Nuijen
1.00
Humans Antineoplastic Agents Administration, Oral Drug Costs Counterfeit Drugs

Smoking Cessation and Incident Cardiovascular Disease.

Jun Hwan Cho, Seung Yong Shin, Hoseob Kim et al.
1.00
Humans Male Smoking Cessation Cardiovascular Diseases Female

Evaluation of Low-Value Services Across Major Medicare Advantage Insurers and Traditional Medicare.

Ciara Duggan, Adam L Beckman, Ishani Ganguli et al.
1.00
Humans United States Aged Cross-Sectional Studies Medicare Part C

Effectiveness of Virtual Yoga for Chronic Low Back Pain: A Randomized Clinical Trial.

Hallie Tankha, Devyn Gaskins, Amanda Shallcross et al.
1.00
Humans Yoga Low Back Pain Female Male

Classifications MeSH

questionsmedicales.fr › Sciences de l'information › Méthodologies informatiques › Algorithmes › Intelligence artificielle › Artificial intelligence for multimodal data...
questionsmedicales.fr › Environnement et santé publique › Santé publique › Méthodes épidémiologiques › Collecte de données › Documents › Dossiers médicaux › Systèmes informatisés de dossiers médicaux › Dossiers médicaux électroniques › Artificial intelligence for multimodal data...
questionsmedicales.fr › Disciplines des sciences naturelles › Disciplines des sciences biologiques › Biologie › Génétique › Génomique › Artificial intelligence for multimodal data...
questionsmedicales.fr › Eucaryotes › Animaux › Chordés › Vertébrés › Mammifères › Eutheria › Primates › Haplorhini › Catarrhini › Hominidae › Humains › Artificial intelligence for multimodal data...
questionsmedicales.fr › Professions de santé › Médecine › Médecine interne › Oncologie médicale › Artificial intelligence for multimodal data...
questionsmedicales.fr › Professions de santé › Médecine › Radiologie › Artificial intelligence for multimodal data...