Performance of explainable artificial intelligence in guiding the management of patients with a pancreatic cyst.

Artificial intelligence Explainable AI IPMN Pancreatic cyst Pancreatic neoplasms

Journal

Pancreatology : official journal of the International Association of Pancreatology (IAP) ... [et al.]
ISSN: 1424-3911
Titre abrégé: Pancreatology
Pays: Switzerland
ID NLM: 100966936

Informations de publication

Date de publication:
02 Sep 2024
Historique:
received: 12 06 2024
revised: 12 08 2024
accepted: 01 09 2024
medline: 12 9 2024
pubmed: 12 9 2024
entrez: 11 9 2024
Statut: aheadofprint

Résumé

Pancreatic cyst management can be distilled into three separate pathways - discharge, monitoring or surgery- based on the risk of malignant transformation. This study compares the performance of artificial intelligence (AI) models to clinical care for this task. Two explainable boosting machine (EBM) models were developed and evaluated using clinical features only, or clinical features and cyst fluid molecular markers (CFMM) using a publicly available dataset, consisting of 850 cases (median age 64; 65 % female) with independent training (429 cases) and holdout test cohorts (421 cases). There were 137 cysts with no malignant potential, 114 malignant cysts, and 599 IPMNs and MCNs. The EBM and EBM with CFMM models had higher accuracy for identifying patients requiring monitoring (0.88 and 0.82) and surgery (0.66 and 0.82) respectively compared with current clinical care (0.62 and 0.58). For discharge, the EBM with CFMM model had a higher accuracy (0.91) than either the EBM model (0.84) or current clinical care (0.86). In the cohort of patients who underwent surgical resection, use of the EBM-CFMM model would have decreased the number of unnecessary surgeries by 59 % (n = 92), increased correct surgeries by 7.5 % (n = 11), identified patients who require monitoring by 122 % (n = 76), and increased the number of patients correctly classified for discharge by 138 % (n = 18) compared to clinical care. EBM models had greater sensitivity and specificity for identifying the correct management compared with either clinical management or previous AI models. The model predictions are demonstrated to be interpretable by clinicians.

Sections du résumé

BACKGROUND/OBJECTIVES OBJECTIVE
Pancreatic cyst management can be distilled into three separate pathways - discharge, monitoring or surgery- based on the risk of malignant transformation. This study compares the performance of artificial intelligence (AI) models to clinical care for this task.
METHODS METHODS
Two explainable boosting machine (EBM) models were developed and evaluated using clinical features only, or clinical features and cyst fluid molecular markers (CFMM) using a publicly available dataset, consisting of 850 cases (median age 64; 65 % female) with independent training (429 cases) and holdout test cohorts (421 cases). There were 137 cysts with no malignant potential, 114 malignant cysts, and 599 IPMNs and MCNs.
RESULTS RESULTS
The EBM and EBM with CFMM models had higher accuracy for identifying patients requiring monitoring (0.88 and 0.82) and surgery (0.66 and 0.82) respectively compared with current clinical care (0.62 and 0.58). For discharge, the EBM with CFMM model had a higher accuracy (0.91) than either the EBM model (0.84) or current clinical care (0.86). In the cohort of patients who underwent surgical resection, use of the EBM-CFMM model would have decreased the number of unnecessary surgeries by 59 % (n = 92), increased correct surgeries by 7.5 % (n = 11), identified patients who require monitoring by 122 % (n = 76), and increased the number of patients correctly classified for discharge by 138 % (n = 18) compared to clinical care.
CONCLUSIONS CONCLUSIONS
EBM models had greater sensitivity and specificity for identifying the correct management compared with either clinical management or previous AI models. The model predictions are demonstrated to be interpretable by clinicians.

Identifiants

DOI: 10.1016/j.pan.2024.09.001 PMID: 39261223
pubmed: 39261223
pii: S1424-3903(24)00730-0
doi: 10.1016/j.pan.2024.09.001
pii:
doi:

Types de publication

Journal Article

Langues

eng

Sous-ensembles de citation

IM

Informations de copyright

Copyright © 2024 The Authors. Published by Elsevier B.V. All rights reserved.

Déclaration de conflit d'intérêts

Declaration of competing interest E.K.F. has educational grant support with GE Medical, Siemens Healthineers research grant support, HipGraphics Inc (co-founder and shareholder), Exact Sciences (consultant), Imaging Endpoints (consultant). A.M.L. is a consultant with Exact Sciences. K.W.K. is also a member of the Scientific Advisory Boards of Eisai-Morphotek, Syxmex-Inostics, CAGE, and NeoPhore. These companies, as well as other companies, have licensed technologies from Johns Hopkins University, on which K.W.K. is an inventor. These licenses and relationships are associated with equity or royalty payments to K.W.K. The terms of these arrangements are being managed by Johns Hopkins University in accordance with its conflict of interest policies. D.S.K. is a consultant and equity holder in PAIGE. AI. The terms of all these arrangements are being managed by Johns Hopkins University in accordance with its conflict of interest policies. The following patents are related to this work: Safe Sequencing System US201161476150P, Rapid Aneuploidy Detection US201261615535P, Mutations in pancreatic neoplasms US9976184B2, and Differential identification of pancreatic cysts US9637796B2.

Auteurs

Juan M Lavista Ferres (JM)

AI for Good Lab, Microsoft Corporation, 4330 150th Ave NE, Redmond, WA, 98052, USA.

Felipe Oviedo (F)

AI for Good Lab, Microsoft Corporation, 4330 150th Ave NE, Redmond, WA, 98052, USA.

Caleb Robinson (C)

AI for Good Lab, Microsoft Corporation, 4330 150th Ave NE, Redmond, WA, 98052, USA.

Linda Chu (L)

Department of Radiology, Johns Hopkins University, 1800 Orleans Street, Baltimore, MD, 21287, USA.

Satomi Kawamoto (S)

Department of Radiology, Johns Hopkins University, 1800 Orleans Street, Baltimore, MD, 21287, USA.

Elham Afghani (E)

Department of Medicine, Johns Hopkins University, 1800 Orleans Street, Baltimore, MD, 21287, USA.

Jin He (J)

Department of Surgery, Johns Hopkins University, 1800 Orleans Street, Baltimore, MD, 21287, USA.

Alison P Klein (AP)

Department of Medicine, Johns Hopkins University, 1800 Orleans Street, Baltimore, MD, 21287, USA; Sidney Kimmel Comprehensive Cancer Center, Department of Oncology, Johns Hopkins University, 1800 Orleans Street, Baltimore, MD, 21287, USA; The Sol Goldman Pancreatic Cancer Research Center, Department of Pathology, Johns Hopkins University, 1800 Orleans Street, Baltimore, MD, 21287, USA.

Mike Goggins (M)

The Sol Goldman Pancreatic Cancer Research Center, Department of Pathology, Johns Hopkins University, 1800 Orleans Street, Baltimore, MD, 21287, USA.

Christopher L Wolfgang (CL)

Department of Surgery, New York University School of Medicine, NYU, New York, NY, 10016, USA.

Ammar A Javed (AA)

Department of Surgery, New York University School of Medicine, NYU, New York, NY, 10016, USA.

Rahul Dodhia (R)

AI for Good Lab, Microsoft Corporation, 4330 150th Ave NE, Redmond, WA, 98052, USA.

Nick Papadopolous (N)

Sidney Kimmel Comprehensive Cancer Center, Department of Oncology, Johns Hopkins University, 1800 Orleans Street, Baltimore, MD, 21287, USA; Ludwig Center and Howard Hughes Medical Institute at the Sidney Kimmel Cancer Center, Johns Hopkins University, 1800 Orleans Street, Baltimore, MD, 21287, USA.

Ken Kinzler (K)

Sidney Kimmel Comprehensive Cancer Center, Department of Oncology, Johns Hopkins University, 1800 Orleans Street, Baltimore, MD, 21287, USA; Ludwig Center and Howard Hughes Medical Institute at the Sidney Kimmel Cancer Center, Johns Hopkins University, 1800 Orleans Street, Baltimore, MD, 21287, USA.

Ralph H Hruban (RH)

The Sol Goldman Pancreatic Cancer Research Center, Department of Pathology, Johns Hopkins University, 1800 Orleans Street, Baltimore, MD, 21287, USA.

William B Weeks (WB)

AI for Good Lab, Microsoft Corporation, 4330 150th Ave NE, Redmond, WA, 98052, USA.

Elliot K Fishman (EK)

Department of Radiology, Johns Hopkins University, 1800 Orleans Street, Baltimore, MD, 21287, USA.

Anne Marie Lennon (AM)

Department of Radiology, Johns Hopkins University, 1800 Orleans Street, Baltimore, MD, 21287, USA; Department of Medicine, Johns Hopkins University, 1800 Orleans Street, Baltimore, MD, 21287, USA; Department of Surgery, Johns Hopkins University, 1800 Orleans Street, Baltimore, MD, 21287, USA; Sidney Kimmel Comprehensive Cancer Center, Department of Oncology, Johns Hopkins University, 1800 Orleans Street, Baltimore, MD, 21287, USA; Ludwig Center and Howard Hughes Medical Institute at the Sidney Kimmel Cancer Center, Johns Hopkins University, 1800 Orleans Street, Baltimore, MD, 21287, USA. Electronic address: felipe.oviedo@microsoft.com.

Classifications MeSH