Effect of machine learning models on clinician prediction of postoperative complications: the Perioperative ORACLE randomised clinical trial.
acute kidney injury
anaesthesiology risk assessment
artificial intelligence
clinical trial
machine learning
postoperative complications
postoperative death
Journal
British journal of anaesthesia
ISSN: 1471-6771
Titre abrégé: Br J Anaesth
Pays: England
ID NLM: 0372541
Informations de publication
Date de publication:
10 Sep 2024
10 Sep 2024
Historique:
received:
23
05
2024
revised:
19
07
2024
accepted:
07
08
2024
medline:
12
9
2024
pubmed:
12
9
2024
entrez:
11
9
2024
Statut:
aheadofprint
Résumé
Anaesthesiologists might be able to mitigate risk if they know which patients are at greatest risk for postoperative complications. This trial examined the impact of machine learning models on clinician risk assessment. This single-centre, prospective, randomised clinical trial enrolled surgical patients aged ≥18 yr. Anaesthesiologists and nurse anaesthetists providing remote telemedicine support reviewed electronic health records with (assisted group) or without (unassisted group) reviewing machine learning predictions. Clinicians predicted the likelihood of postoperative 30-day all-cause mortality and postoperative acute kidney injury (AKI) within 7 days. The primary outcome was area under the receiver operating characteristic curve (AUROC) for clinician predictions of mortality and AKI, comparing AUROCs between assisted and unassisted assessments. We analysed 5071 patients (mean [range] age: 58 [18-100] yr; 52% female) assessed by 89 clinicians. Of these, 98 (2.2%) patients died within 30 days of surgery and 450 (11.1%) patients sustained AKI. Clinician predictions agreed with the models more strongly in the assisted vs unassisted group (weighted kappa 0.75 vs 0.62 for death, mean difference: 0.13 [95% CI 0.10-0.17]; and 0.79 vs 0.54 for AKI, mean difference: 0.25 [95% CI 0.21-0.29]). Clinical prediction of death was similar between the assisted (AUROC 0.793) and unassisted (AUROC 0.780) groups (mean difference: 0.013 [95% CI -0.070 to 0.097]; P=0.76). Prediction of AKI had an AUROC of 0.734 in the assisted group vs 0.688 in the unassisted group (difference 0.046 [95% CI -0.003 to 0.091]; P=0.06). Clinician performance was not improved by machine learning assistance. Further work is needed to clarify the role of machine learning in real-time perioperative risk stratification. NCT05042804.
Sections du résumé
BACKGROUND
BACKGROUND
Anaesthesiologists might be able to mitigate risk if they know which patients are at greatest risk for postoperative complications. This trial examined the impact of machine learning models on clinician risk assessment.
METHODS
METHODS
This single-centre, prospective, randomised clinical trial enrolled surgical patients aged ≥18 yr. Anaesthesiologists and nurse anaesthetists providing remote telemedicine support reviewed electronic health records with (assisted group) or without (unassisted group) reviewing machine learning predictions. Clinicians predicted the likelihood of postoperative 30-day all-cause mortality and postoperative acute kidney injury (AKI) within 7 days. The primary outcome was area under the receiver operating characteristic curve (AUROC) for clinician predictions of mortality and AKI, comparing AUROCs between assisted and unassisted assessments.
RESULTS
RESULTS
We analysed 5071 patients (mean [range] age: 58 [18-100] yr; 52% female) assessed by 89 clinicians. Of these, 98 (2.2%) patients died within 30 days of surgery and 450 (11.1%) patients sustained AKI. Clinician predictions agreed with the models more strongly in the assisted vs unassisted group (weighted kappa 0.75 vs 0.62 for death, mean difference: 0.13 [95% CI 0.10-0.17]; and 0.79 vs 0.54 for AKI, mean difference: 0.25 [95% CI 0.21-0.29]). Clinical prediction of death was similar between the assisted (AUROC 0.793) and unassisted (AUROC 0.780) groups (mean difference: 0.013 [95% CI -0.070 to 0.097]; P=0.76). Prediction of AKI had an AUROC of 0.734 in the assisted group vs 0.688 in the unassisted group (difference 0.046 [95% CI -0.003 to 0.091]; P=0.06).
CONCLUSIONS
CONCLUSIONS
Clinician performance was not improved by machine learning assistance. Further work is needed to clarify the role of machine learning in real-time perioperative risk stratification.
CLINICAL TRIAL REGISTRATION
BACKGROUND
NCT05042804.
Identifiants
pubmed: 39261226
pii: S0007-0912(24)00468-9
doi: 10.1016/j.bja.2024.08.004
pii:
doi:
Banques de données
ClinicalTrials.gov
['NCT05042804']
Types de publication
Journal Article
Langues
eng
Sous-ensembles de citation
IM
Informations de copyright
Copyright © 2024 British Journal of Anaesthesia. Published by Elsevier Ltd. All rights reserved.