The Diagnostic Efficacy of an App-based Diagnostic Health Care Application in the Emergency Room: eRadaR-Trial. A prospective, Double-blinded, Observational Study.
Journal
Annals of surgery
ISSN: 1528-1140
Titre abrégé: Ann Surg
Pays: United States
ID NLM: 0372354
Informations de publication
Date de publication:
01 11 2022
01 11 2022
Historique:
pubmed:
5
8
2022
medline:
12
10
2022
entrez:
4
8
2022
Statut:
ppublish
Résumé
To evaluate the diagnostic accuracy of the app-based diagnostic tool Ada and the impact on patient outcome in the emergency room (ER). Artificial intelligence-based diagnostic tools can improve targeted processes in health care delivery by integrating patient information with a medical knowledge base and a machine learning system, providing clinicians with differential diagnoses and recommendations. Patients presenting to the ER with abdominal pain self-assessed their symptoms using the Ada-App under supervision and were subsequently assessed by the ER physician. Diagnostic accuracy was evaluated by comparing the App-diagnoses with the final discharge diagnoses. Timing of diagnosis and time to treatment were correlated with complications, overall survival, and length of hospital stay. In this prospective, double-blinded study, 450 patients were enrolled and followed up until day 90. Ada suggested the final discharge diagnosis in 52.0% (95% CI [0.47, 0.57]) of patients compared with the classic doctor-patient interaction, which was significantly superior with 80.9% (95% CI [0.77, 0.84], P <0.001). However, when diagnostic accuracy of both were assessed together, Ada significantly increased the accuracy rate (87.3%, P <0.001), when compared with the ER physician alone. Patients with an early time point of diagnosis and rapid treatment allocation exhibited significantly reduced complications ( P< 0.001) and length of hospital stay ( P< 0.001). Currently, the classic patient-physician interaction is superior to an AI-based diagnostic tool applied by patients. However, AI tools have the potential to additionally benefit the diagnostic efficacy of clinicians and improve quality of care.
Sections du résumé
OBJECTIVE
To evaluate the diagnostic accuracy of the app-based diagnostic tool Ada and the impact on patient outcome in the emergency room (ER).
BACKGROUND
Artificial intelligence-based diagnostic tools can improve targeted processes in health care delivery by integrating patient information with a medical knowledge base and a machine learning system, providing clinicians with differential diagnoses and recommendations.
METHODS
Patients presenting to the ER with abdominal pain self-assessed their symptoms using the Ada-App under supervision and were subsequently assessed by the ER physician. Diagnostic accuracy was evaluated by comparing the App-diagnoses with the final discharge diagnoses. Timing of diagnosis and time to treatment were correlated with complications, overall survival, and length of hospital stay.
RESULTS
In this prospective, double-blinded study, 450 patients were enrolled and followed up until day 90. Ada suggested the final discharge diagnosis in 52.0% (95% CI [0.47, 0.57]) of patients compared with the classic doctor-patient interaction, which was significantly superior with 80.9% (95% CI [0.77, 0.84], P <0.001). However, when diagnostic accuracy of both were assessed together, Ada significantly increased the accuracy rate (87.3%, P <0.001), when compared with the ER physician alone. Patients with an early time point of diagnosis and rapid treatment allocation exhibited significantly reduced complications ( P< 0.001) and length of hospital stay ( P< 0.001).
CONCLUSION
Currently, the classic patient-physician interaction is superior to an AI-based diagnostic tool applied by patients. However, AI tools have the potential to additionally benefit the diagnostic efficacy of clinicians and improve quality of care.
Identifiants
pubmed: 35925755
doi: 10.1097/SLA.0000000000005614
pii: 00000658-202211000-00027
doi:
Types de publication
Journal Article
Observational Study
Randomized Controlled Trial
Langues
eng
Sous-ensembles de citation
IM
Pagination
935-942Informations de copyright
Copyright © 2022 Wolters Kluwer Health, Inc. All rights reserved.
Déclaration de conflit d'intérêts
The authors report no conflicts of interest.
Références
Kohn LT, Corrigan JM, Donaldson MS. To Err Is Human: Building a Safer Health System. Washington, DC, 2000.
Graber ML, Franklin N, Gordon R. Diagnostic error in internal medicine. Arch Intern Med. 2005;165:1493–1499.
Okafor N, Payne VL, Chathampally Y, et al. Using voluntary reports from physicians to learn from diagnostic errors in emergency medicine. Emerg Med J. 2016;33:245–252.
Eames J, Eisenman A, Schuster RJ. Disagreement between emergency department admission diagnosis and hospital discharge diagnosis: mortality and morbidity. Diagnosis. 2016;3:23–30.
McWilliams A, Tapp H, Barker J, et al. Cost analysis of the use of emergency departments for primary care services in Charlotte, North Carolina. N C Med J. 2011;72:265–271.
Sutton RT, Pincock D, Baumgart DC, et al. An overview of clinical decision support systems: benefits, risks, and strategies for success. NPJ Digit Med. 2020;3:17.
Semigran HL, Linder JA, Gidengil C, et al. Evaluation of symptom checkers for self diagnosis and triage: audit study. BMJ. 2015;351:h3480.
Miller S, Gilbert S, Virani V, et al. Patients’ utilization and perception of an artificial intelligence-based symptom assessment and advice technology in a british primary care waiting room: exploratory pilot study. JMIR Hum Factors. 2020;7:e19713.
Montazeri M, Multmeier J, Novorol C, et al. Optimization of patient flow in urgent care centers using a digital tool for recording patient symptoms and history: simulation study. JMIR Form Res. 2021;5:e26402.
Gilbert S, Mehl A, Baluch A, et al. How accurate are digital symptom assessment apps for suggesting conditions and urgency advice? A clinical vignettes comparison to GPs. BMJ Open. 2020;10:e040269.
Ceney A, Tolond S, Glowinski A, et al. Accuracy of online symptom checkers and the potential impact on service utilisation. PLoS One. 2021;16:e0254088.
Ronicke S, Hirsch MC, Türk E, et al. Can a decision support system accelerate rare disease diagnosis? Evaluating the potential impact of Ada DX in a retrospective study. Orphanet J Rare Dis. 2019;14:69.
Faqar-Uz-Zaman SF, Filmann N, Mahkovic D, et al. Study protocol for a prospective, double-blinded, observational study investigating the diagnostic accuracy of an app-based diagnostic health care application in an emergency room setting: the eRadaR trial. BMJ Open. 2021;11:e041396.
Bannay A, Chaignot C, Blotière P-O, et al. The best use of the charlson comorbidity index with electronic health care database to predict mortality. Med Care. 2016;54:188–194.
Hall DE, Arya S, Schmid KK, et al. Development and initial validation of the risk analysis index for measuring frailty in surgical populations. JAMA Surg. 2017;152:175–182.
Mayhew D, Mendonca V, Murthy BVS. A review of ASA physical status - historical perspectives and modern developments. Anaesthesia. 2019;74:373–379.
Clavien PA, Barkun J, Oliveira ML de, et al. The Clavien-Dindo classification of surgical complications: five-year experience. Ann Surg. 2009;250:187–196.
Clavien P-A, Vetter D, Staiger RD, et al. The comprehensive complication index (CCI®): added value and clinical perspectives 3 years “down the line”. Ann Surg. 2017;265:1045–1050.
Hirsch MC, Ronicke S, Krusche M, et al. Rare diseases 2030: how augmented AI will support diagnosis and treatment of rare diseases in the future. Ann Rheum Dis. 2020;79:740–743.
Mehl A, Bergey F, Cawley C, et al. Syndromic surveillance insights from a symptom assessment app before and during COVID-19 measures in germany and the united kingdom: results from repeated cross-sectional analyses. JMIR Mhealth Uhealth. 2020;8:e21364.
Jungmann SM, Klan T, Kuhn S, et al. Accuracy of a chatbot (Ada) in the diagnosis of mental disorders: comparative case study with lay and expert users. JMIR Form Res. 2019;3:e13863.
Kawamoto K, Houlihan CA, Balas EA, et al. Improving clinical practice using clinical decision support systems: a systematic review of trials to identify features critical to success. BMJ. 2005;330:765.
Rowland SP, Fitzgerald JE, Holme T, et al. What is the clinical value of mHealth for patients? NPJ Digit Med. 2020;3:4.
Chambers D, Cantrell AJ, Johnson M, et al. Digital and online symptom checkers and health assessment/triage services for urgent health problems: systematic review. BMJ Open. 2019;9:e027743.
Semigran HL, Levine DM, Nundy S, et al. Comparison of physician and computer diagnostic accuracy. JAMA Intern Med. 2016;176:1860–1861.
Berry AC, Berry B, Wang B, et al. Evaluation of baseline health literacy for diagnostic and triage accuracy of online symptom checkers for gastrointestinal illnesses. Am J Gastroenterol. 2016. doi: 10.14309/00000434-201610001-00998
doi: 10.14309/00000434-201610001-00998
Berry AC, Berry NA, Wang B, et al. Symptom checkers versus doctors: a prospective, head-to-head comparison for cough. Clin Respir J. 2020;14:413–415.
Martinez-Franco AI, Sanchez-Mendiola M, Mazon-Ramirez JJ, et al. Diagnostic accuracy in Family Medicine residents using a clinical decision support system (DXplain): a randomized-controlled trial. Diagnosis). 2018;5:71–76.
Riches N, Panagioti M, Alam R, et al. The effectiveness of electronic differential diagnoses (DDX) generators: a systematic review and meta-analysis. PLoS One. 2016;11:e0148991.
Shortliffe EH, Sepúlveda MJ. Clinical decision support in the era of artificial intelligence. JAMA. 2018;320:2199–2200 3.
doi: 3
Fraser H, Coiera E, Wong D. Safety of patient-facing digital symptom checkers. The Lancet. 2018;392:2263–2264.
Wallace W, Chan C, Chidambaram S, et al. The Diagnostic and Triage Accuracy of Digital and Online Symptom Checker Tools: A Systematic Review . 2021. Accessed December 21, 2021. doi: 10.1101/2021.12.21.21268167
Graber ML. The incidence of diagnostic error in medicine. BMJ Qual Saf. 2013;22(Suppl 2):ii21–ii27.