Clinical integration of machine learning for curative-intent radiation treatment of patients with prostate cancer.
Journal
Nature medicine
ISSN: 1546-170X
Titre abrégé: Nat Med
Pays: United States
ID NLM: 9502015
Informations de publication
Date de publication:
06 2021
06 2021
Historique:
received:
23
09
2020
accepted:
20
04
2021
pubmed:
5
6
2021
medline:
20
8
2021
entrez:
4
6
2021
Statut:
ppublish
Résumé
Machine learning (ML) holds great promise for impacting healthcare delivery; however, to date most methods are tested in 'simulated' environments that cannot recapitulate factors influencing real-world clinical practice. We prospectively deployed and evaluated a random forest algorithm for therapeutic curative-intent radiation therapy (RT) treatment planning for prostate cancer in a blinded, head-to-head study with full integration into the clinical workflow. ML- and human-generated RT treatment plans were directly compared in a retrospective simulation with retesting (n = 50) and a prospective clinical deployment (n = 50) phase. Consistently throughout the study phases, treating physicians assessed ML- and human-generated RT treatment plans in a blinded manner following a priori defined standardized criteria and peer review processes, with the selected RT plan in the prospective phase delivered for patient treatment. Overall, 89% of ML-generated RT plans were considered clinically acceptable and 72% were selected over human-generated RT plans in head-to-head comparisons. RT planning using ML reduced the median time required for the entire RT planning process by 60.1% (118 to 47 h). While ML RT plan acceptability remained stable between the simulation and deployment phases (92 versus 86%), the number of ML RT plans selected for treatment was significantly reduced (83 versus 61%, respectively). These findings highlight that retrospective or simulated evaluation of ML methods, even under expert blinded review, may not be representative of algorithm acceptance in a real-world clinical setting when patient care is at stake.
Identifiants
pubmed: 34083812
doi: 10.1038/s41591-021-01359-w
pii: 10.1038/s41591-021-01359-w
doi:
Types de publication
Journal Article
Research Support, Non-U.S. Gov't
Langues
eng
Sous-ensembles de citation
IM
Pagination
999-1005Subventions
Organisme : CIHR
ID : 381340
Pays : Canada
Commentaires et corrections
Type : CommentIn
Type : CommentIn
Références
Liu, X. et al. A comparison of deep learning performance against health-care professionals in detecting diseases from medical imaging: a systematic review and meta-analysis. Lancet Digit. Health 1, e271–e297 (2019).
doi: 10.1016/S2589-7500(19)30123-2
Nagendran, M. et al. Artificial intelligence versus clinicians: systematic review of design, reporting standards, and claims of deep learning studies. Brit. Med. J. 368, m689 (2020).
doi: 10.1136/bmj.m689
Shilo, S., Rossman, H. & Segal, E. Axes of a revolution: challenges and promises of big data in healthcare. Nat. Med. 26, 29–38 (2020).
doi: 10.1038/s41591-019-0727-5
McKinney, S. M. et al. International evaluation of an AI system for breast cancer screening. Nature 577, 89–94 (2020).
Ardila, D. et al. End-to-end lung cancer screening with three-dimensional deep learning on low-dose chest computed tomography. Nat. Med. 25, 954–961 (2019).
Esteva, A. et al. Dermatologist-level classification of skin cancer with deep neural networks. Nature 542, 115–118 (2017).
Hannun, A. Y. et al. Cardiologist-level arrhythmia detection and classification in ambulatory electrocardiograms using a deep neural network. Nat. Med. 25, 65–69 (2019).
doi: 10.1038/s41591-018-0268-3
Hyland, S. L. et al. Early prediction of circulatory failure in the intensive care unit using machine learning. Nat. Med. 26, 364–373 (2020).
doi: 10.1038/s41591-020-0789-4
Hollon, T. C. et al. Near real-time intraoperative brain tumor diagnosis using stimulated Raman histology and deep neural networks. Nat. Med. 26, 52–58 (2020).
doi: 10.1038/s41591-019-0715-9
McCarroll, R. E. et al. Retrospective validation and clinical implementation of automated contouring of organs at risk in the head and neck: a step toward automated radiation treatment planning for low- and middle-income countries. J. Glob. Oncol. 4, 1–11 (2018).
Wijnberge, M. et al. Effect of a machine learning-derived early warning system for intraoperative hypotension vs standard care on depth and duration of intraoperative hypotension during elective noncardiac surgery: the HYPE randomized clinical trial. J. Am. Med. Assoc. 323, 1052–1060 (2020).
doi: 10.1001/jama.2020.0592
Nimri, R. et al. Insulin dose optimization using an automated artificial intelligence-based decision support system in youths with type 1 diabetes. Nat. Med. 26, 1380–1384 (2020).
doi: 10.1038/s41591-020-1045-7
Topol, E. J. High-performance medicine: the convergence of human and artificial intelligence. Nat. Med. 25, 44–56 (2019).
doi: 10.1038/s41591-018-0300-7
Wiens, J. et al. Do no harm: a roadmap for responsible machine learning for health care. Nat. Med. 25, 1337–1340 (2019).
doi: 10.1038/s41591-019-0548-6
Hong, J. C. et al. System for high-intensity evaluation during radiation therapy (SHIELD-RT): a prospective randomized study of machine learning-directed clinical evaluations during radiation and chemoradiation. J. Clin. Oncol. 38, 3652–3661 (2020).
doi: 10.1200/JCO.20.01688
Challener, D. W., Prokop, L. J. & Abu-Saleh, O. The proliferation of reports on clinical scoring systems. J. Am. Med. Assoc. 321, 2405–2406 (2019).
doi: 10.1001/jama.2019.5284
Angus, D. C. Randomized clinical trials of artificial intelligence. J. Am. Med. Assoc. 323, 1043–1045 (2020).
doi: 10.1001/jama.2020.1039
Challen, R. et al. Artificial intelligence, bias and clinical safety. BMJ Qual. Saf. 28, 231–237 (2019).
doi: 10.1136/bmjqs-2018-008370
Parikh, R. B., Teeple, S. & Navathe, A. S. Addressing bias in artificial intelligence in health care. J. Am. Med. Assoc. 322, 2377 (2019).
doi: 10.1001/jama.2019.18058
Komorowski, M., Celi, L. A., Badawi, O., Gordon, A. C. & Faisal, A. A. The artificial intelligence clinician learns optimal treatment strategies for sepsis in intensive care. Nat. Med. 24, 1716–1720 (2018).
He, J. et al. The practical implementation of artificial intelligence technologies in medicine. Nat. Med. 25, 30–36 (2019).
doi: 10.1038/s41591-018-0307-0
Holzinger, A., Langs, G., Denk, H., Zatloukal, K. & Müller, H. Causability and explainability of artificial intelligence in medicine. Wiley Interdiscip. Rev. Data Min. Knowl. Discov. 9, e1312 (2019).
Gaube, S. et al. Do as AI say: susceptibility in deployment of clinical decision-aids. NPJ Digit. Med. 4, 31 (2021).
doi: 10.1038/s41746-021-00385-9
Cornell, M. et al. Noninferiority study of automated knowledge-based planning versus human-driven optimization across multiple disease sites. Int. J. Radiat. Oncol. Biol. Phys. 106, 430–439 (2020).
doi: 10.1016/j.ijrobp.2019.10.036
Peters, L. J. et al. Critical impact of radiotherapy protocol compliance and quality in the treatment of advanced head and neck cancer: results from TROG 02.02. J. Clin. Oncol. 28, 2996–3001 (2010).
doi: 10.1200/JCO.2009.27.4498
Abrams, R. A. et al. Failure to adhere to protocol specified radiation therapy guidelines was associated with decreased survival in RTOG 9704—a phase III trial of adjuvant chemotherapy and chemoradiotherapy for patients with resected adenocarcinoma of the pancreas. Int. J. Radiat. Oncol. Biol. Phys. 82, 809–816 (2012).
doi: 10.1016/j.ijrobp.2010.11.039
Jaffray, D. A. et al. Global Task Force on Radiotherapy for Cancer Control. Lancet Oncol. 16, P1144–P1146 (2015).
McIntosh, C. & Purdie, T. G. Voxel-based dose prediction with multi-patient atlas selection for automated radiotherapy treatment planning. Phys. Med. Biol. 62, 415–431 (2017).
doi: 10.1088/1361-6560/62/2/415
McIntosh, C. & Purdie, T. G. Contextual atlas regression forests: multiple-atlas-based automated dose prediction in radiation therapy. IEEE Trans. Med. Imaging 35, 1000–1012 (2016).
doi: 10.1109/TMI.2015.2505188
McIntosh, C., Welch, M., McNiven, A., Jaffray, D. A. & Purdie, T. G. Fully automated treatment planning for head and neck radiotherapy using a voxel-based dose prediction and dose mimicking method. Phys. Med. Biol. 62, 5926–5944 (2017).
doi: 10.1088/1361-6560/aa71f8
Breiman, L. Random forests. Mach. Learn. 45, 5–32 (2001).
Babier, A., Mahmood, R., McNiven, A. L., Diamant, A. & Chan, T. C. Knowledge-based automated planning with three-dimensional generative adversarial networks. Med. Phys. 47, 297–306 (2020).
Kiser, K. J., Fuller, C. D. & Reed, V. K. Artificial intelligence in radiation oncology treatment planning: a brief overview. J. Med. Artif. Intell. 2, 9 (2019).
doi: 10.21037/jmai.2019.04.02
Siddique, S. & Chow, J. C. Artificial intelligence in radiotherapy. Rep. Pract. Oncol. Radiother. 25, 656–666 (2020).
doi: 10.1016/j.rpor.2020.03.015
Liu, X. et al. Reporting guidelines for clinical trial reports for interventions involving artificial intelligence: the CONSORT-AI extension. Nat. Med. 26, 1364–1374 (2020).
doi: 10.1038/s41591-020-1034-x
Schuirmann, D. J. A comparison of the two one-sided tests procedure and the power approach for assessing the equivalence of average bioavailability. J. Pharmacokinet. Biopharm. 15, 657–680 (1987).