Technical note: Determining the applicability of a clinical knowledge-based learning model via prospective outlier detection.
automated planning
knowledge-based planning
outlier detection
Journal
Medical physics
ISSN: 2473-4209
Titre abrégé: Med Phys
Pays: United States
ID NLM: 0425746
Informations de publication
Date de publication:
Apr 2022
Apr 2022
Historique:
revised:
17
01
2022
received:
18
11
2021
accepted:
20
01
2022
pubmed:
15
2
2022
medline:
14
4
2022
entrez:
14
2
2022
Statut:
ppublish
Résumé
Knowledge-based planning (KBP) is increasingly implemented clinically because of its demonstrated ability to improve treatment planning efficiency and reduce plan quality variations. However, cases with large dose-volume histogram (DVH) prediction uncertainties may still need manual adjustments by the planner to achieve high plan quality. The purpose of this study is to develop a data-driven method to detect patients with high prediction uncertainties so that intentional effort is directed to these patients. We apply an anomaly detection method known as the local outlier factor (LOF) to a dataset consisting of the training set and each of the prospective patients considered, to evaluate their likelihood of being an anomaly when compared with the training cases. Features used in the LOF analysis include anatomical features and the model-generated DVH principal component scores. To test the efficacy of the proposed model, 142 prostate patients were retrieved from the clinical database and split into a training dataset of 100 patients and a test dataset of 42 patients. The outlier identification performance was quantified by the difference between the DVH prediction root-mean-squared errors (RMSE) of the identified outlier cases and that of the remaining inlier cases. With a predefined LOF threshold of 1.4, the inlier cases achieved average RMSEs of 5.0 and 6.7 for bladder and rectum, while the outlier cases have substantially higher RMSEs of 6.7 and 13.0 in comparison. We propose a method that can determine the prospective patient's outlier status. This method can be integrated into existing automated treatment planning workflows to reduce the risk of generating suboptimal treatment plans while providing an upfront alert to the treatment planner.
Sections du résumé
BACKGROUND
BACKGROUND
Knowledge-based planning (KBP) is increasingly implemented clinically because of its demonstrated ability to improve treatment planning efficiency and reduce plan quality variations. However, cases with large dose-volume histogram (DVH) prediction uncertainties may still need manual adjustments by the planner to achieve high plan quality.
PURPOSE
OBJECTIVE
The purpose of this study is to develop a data-driven method to detect patients with high prediction uncertainties so that intentional effort is directed to these patients.
METHODS
METHODS
We apply an anomaly detection method known as the local outlier factor (LOF) to a dataset consisting of the training set and each of the prospective patients considered, to evaluate their likelihood of being an anomaly when compared with the training cases. Features used in the LOF analysis include anatomical features and the model-generated DVH principal component scores. To test the efficacy of the proposed model, 142 prostate patients were retrieved from the clinical database and split into a training dataset of 100 patients and a test dataset of 42 patients. The outlier identification performance was quantified by the difference between the DVH prediction root-mean-squared errors (RMSE) of the identified outlier cases and that of the remaining inlier cases.
RESULTS
RESULTS
With a predefined LOF threshold of 1.4, the inlier cases achieved average RMSEs of 5.0 and 6.7 for bladder and rectum, while the outlier cases have substantially higher RMSEs of 6.7 and 13.0 in comparison.
CONCLUSIONS
CONCLUSIONS
We propose a method that can determine the prospective patient's outlier status. This method can be integrated into existing automated treatment planning workflows to reduce the risk of generating suboptimal treatment plans while providing an upfront alert to the treatment planner.
Types de publication
Journal Article
Langues
eng
Sous-ensembles de citation
IM
Pagination
2193-2202Informations de copyright
© 2022 American Association of Physicists in Medicine.
Références
Ge Y, Wu QJ. Knowledge-based planning for intensity-modulated radiation therapy: a review of data-driven approaches. Med Phys. 2019;46(6):2760-2775.
McIntosh C, Welch M, McNiven A, et al. Fully automated treatment planning for head and neck radiotherapy using a voxel-based dose prediction and dose mimicking method. Phys Med Bio. 2017;62(15):5926-5944.
Li X, Zhang J, Sheng Y, et al. Automatic IMRT planning via static field fluence prediction (AIP-SFFP): a deep learning algorithm for real-time prostate treatment planning. Phys Med Biol. 2020;65(17):175014.
Ma L, Gu X, Lu W, et al. Deep learning-based inverse mapping for fluence map prediction. Phys Med Biol. 2020;65(23):235035.
Shen C, Chen L, Jia X. A hierarchical deep reinforcement learning framework for intelligent automatic treatment planning of prostate cancer intensity modulated radiation therapy. Phy Med Biolo. 2021;66(13):134002.
Zhang J, Wang C, Sheng Y, et al. An interpretable planning bot for pancreas stereotactic body radiation therapy. Int J Radiat Oncol Biol Phys. 2021;109(4):1076-1085.
Yuan L, Wu QJ, Yin F-F, Jiang Y, Yoo D, Ge Y. Incorporating single-side sparing in models for predicting parotid dose sparing in head and neck IMRT. Med Phys. 2014;41(2):021728.
Kaderka R, Hild SJ, Bry VN, et al. Wide-scale clinical implementation of knowledge-based planning: an investigation of workforce efficiency, need for post-automation refinement, and data-driven model maintenance. Int J Radiat Oncol Biol Phys. 2021;111:705-715.
Cornell M, Kaderka R, Hild SJ, et al. Noninferiority study of automated knowledge-based planning versus human-driven optimization across multiple disease sites. Int J Radiat Oncol Biol Phys. 2020;106(2):430-439.
Tol JP, Delaney AR, Dahele M, Slotman BJ, Verbakel WFAR. Evaluation of a knowledge-based planning solution for head and neck cancer. Int J Radiat Oncol Biol Phys. 2015;91(3):612-620.
Fogliata A, Belosi F, Clivio A, et al. On the pre-clinical validation of a commercial model-based optimisation engine: application to volumetric modulated arc therapy for patients with lung or prostate cancer. Radiother Oncol. 2014;113(3):385-391.
Cilla S, Ianiro A, Romano C, et al. Template-based automation of treatment planning in advanced radiotherapy: a comprehensive dosimetric and clinical evaluation. Sci Rep. 2020;10(1):423.
Hussein M, Heijmen BJM, Verellen D, Nisbet A. Automation in intensity modulated radiotherapy treatment planning-a review of recent innovations. Br J Radiol. 2018;91(1092):20180270.
Tol JP, Dahele M, Delaney AR, Slotman BJ, Verbakel WFAR. Can knowledge-based DVH predictions be used for automated, individualized quality assurance of radiotherapy treatment plans? Radiat Oncol. 2015;10(1):234.
Zhang J, Ge Y, Sheng Y, et al. Knowledge-based tradeoff hyperplanes for head and neck treatment planning. Int J Radiat Oncol Biol Phys. 2020;106(5):1095-1103.
Babier A, Mahmood R, Mcniven AL, Diamant A, Chan TCY. The importance of evaluating the complete automated knowledge-based planning pipeline. Phys Med. 2020;72:73-79.
Covele BM, Carroll CJ, Moore KL. A practical method to quantify knowledge-based DVH prediction accuracy and uncertainty with reference cohorts. J Appl Clin Med Phys. 2021;22(3):279-284.
Yuan L, Ge Y, Lee WR, Yin FF, Kirkpatrick JP, Wu QJ. Quantitative analysis of the factors which affect the interpatient organ-at-risk dose sparing variation in IMRT plans. Med Phys. 2012;39(11):6868-6878.
Zhu X, Ge Y, Li T, Thongphiew D, Yin F-F, Wu QJ. A planning quality evaluation tool for prostate adaptive IMRT based on machine learning. Med Phys. 2011;38(2):719-726.
Kutner MH, Nachtsheim CJ, Neter J. Applied Linear Regression Models. 4th ed. McGraw-Hill/Irwin; 2004.
Breunig MM, Kriegel H-P, Ng RT, Sander J. LOF: identifying density-based local outliers. J SIGMOD Rec. 2000;29(2):93-104.
Zhang J. An ensemble approach to knowledge-based intensity-modulated radiation therapy planning. Front Oncol. 2018;8:57.
Delaney AR, Tol JP, Dahele M, Cuijpers J, Slotman BJ, Verbakel WFAR. Effect of dosimetric outliers on the performance of a commercial knowledge-based planning solution. Int J Radiat Oncol Biol Phys. 2016;94(3):469-477.
Alpuche Aviles JE, Cordero Marcos MI, Sasaki D, Sutherland K, Kane B, Kuusela E. Creation of knowledge-based planning models intended for large scale distribution: minimizing the effect of outlier plans. J Appl Clin Med Phys. 2018;19(3):215-226.
Sheng Y, Ge Y, Yuan L, Li T, Yin F-F, Wu QJ. Outlier identification in radiation therapy knowledge-based planning: a study of pelvic cases. Med Phys. 2017;44(11):5617-5626.
Nguyen D, Sadeghnejad Barkousaraie A, Bohara G, et al. A comparison of Monte Carlo dropout and bootstrap aggregation on the performance and uncertainty estimation in radiation therapy dose prediction with deep learning neural networks. Phys Med Biol. 2021;66(5):054002.
Vanginderdeuren A, Huet-Dastarac M, Barragan-Montero AM, Lee JA. Estimating uncertainty in radiation oncology dose prediction with dropout and bootstrap in U-Net models. Paper presented at: European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning; October 8, 2021; Bruges, Belgium.
Mcintosh C, Conroy L, Tjong MC, et al. Clinical integration of machine learning for curative-intent radiation treatment of patients with prostate cancer. Nat Med. 2021;27(6):999-1005.
Varian Medical Systems. DVH Estimation Algorithm for RapidPlan, in Eclipse Photon and Electron Algorithms Reference Guide. Varian Medical Systems; 2014.