SMART choice (knee) tool: a patient-focused predictive model to predict improvement in health-related quality of life after total knee arthroplasty.
artificial intelligence
health-related quality of life
machine learning
predictive model
total knee arthroplasty
Journal
ANZ journal of surgery
ISSN: 1445-2197
Titre abrégé: ANZ J Surg
Pays: Australia
ID NLM: 101086634
Informations de publication
Date de publication:
01 2023
01 2023
Historique:
revised:
11
12
2022
received:
16
09
2022
accepted:
21
12
2022
pubmed:
14
1
2023
medline:
22
2
2023
entrez:
13
1
2023
Statut:
ppublish
Résumé
Current predictive tools for TKA focus on clinicians rather than patients as the intended user. The purpose of this study was to develop a patient-focused model to predict health-related quality of life outcomes at 1-year post-TKA. Patients who underwent primary TKA for osteoarthritis from a tertiary institutional registry after January 2006 were analysed. The primary outcome was improvement after TKA defined by the minimal clinically important difference in utility score at 1-year post-surgery. Potential predictors included demographic information, comorbidities, lifestyle factors, and patient-reported outcome measures. Four models were developed, including both conventional statistics and machine learning (artificial intelligence) methods: logistic regression, classification tree, extreme gradient boosted trees, and random forest models. Models were evaluated using discrimination and calibration metrics. A total of 3755 patients were included in the study. The logistic regression model performed the best with respect to both discrimination (AUC = 0.712) and calibration (intercept = -0.083, slope = 1.123, Brier score = 0.202). Less than 2% (n = 52) of the data were missing and therefore removed for complete case analysis. The final model used age (categorical), sex, baseline utility score, and baseline Veterans-RAND 12 responses as predictors. The logistic regression model performed better than machine learning algorithms with respect to AUC and calibration plot. The logistic regression model was well calibrated enough to stratify patients into risk deciles based on their likelihood of improvement after surgery. Further research is required to evaluate the performance of predictive tools through pragmatic clinical trials. Level II, decision analysis.
Sections du résumé
BACKGROUND
Current predictive tools for TKA focus on clinicians rather than patients as the intended user. The purpose of this study was to develop a patient-focused model to predict health-related quality of life outcomes at 1-year post-TKA.
METHODS
Patients who underwent primary TKA for osteoarthritis from a tertiary institutional registry after January 2006 were analysed. The primary outcome was improvement after TKA defined by the minimal clinically important difference in utility score at 1-year post-surgery. Potential predictors included demographic information, comorbidities, lifestyle factors, and patient-reported outcome measures. Four models were developed, including both conventional statistics and machine learning (artificial intelligence) methods: logistic regression, classification tree, extreme gradient boosted trees, and random forest models. Models were evaluated using discrimination and calibration metrics.
RESULTS
A total of 3755 patients were included in the study. The logistic regression model performed the best with respect to both discrimination (AUC = 0.712) and calibration (intercept = -0.083, slope = 1.123, Brier score = 0.202). Less than 2% (n = 52) of the data were missing and therefore removed for complete case analysis. The final model used age (categorical), sex, baseline utility score, and baseline Veterans-RAND 12 responses as predictors.
CONCLUSION
The logistic regression model performed better than machine learning algorithms with respect to AUC and calibration plot. The logistic regression model was well calibrated enough to stratify patients into risk deciles based on their likelihood of improvement after surgery. Further research is required to evaluate the performance of predictive tools through pragmatic clinical trials.
LEVEL OF EVIDENCE
Level II, decision analysis.
Types de publication
Journal Article
Langues
eng
Sous-ensembles de citation
IM
Pagination
316-327Subventions
Organisme : HCF Research Foundation
ID : N/A
Informations de copyright
© 2023 The Authors. ANZ Journal of Surgery published by John Wiley & Sons Australia, Ltd on behalf of Royal Australasian College of Surgeons.
Références
Cochrane JA, Flynn T, Wills A, Walker FR, Nilsson M, Johnson SJ. Clinical decision support tools for predicting outcomes in patients undergoing total knee arthroplasty: a systematic review. J. Arthroplasty [Internet] U.S. 2021; 36: 1832-1845.e1.
Myers TG, Ramkumar PN, Ricciardi BF, Urish KL, Kipper J, Ketonis C. Artificial intelligence and orthopaedics: an introduction for clinicians. J. Bone Joint Surg. Am. [Internet] 2020; 102: 830-40.
Garriga C, Sanchez-Santos MT, Judge A et al. Development of a model predicting non-satisfaction 1 year after primary total knee replacement in the UK and transportation to Switzerland. Sci. Rep. [Internet] Engl. 2018; 8: 3380.
Van Onsem S, Van Der Straeten C, Arnout N, Deprez P, Van Damme G, Victor J. A new prediction model for patient satisfaction after total knee arthroplasty. J. Arthroplasty [Internet] U.S. 2016; 31: 2660-2667.e1.
Tolk JJ, Waarsing JEH, Janssen RPA, van Steenbergen LN, Bierma-Zeinstra SMA, Reijman M. Development of preoperative prediction models for pain and functional outcome after Total knee arthroplasty using the Dutch arthroplasty register data. J. Arthroplasty [Internet] U.S. 2020; 35: 690-698.e2.
Kunze KN, Akram F, Fuller BC, Zabawa L, Sporer SM, Levine BR. Internal validation of a predictive model for satisfaction after primary total knee arthroplasty. J. Arthroplasty [Internet] U.S. 2019; 34: 663-70.
Kunze KN, Polce EM, Sadauskas AJ, Levine BR. Development of machine learning algorithms to predict patient dissatisfaction after primary total knee arthroplasty. J. Arthroplasty [Internet] U.S. 2020; 35: 3117-22.
Pronk Y, Peters MCWM, Brinkman J-M. Is patient satisfaction after total knee arthroplasty predictable using patient characteristics and preoperative patient-reported outcomes? J. Arthroplasty [Internet] U.S. 2021; 36: 2458-65.
Pua Y-H, Poon CL-L, Seah FJ-T et al. Predicting individual knee range of motion, knee pain, and walking limitation outcomes following total knee arthroplasty. Acta Orthop. [Internet] Engl. 2019; 90: 179-86.
Twiggs JG, Wakelin EA, Fritsch BA et al. Clinical and statistical validation of a probabilistic prediction tool of total knee arthroplasty outcome. J. Arthroplasty [Internet] U.S. 2019; 34: 2624-31.
Lungu E, Desmeules F, Dionne CE, Belzile EL, Vendittoli P-A. Prediction of poor outcomes six months following total knee arthroplasty in patients awaiting surgery. BMC Musculoskelet. Disord. [Internet] 2014; 15: 299.
Sanchez-Santos MT, Garriga C, Judge A et al. Development and validation of a clinical prediction model for patient-reported pain and function after primary total knee replacement surgery. Sci. Rep. [Internet] Engl. 2018; 8: 3381.
Anis HK, Strnad GJ, Klika AK et al. Developing a personalized outcome prediction tool for knee arthroplasty. Bone Joint J. [Internet] Engl. 2020; 102-B: 1183-93.
Dowsey MM, Spelman T, Choong PFM. Development of a prognostic nomogram for predicting the probability of nonresponse to total knee arthroplasty 1 year after surgery. J. Arthroplasty [Internet] Elsevier 2016; 31: 1654-60.
Pencina MJ, Peterson ED. Moving from clinical trials to precision medicine: the role for predictive modeling. JAMA 2016; 315: 1713-4.
Farrow L, Zhong M, Ashcroft GP, Anderson L, Meek RMD. Interpretation and reporting of predictive or diagnostic machine-learning research in Trauma & Orthopaedics. Bone Joint J. [Internet] 2021; 103-B: 1754-8.
Zhou Y, Gould D, Choong P, Dowsey M, Schilling C. Implementing predictive tools in surgery: a narrative review in the context of orthopaedic surgery. ANZ J. Surg. [Internet] 2022; 92: 3162-9.
Atkins D, Makridis CA, Alterovitz G, Ramoni R, Clancy C. Developing and implementing predictive models in a learning healthcare system: traditional and artificial intelligence approaches in the veterans health administration. Annu. Rev. Biomed. Data Sci. [Internet] 2022; 5: 393-413.
Riddle DL, Sando T, Tarver T et al. Shared decision-making applied to knee arthroplasty: a systematic review of randomized trials. Arthritis Care Res. [Internet] 2021; 73: 1125-33.
Steyerberg EW, Vickers AJ, Cook NR et al. Assessing the performance of prediction models: a framework for traditional and novel measures. Epidemiology [Internet] 2010; 21: 128-38.
Bansback N, Trenaman L, MacDonald KV et al. An online individualised patient decision aid improves the quality of decisions in patients considering total knee arthroplasty in routine care: a randomized controlled trial. Osteoarthritis Cartilage Open [Internet] 2022; 4: 100286.
Bleeker SE, Moll HA, Steyerberg EW et al. External validation is necessary in prediction research: a clinical example. J. Clin. Epidemiol. [Internet] 2003; 56: 826-32.
Scott CEH, Howie CR, MacDonald D, Biant LC. Predicting dissatisfaction following total knee replacement. J. Bone Joint Surg. Br. [Internet] 2010; 92-B: 1253-8.
DeFrance M, Scuderi G. Are 20% of patients actually dissatisfied following total knee arthroplasty? A systematic review of the literature. J. Arthroplasty [Internet] 2022 (In press).
Klem N-R, Smith A, O'Sullivan P et al. What influences patient satisfaction after TKA? A qualitative investigation. Clin. Orthop. Relat. Res. [Internet] 2020; 478: 1850-66.
Klem N-R, Kent P, Smith A et al. Satisfaction after total knee replacement for osteoarthritis is usually high, but what are we measuring? A systematic review. Osteoarthritis Cartilage Open [Internet] 2020; 2: 100032.
Zhou Y, Weeden C, Patten L et al. Evaluating willingness for surgery using the SMART choice (knee) patient prognostic tool for total knee arthroplasty: study protocol for a pragmatic randomised controlled trial. BMC Musculoskelet. Disord. [Internet] 2022; 23: 179.
Collins GS, Reitsma JB, Altman DG, Moons KGM. Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD statement. Ann. Intern. Med. [Internet] 2015; 162: 55-63.
Gould D, Thuraisingam S, Shadbolt C et al. Cohort profile: the St Vincent's Melbourne arthroplasty outcomes (SMART) registry, a pragmatic prospective database defining outcomes in total hip and knee replacement patients. BMJ Open [Internet] 2021; 11: e040408.
Osteoarthritis. Australian Institute of Health and Welfare. [Cited 13 Jan 2022.] Available from URL: https://www.aihw.gov.au/reports/phe/232/osteoarthritis/contents/treatment-management
Brazier JE, Roberts J. The Estimation of a preference-based measure of health from the SF-12. Med Care [Internet] 2004; 42: 851-9.
Bakker C, van der Linden S. Health related utility measurement: an introduction. J. Rheumatol. [Internet] 1995; 22: 1197-9.
Gutacker N, Street A. Use of large-scale HRQoL datasets to generate individualised predictions and inform patients about the likely benefit of surgery. Qual. Life Res. [Internet] 2017; 26: 2497-505.
Schilling C, Dowsey MM, Clarke PM, Choong PF. Using patient-reported outcomes for economic evaluation: getting the timing right. Value Health [Internet] 2016; 19: 945-50.
Obuchowski NA. Receiver operating characteristic curves and their use in radiology. Radiology [Internet] 2003; 229: 3-8.
Vieira SM, Kaymak U, Sousa JMC. Cohen's kappa coefficient as a performance measure for feature selection. In Proceedings of IEEE international conference on fuzzy systems. 2010;1-8.
Hilden J, Habbema JD, Bjerregaard B. The measurement of performance in probabilistic diagnosis. II. Trustworthiness of the exact values of the diagnostic probabilities. Methods Inf. Med. [Internet] 1978; 17: 227-37.
Steyerberg EW, Vergouwe Y. Towards better clinical prediction models: seven steps for development and an ABCD for validation. Eur. Heart J. [Internet] 2014; 35: 1925-31.
Kattan MW, Gerds TA. The index of prediction accuracy: an intuitive measure useful for evaluating risk prediction models. Diagn. Progn. Res. [Internet] 2018; 2: 7.
Brier GW. Verification of forecasts expressed in terms of probability. Mon Weather Rev. [Internet] 1950; 78: 1-3.
Singh JA, O'Byrne M, Harmsen S, Lewallen D. Predictors of moderate-severe functional limitation after primary Total knee arthroplasty (TKA): 4701 TKAs at 2-years and 2935 TKAs at 5-years. Osteoarthritis Cartilage [Internet] Elsevier 2010; 18: 515-21.
Jiang Y, Sanchez-Santos MT, Judge AD, Murray DW, Arden NK. Predictors of patient-reported pain and functional outcomes over 10 years after primary Total knee arthroplasty: a prospective cohort study. J. Arthroplasty [Internet] Elsevier 2017; 32: 92-100.e2.
Ethgen O, Bruyère O, Richy F, Dardennes C, Reginster J-Y. Health-related quality of life in total hip and total knee arthroplasty. A qualitative and systematic review of the literature. J. Bone Joint Surg. Am. 2004; 86: 963-74.
Canovas F, Dagneaux L. Quality of life after total knee arthroplasty. Orthop. Traumatol. Surg. Res. [Internet] Elsevier 2018; 104: S41-6.
Woo BJ, Chen JY, Lai YM, Liow MHL, Lo NN, Yeo SJ. Improvements in functional outcome and quality of life are not sustainable for patients ≥ 68 years old 10 years after total knee arthroplasty. Knee Surg. Sports Traumatol. Arthrosc. [Internet] 2021; 29: 3330-6.
Schilling CG, Dowsey MM, Petrie DJ, Clarke PM, Choong PF. Predicting the long-term gains in health-related quality of life after Total knee arthroplasty. J. Arthroplasty [Internet] 2017; 32: 395-401.e2.
Batista GEAPA, Monard MC. An analysis of four missing data treatment methods for supervised learning. Appl. Artif. Intell. [Internet] 2003; 17: 519-33.
Bertsimas D, Pawlowski C, Zhuo YD. From predictive methods to missing data imputation: an optimization approach. J. Mach. Learn. Res. [Internet] 2017; 18: 7133-71.
Steyerberg EW, van Veen M. Imputation is beneficial for handling missing data in predictive models. J. Clin. Epidemiol. 2007; 60: 979.
Calcagno V, de Mazancourt C. Glmulti: an R package for easy automated model selection with (generalized) linear models. J. Stat. Softw. [Internet] 2010; 34: 1-29.
Therneau TM, Atkinson B, Ripley MB. The rpart package. In: R Foundation for Statistical Computing. UK: Oxford, 2010.
Chen T, He T, Benesty M, et al. Extreme gradient boosting [R package xgboost version 1.2.0.1]; 2020. [Cited 13 Dec 2021.].
Wright MN, Ziegler A. Ranger: A Fast Implementation of Random Forests for High Dimensional Data in C++ and R. arXiv [stat.ML]; 2015.
Hocking RR. A biometrics invited paper. The analysis and selection of variables in linear regression. Biometrics [Internet] 1976; 32: 1-49.
Suthaharan S. Decision tree learning. In: Suthaharan S (ed). Machine Learning Models and Algorithms for Big Data Classification: Thinking with Examples for Effective Learning [Internet]. Boston, MA: Springer US, 2016; 237-69.
https://www.analyticsvidhya.com/blog/2016/04/tree-based-algorithms-complete-tutorial-scratch-in-python/?utm_source=blog&utm_medium=4-ways-split-decision-tree
NJR. Patient decision support tool. The National Joint Registry. [Cited 16 Sept 2022.] Available from URL: https://www.njrcentre.org.uk/patients/patient-decision-support-tool/
Fleisher IT, Thompson MC, Mensah CJ et al. Development and validation of crosswalks between the Western Ontario & McMaster universities osteoarthritis index and hip disability and osteoarthritis outcome score joint replacement/knee injury and osteoarthritis outcome score joint replacement. J. Arthroplasty [Internet] 2022; 37: 1034-1039.e3.
Watters TS, Mather RC 3rd, Browne JA, Berend KR, Lombardi AV Jr, Bolognesi MP. Analysis of procedure-related costs and proposed benefits of using patient-specific approach in total knee arthroplasty. J. Surg. Orthop. Adv. [Internet] 2011; 20: 112-6.
Alba AC, Agoritsas T, Walsh M et al. Discrimination and calibration of clinical prediction models: users' guides to the medical literature. JAMA [Internet] 2017; 318: 1377-84.
Steyerberg EW, Harrell FE. Regression Modeling Strategies: With Applications, to Linear Models, Logistic and Ordinal Regression, and Survival Analysis, 2nd edn, Vol. 72. Heidelberg: Springer, 2016; 1006-7.
Spiegelhalter DJ. Probabilistic prediction in patient management and clinical trials. Stat. Med. [Internet] 1986; 5: 421-33.
Westreich D, Lessler J, Funk MJ. Propensity score estimation: neural networks, support vector machines, decision trees (CART), and meta-classifiers as alternatives to logistic regression. J. Clin. Epidemiol. [Internet] 2010; 63: 826-33.
Osteoarthritis. Arthritis Australia. [Cited 6 Dec 2022.] Available from URL: https://arthritisaustralia.com.au/types-of-arthritis/osteoarthritis/
Osteoarthritis. Arthritis South Australia. [Cited 6 Dec 2022.] Available from URL: https://arthritissa.org.au/types-of-arthritis/osteoarthritis/?gclid=Cj0KCQiAyracBhDoARIsACGFcS57J2F6_pAoenJkAaLGtyGfL83cmojBdDRpy8UeRD1uPCzvylaUURMaAqouEALw_wcB
Osteoarthritis. Arthritis New Zealand. [Cited 6 Dec 2022.] Available from URL: https://www.arthritis.org.nz/osteoarthritis/
Choi Y-J, Ra HJ. Patient satisfaction after total knee arthroplasty. Knee Surg. Relat. Res. [Internet] 2016; 28: 1-15.
Souza JMFS, Ferreira RDS, Lima AJP, de Filho SACP, Albuquerque PCVC. Clinical demographic characteristics of total knee arthroplasty in a university hospital. Acta Ortop. Bras. [Internet] 2016; 24: 300-3.
Vabalas A, Gowen E, Poliakoff E, Casson AJ. Machine learning algorithm validation with a limited sample size. PLoS One [Internet] 2019; 14: e0224365.
Ajiboye AR, Abdullah-Arshah R, Hongwu Q. Evaluating the effect of dataset size on predictive model using supervised learning technique. Int. J. Software Eng. Comput. Sci 2015; 1: 75-84.