Machine learning prediction and explanatory models of serious infections in patients with rheumatoid arthritis treated with tofacitinib.
Extreme gradient boosted trees
Infectious diseases
Janus kinase inhibitor
Machine learning
Prediction models
Random forest
Rheumatic diseases
Risk stratification
Support vector machines with linear kernel
Treatment safety
Journal
Arthritis research & therapy
ISSN: 1478-6362
Titre abrégé: Arthritis Res Ther
Pays: England
ID NLM: 101154438
Informations de publication
Date de publication:
27 Aug 2024
27 Aug 2024
Historique:
received:
18
12
2023
accepted:
10
07
2024
medline:
28
8
2024
pubmed:
28
8
2024
entrez:
27
8
2024
Statut:
epublish
Résumé
Patients with rheumatoid arthritis (RA) have an increased risk of developing serious infections (SIs) vs. individuals without RA; efforts to predict SIs in this patient group are ongoing. We assessed the ability of different machine learning modeling approaches to predict SIs using baseline data from the tofacitinib RA clinical trials program. This analysis included data from 19 clinical trials (phase 2, n = 10; phase 3, n = 6; phase 3b/4, n = 3). Patients with RA receiving tofacitinib 5 or 10 mg twice daily (BID) were included in the analysis; patients receiving tofacitinib 11 mg once daily were considered as tofacitinib 5 mg BID. All available patient-level baseline variables were extracted. Statistical and machine learning methods (logistic regression, support vector machines with linear kernel, random forest, extreme gradient boosting trees, and boosted trees) were implemented to assess the association of baseline variables with SI (logistic regression only), and to predict SI using selected baseline variables using 5-fold cross-validation. Missing values were handled individually per prediction model. A total of 8404 patients with RA treated with tofacitinib were eligible for inclusion (15,310 patient-years of total follow-up) of which 473 patients reported SIs. Amongst other baseline factors, age, previous infection, and corticosteroid use were significantly associated with SI. When applying prediction modeling for SI across data from all studies, the area under the receiver operating characteristic (AUROC) curve ranged from 0.656 to 0.739. AUROC values ranged from 0.599 to 0.730 in data from phase 3 and 3b/4 studies, and from 0.563 to 0.643 in data from ORAL Surveillance only. Baseline factors associated with SIs in the tofacitinib RA clinical trial program were similar to established SI risk factors associated with advanced treatments for RA. Furthermore, while model performance in predicting SI was similar to other published models, this did not meet the threshold for accurate prediction (AUROC > 0.85). Thus, predicting the occurrence of SIs at baseline remains challenging and may be complicated by the changing disease course of RA over time. Inclusion of other patient-associated and healthcare delivery-related factors and harmonization of the duration of studies included in the models may be required to improve prediction. ClinicalTrials.gov: NCT00147498; NCT00413660; NCT00550446; NCT00603512; NCT00687193; NCT01164579; NCT00976599; NCT01059864; NCT01359150; NCT02147587; NCT00960440; NCT00847613; NCT00814307; NCT00856544; NCT00853385; NCT01039688; NCT02187055; NCT02831855; NCT02092467.
Sections du résumé
BACKGROUND
BACKGROUND
Patients with rheumatoid arthritis (RA) have an increased risk of developing serious infections (SIs) vs. individuals without RA; efforts to predict SIs in this patient group are ongoing. We assessed the ability of different machine learning modeling approaches to predict SIs using baseline data from the tofacitinib RA clinical trials program.
METHODS
METHODS
This analysis included data from 19 clinical trials (phase 2, n = 10; phase 3, n = 6; phase 3b/4, n = 3). Patients with RA receiving tofacitinib 5 or 10 mg twice daily (BID) were included in the analysis; patients receiving tofacitinib 11 mg once daily were considered as tofacitinib 5 mg BID. All available patient-level baseline variables were extracted. Statistical and machine learning methods (logistic regression, support vector machines with linear kernel, random forest, extreme gradient boosting trees, and boosted trees) were implemented to assess the association of baseline variables with SI (logistic regression only), and to predict SI using selected baseline variables using 5-fold cross-validation. Missing values were handled individually per prediction model.
RESULTS
RESULTS
A total of 8404 patients with RA treated with tofacitinib were eligible for inclusion (15,310 patient-years of total follow-up) of which 473 patients reported SIs. Amongst other baseline factors, age, previous infection, and corticosteroid use were significantly associated with SI. When applying prediction modeling for SI across data from all studies, the area under the receiver operating characteristic (AUROC) curve ranged from 0.656 to 0.739. AUROC values ranged from 0.599 to 0.730 in data from phase 3 and 3b/4 studies, and from 0.563 to 0.643 in data from ORAL Surveillance only.
CONCLUSIONS
CONCLUSIONS
Baseline factors associated with SIs in the tofacitinib RA clinical trial program were similar to established SI risk factors associated with advanced treatments for RA. Furthermore, while model performance in predicting SI was similar to other published models, this did not meet the threshold for accurate prediction (AUROC > 0.85). Thus, predicting the occurrence of SIs at baseline remains challenging and may be complicated by the changing disease course of RA over time. Inclusion of other patient-associated and healthcare delivery-related factors and harmonization of the duration of studies included in the models may be required to improve prediction.
TRIAL REGISTRATION
BACKGROUND
ClinicalTrials.gov: NCT00147498; NCT00413660; NCT00550446; NCT00603512; NCT00687193; NCT01164579; NCT00976599; NCT01059864; NCT01359150; NCT02147587; NCT00960440; NCT00847613; NCT00814307; NCT00856544; NCT00853385; NCT01039688; NCT02187055; NCT02831855; NCT02092467.
Identifiants
pubmed: 39192350
doi: 10.1186/s13075-024-03376-9
pii: 10.1186/s13075-024-03376-9
doi:
Substances chimiques
tofacitinib
87LA6FU830
Piperidines
0
Pyrimidines
0
Pyrroles
0
Protein Kinase Inhibitors
0
Antirheumatic Agents
0
Banques de données
ClinicalTrials.gov
['NCT00853385', 'NCT02187055', 'NCT00814307', 'NCT00413660', 'NCT00847613', 'NCT01164579', 'NCT00687193', 'NCT00856544', 'NCT00603512', 'NCT00976599', 'NCT00960440', 'NCT02831855', 'NCT02092467', 'NCT01039688', 'NCT00147498', 'NCT01359150', 'NCT00550446', 'NCT02147587', 'NCT01059864']
Types de publication
Journal Article
Langues
eng
Sous-ensembles de citation
IM
Pagination
153Informations de copyright
© 2024. The Author(s).
Références
Schooling CM, Jones HE. Clarifying questions about risk factors: predictors versus explanation. Emerg Themes Epidemiol. 2018;15:10.
pubmed: 30116285
pmcid: 6083579
doi: 10.1186/s12982-018-0080-z
Sainani KL. Explanatory versus predictive modeling. PM R. 2014;6:841–4.
pubmed: 25150778
doi: 10.1016/j.pmrj.2014.08.941
Varga TV, Niss K, Estampador AC, Collin CB, Moseley PL. Association is not prediction: a landscape of confused reporting in diabetes - a systematic review. Diabetes Res Clin Pract. 2020;170:108497.
pubmed: 33068662
doi: 10.1016/j.diabres.2020.108497
Tian Z, McLaughlin J, Verma A, Chinoy H, Heald AH. The relationship between rheumatoid arthritis and diabetes mellitus: a systematic review and meta-analysis. Cardiovasc Endocrinol Metab. 2021;10:125–31.
pubmed: 34124603
pmcid: 8189616
doi: 10.1097/XCE.0000000000000244
Smolen JS, Aletaha D, Barton A, Burmester GR, Emery P, Firestein GS, et al. Rheumatoid arthritis. Nat Rev Dis Primers. 2018;4:18001.
pubmed: 29417936
doi: 10.1038/nrdp.2018.1
Berbudi A, Rahmadika N, Tjahjadi AI, Ruslami R. Type 2 diabetes and its impact on the immune system. Curr Diabetes Rev. 2020;16:442–9.
pubmed: 31657690
pmcid: 7475801
Donath MY, Shoelson SE. Type 2 diabetes as an inflammatory disease. Nat Rev Immunol. 2011;11:98–107.
pubmed: 21233852
doi: 10.1038/nri2925
Mahler M, Martinez-Prat L, Sparks JA, Deane KD. Precision medicine in the care of rheumatoid arthritis: focus on prediction and prevention of future clinically-apparent disease. Autoimmun Rev. 2020;19:102506.
pubmed: 32173516
doi: 10.1016/j.autrev.2020.102506
Doran MF, Crowson CS, Pond GR, O’Fallon WM, Gabriel SE. Frequency of infection in patients with rheumatoid arthritis compared with controls: a population-based study. Arthritis Rheum. 2002;46:2287–93.
pubmed: 12355475
doi: 10.1002/art.10524
Pawar A, Desai RJ, Gautam N, Kim SC. Risk of admission to hospital for serious infection after initiating tofacitinib versus biologic DMARDs in patients with rheumatoid arthritis: a multidatabase cohort study. Lancet Rheumatol. 2020;2:E84–98.
pubmed: 38263664
doi: 10.1016/S2665-9913(19)30137-7
Galloway JB, Hyrich KL, Mercer LK, Dixon WG, Fu B, Ustianowski AP, et al. Anti-TNF therapy is associated with an increased risk of serious infections in patients with rheumatoid arthritis especially in the first 6 months of treatment: updated results from the British Society for Rheumatology Biologics Register with special emphasis on risks in the elderly. Rheumatology (Oxford). 2011;50:124–31.
pubmed: 20675706
doi: 10.1093/rheumatology/keq242
Cohen SB, Tanaka Y, Mariette X, Curtis JR, Lee EB, Nash P, et al. Long-term safety of tofacitinib up to 9.5 years: a comprehensive integrated analysis of the rheumatoid arthritis clinical development programme. RMD Open. 2020;6:e001395.
pubmed: 33127856
pmcid: 7722371
doi: 10.1136/rmdopen-2020-001395
Balanescu A, Citera G, Pascual-Ramos V, Bhatt DL, Connell CA, Gold D, et al. Infections in patients with rheumatoid arthritis receiving tofacitinib versus tumour necrosis factor inhibitors: results from the open-label, randomised controlled ORAL Surveillance trial. Ann Rheum Dis. 2022;81:1491–503.
pubmed: 35922124
doi: 10.1136/ard-2022-222405
Salmon JH, Gottenberg JE, Ravaud P, Cantagrel A, Combe B, Flipo RM, et al. Predictive risk factors of serious infections in patients with rheumatoid arthritis treated with abatacept in common practice: results from the Orencia and Rheumatoid Arthritis (ORA) registry. Ann Rheum Dis. 2016;75:1108–13.
pubmed: 26048170
doi: 10.1136/annrheumdis-2015-207362
Strangfeld A, Eveslage M, Schneider M, Bergerhausen HJ, Klopsch T, Zink A, et al. Treatment benefit or survival of the fittest: what drives the time-dependent decrease in serious infection rates under TNF inhibition and what does this imply for the individual patient? Ann Rheum Dis. 2011;70:1914–20.
pubmed: 21791449
doi: 10.1136/ard.2011.151043
Yang C, Williams RD, Swerdel JN, Almeida JR, Brouwer ES, Burn E, et al. Development and external validation of prediction models for adverse health outcomes in rheumatoid arthritis: a multinational real-world cohort analysis. Semin Arthritis Rheum. 2022;56:152050.
pubmed: 35728447
doi: 10.1016/j.semarthrit.2022.152050
Krabbe S, Grøn KL, Glintborg B, Nørgaard M, Mehnert F, Jarbøl DE, et al. Risk of serious infections in arthritis patients treated with biological drugs: a matched cohort study and development of prediction model. Rheumatology (Oxford). 2021;60:3834–44.
pubmed: 33493342
Zink A, Manger B, Kaufmann J, Eisterhues C, Krause A, Listing J, et al. Evaluation of the RABBIT risk score for serious infections. Ann Rheum Dis. 2014;73:1673–6.
pubmed: 23740236
doi: 10.1136/annrheumdis-2013-203341
Kremer JM, Bloom BJ, Breedveld FC, Coombs JH, Fletcher MP, Gruben D, et al. The safety and efficacy of a JAK inhibitor in patients with active rheumatoid arthritis: results of a double-blind, placebo-controlled phase IIa trial of three dosage levels of CP-690,550 versus placebo. Arthritis Rheum. 2009;60:1895–905.
pubmed: 19565475
doi: 10.1002/art.24567
Kremer JM, Cohen S, Wilkinson BE, Connell CA, French JL, Gomez-Reino J, et al. A phase IIb dose-ranging study of the oral JAK inhibitor tofacitinib (CP-690,550) versus placebo in combination with background methotrexate in patients with active rheumatoid arthritis and an inadequate response to methotrexate alone. Arthritis Rheum. 2012;64:970–81.
pubmed: 22006202
doi: 10.1002/art.33419
Fleischmann R, Cutolo M, Genovese MC, Lee EB, Kanik KS, Sadis S, et al. Phase IIb dose-ranging study of the oral JAK inhibitor tofacitinib (CP-690,550) or adalimumab monotherapy versus placebo in patients with active rheumatoid arthritis with an inadequate response to disease-modifying antirheumatic drugs. Arthritis Rheum. 2012;64:617–29.
pubmed: 21952978
doi: 10.1002/art.33383
Tanaka Y, Suzuki M, Nakamura H, Toyoizumi S, Zwillich SH, Tofacitinib Study Investigators. Phase II study of tofacitinib (CP-690,550) combined with methotrexate in patients with rheumatoid arthritis and an inadequate response to methotrexate. Arthritis Care Res (Hoboken). 2011;63:1150–8.
pubmed: 21584942
doi: 10.1002/acr.20494
Tanaka Y, Takeuchi T, Yamanaka H, Nakamura H, Toyoizumi S, Zwillich S. Efficacy and safety of tofacitinib as monotherapy in Japanese patients with active rheumatoid arthritis: a 12-week, randomized, phase 2 study. Mod Rheumatol. 2015;25:514–21.
pubmed: 25496464
pmcid: 4819568
doi: 10.3109/14397595.2014.995875
Conaghan PG, Østergaard M, Bowes MA, Wu C, Fuerst T, van der Heijde D, et al. Comparing the effects of tofacitinib, methotrexate and the combination, on bone marrow oedema, synovitis and bone erosion in methotrexate-naive, early active rheumatoid arthritis: results of an exploratory randomised MRI study incorporating semiquantitative and quantitative techniques. Ann Rheum Dis. 2016;75:1024–33.
pubmed: 27002108
doi: 10.1136/annrheumdis-2015-208267
Boyle DL, Soma K, Hodge J, Kavanaugh A, Mandel D, Mease P, et al. The JAK inhibitor tofacitinib suppresses synovial JAK1-STAT signalling in rheumatoid arthritis. Ann Rheum Dis. 2015;74:1311–6.
pubmed: 25398374
doi: 10.1136/annrheumdis-2014-206028
McInnes IB, Kim HY, Lee SH, Mandel D, Song YW, Connell CA, et al. Open-label tofacitinib and double-blind atorvastatin in rheumatoid arthritis patients: a randomised study. Ann Rheum Dis. 2014;73:124–31.
pubmed: 23482473
doi: 10.1136/annrheumdis-2012-202442
Winthrop KL, Silverfield J, Racewicz A, Neal J, Lee EB, Hrycaj P, et al. The effect of tofacitinib on pneumococcal and influenza vaccine responses in rheumatoid arthritis. Ann Rheum Dis. 2016;75:687–95.
pubmed: 25795907
doi: 10.1136/annrheumdis-2014-207191
Winthrop KL, Wouters AG, Choy EH, Soma K, Hodge JA, Nduaka CI, et al. The safety and immunogenicity of live zoster vaccination in patients with rheumatoid arthritis before starting tofacitinib: a randomized phase II trial. Arthritis Rheumatol. 2017;69:1969–77.
pubmed: 28845577
pmcid: 5656925
doi: 10.1002/art.40187
Burmester GR, Blanco R, Charles-Schoeman C, Wollenhaupt J, Zerbini C, Benda B, et al. Tofacitinib (CP-690,550) in combination with methotrexate in patients with active rheumatoid arthritis with an inadequate response to tumour necrosis factor inhibitors: a randomised phase 3 trial. Lancet. 2013;381:451–60.
pubmed: 23294500
doi: 10.1016/S0140-6736(12)61424-X
van der Heijde D, Tanaka Y, Fleischmann R, Keystone E, Kremer J, Zerbini C, et al. Tofacitinib (CP-690,550) in patients with rheumatoid arthritis receiving methotrexate: twelve-month data from a twenty-four-month phase III randomized radiographic study. Arthritis Rheum. 2013;65:559–70.
pubmed: 23348607
doi: 10.1002/art.37816
Fleischmann R, Kremer J, Cush J, Schulze-Koops H, Connell CA, Bradley JD, et al. Placebo-controlled trial of tofacitinib monotherapy in rheumatoid arthritis. N Engl J Med. 2012;367:495–507.
pubmed: 22873530
doi: 10.1056/NEJMoa1109071
Kremer J, Li Z-G, Hall S, Fleischmann R, Genovese M, Martin-Mola E, et al. Tofacitinib in combination with nonbiologic disease-modifying antirheumatic drugs in patients with active rheumatoid arthritis: a randomized trial. Ann Intern Med. 2013;159:253–61.
pubmed: 24026258
doi: 10.7326/0003-4819-159-4-201308200-00006
van Vollenhoven RF, Fleischmann R, Cohen S, Lee EB, García Meijide JA, Wagner S, et al. Tofacitinib or adalimumab versus placebo in rheumatoid arthritis. N Engl J Med. 2012;367:508–19.
pubmed: 22873531
doi: 10.1056/NEJMoa1112072
Lee EB, Fleischmann R, Hall S, Wilkinson B, Bradley J, Gruben D, et al. Tofacitinib versus methotrexate in rheumatoid arthritis. N Engl J Med. 2014;370:2377–86.
pubmed: 24941177
doi: 10.1056/NEJMoa1310476
Fleischmann R, Mysler E, Hall S, Kivitz AJ, Moots RJ, Luo Z, et al. Efficacy and safety of tofacitinib monotherapy, tofacitinib with methotrexate, and adalimumab with methotrexate in patients with rheumatoid arthritis (ORAL Strategy): a phase 3b/4, double-blind, head-to-head, randomised controlled trial. Lancet. 2017;390:457–68.
pubmed: 28629665
doi: 10.1016/S0140-6736(17)31618-5
Cohen SB, Pope J, Haraoui B, Mysler E, Diehl A, Lukic T, et al. Efficacy and safety of tofacitinib modified-release 11 mg once daily plus methotrexate in adult patients with rheumatoid arthritis: 24-week open-label phase results from a phase 3b/4 methotrexate withdrawal non-inferiority study (ORAL Shift). RMD Open. 2021;7:e001673.
pubmed: 34103405
pmcid: 8190053
doi: 10.1136/rmdopen-2021-001673
Ytterberg SR, Bhatt DL, Mikuls TR, Koch GG, Fleischmann R, Rivas JL, et al. Cardiovascular and cancer risk with tofacitinib in rheumatoid arthritis. N Engl J Med. 2022;386:316–26.
pubmed: 35081280
doi: 10.1056/NEJMoa2109927
Sande SZ, Seng L, Li J, D’Agostino R. Statistical learning in medical research with decision threshold and accuracy evaluation. J Data Sci. 2021;19:634–57.
doi: 10.6339/21-JDS1022
Nakatsu RT. An evaluation of four resampling methods used in machine learning classification. IEEE Intell Syst. 2021;36:51–7.
doi: 10.1109/MIS.2020.2978066
Chawla NV, Bowyer KW, Hall LO, Kegelmeyer WP. SMOTE: synthetic minority over-sampling technique. J Artif Intell Res. 2002;16:321–57.
doi: 10.1613/jair.953
Hoang U, Liyanage H, Coyle R, Godden C, Jones S, Blair M, et al. Determinants of inter-practice variation in childhood asthma and respiratory infections: cross-sectional study of a national sentinel network. BMJ Open. 2019;9:e024372.
pubmed: 30679295
pmcid: 6347957
doi: 10.1136/bmjopen-2018-024372
Smith S, Morbey R, de Lusignan S, Pebody RG, Smith GE, Elliot AJ. Investigating regional variation of respiratory infections in a general practice syndromic surveillance system. J Public Health (Oxf). 2021;43:e153–60.
pubmed: 32009178
doi: 10.1093/pubmed/fdaa014
Sunzini F, McInnes I, Siebert S. JAK inhibitors and infections risk: focus on herpes zoster. Ther Adv Musculoskelet Dis. 2020;12:1759720x20936059.
pubmed: 32655703
pmcid: 7328488
doi: 10.1177/1759720X20936059
Redeker I, Albrecht K, Kekow J, Burmester GR, Braun J, Schäfer M, et al. Risk of herpes zoster (shingles) in patients with rheumatoid arthritis under biologic, targeted synthetic and conventional synthetic DMARD treatment: data from the German RABBIT register. Ann Rheum Dis. 2022;81:41–7.
pubmed: 34321218
doi: 10.1136/annrheumdis-2021-220651
Jani M, Barton A, Hyrich K. Prediction of infection risk in rheumatoid arthritis patients treated with biologics: are we any closer to risk stratification? Curr Opin Rheumatol. 2019;31:285–92.
pubmed: 30789850
pmcid: 6443047
doi: 10.1097/BOR.0000000000000598
Scott DL, Steer S. The course of established rheumatoid arthritis. Best Pract Res Clin Rheumatol. 2007;21:943–67.
pubmed: 17870037
doi: 10.1016/j.berh.2007.05.006