Development and multi-site external validation of a generalizable risk prediction model for bipolar disorder.
Journal
Translational psychiatry
ISSN: 2158-3188
Titre abrégé: Transl Psychiatry
Pays: United States
ID NLM: 101562664
Informations de publication
Date de publication:
25 Jan 2024
25 Jan 2024
Historique:
received:
21
02
2023
accepted:
15
12
2023
revised:
29
11
2023
medline:
26
1
2024
pubmed:
26
1
2024
entrez:
25
1
2024
Statut:
epublish
Résumé
Bipolar disorder is a leading contributor to disability, premature mortality, and suicide. Early identification of risk for bipolar disorder using generalizable predictive models trained on diverse cohorts around the United States could improve targeted assessment of high risk individuals, reduce misdiagnosis, and improve the allocation of limited mental health resources. This observational case-control study intended to develop and validate generalizable predictive models of bipolar disorder as part of the multisite, multinational PsycheMERGE Network across diverse and large biobanks with linked electronic health records (EHRs) from three academic medical centers: in the Northeast (Massachusetts General Brigham), the Mid-Atlantic (Geisinger) and the Mid-South (Vanderbilt University Medical Center). Predictive models were developed and valid with multiple algorithms at each study site: random forests, gradient boosting machines, penalized regression, including stacked ensemble learning algorithms combining them. Predictors were limited to widely available EHR-based features agnostic to a common data model including demographics, diagnostic codes, and medications. The main study outcome was bipolar disorder diagnosis as defined by the International Cohort Collection for Bipolar Disorder, 2015. In total, the study included records for 3,529,569 patients including 12,533 cases (0.3%) of bipolar disorder. After internal and external validation, algorithms demonstrated optimal performance in their respective development sites. The stacked ensemble achieved the best combination of overall discrimination (AUC = 0.82-0.87) and calibration performance with positive predictive values above 5% in the highest risk quantiles at all three study sites. In conclusion, generalizable predictive models of risk for bipolar disorder can be feasibly developed across diverse sites to enable precision medicine. Comparison of a range of machine learning methods indicated that an ensemble approach provides the best performance overall but required local retraining. These models will be disseminated via the PsycheMERGE Network website.
Identifiants
pubmed: 38272862
doi: 10.1038/s41398-023-02720-y
pii: 10.1038/s41398-023-02720-y
doi:
Types de publication
Journal Article
Langues
eng
Sous-ensembles de citation
IM
Pagination
58Subventions
Organisme : U.S. Department of Health & Human Services | NIH | National Institute of Mental Health (NIMH)
ID : R01MH118233
Organisme : U.S. Department of Health & Human Services | NIH | National Institute of Mental Health (NIMH)
ID : R01MH118233
Organisme : U.S. Department of Health & Human Services | NIH | National Institute of Mental Health (NIMH)
ID : R01MH118233
Organisme : U.S. Department of Health & Human Services | NIH | National Institute of Mental Health (NIMH)
ID : R01MH118233
Organisme : U.S. Department of Health & Human Services | NIH | National Institute of Mental Health (NIMH)
ID : R01MH118233
Organisme : U.S. Department of Health & Human Services | NIH | National Institute of Mental Health (NIMH)
ID : R01MH118233
Organisme : U.S. Department of Health & Human Services | NIH | National Institute of Mental Health (NIMH)
ID : R01MH118233
Organisme : U.S. Department of Health & Human Services | NIH | National Institute of Mental Health (NIMH)
ID : R01MH118233
Organisme : U.S. Department of Health & Human Services | NIH | National Institute of Mental Health (NIMH)
ID : R01MH118233
Organisme : U.S. Department of Health & Human Services | NIH | National Institute of Mental Health (NIMH)
ID : R01MH118233
Organisme : U.S. Department of Health & Human Services | NIH | National Institute of Mental Health (NIMH)
ID : R01MH118233
Organisme : U.S. Department of Health & Human Services | NIH | National Institute of Mental Health (NIMH)
ID : R01MH118233
Organisme : U.S. Department of Health & Human Services | NIH | National Institute of Mental Health (NIMH)
ID : R01MH118233
Organisme : U.S. Department of Health & Human Services | NIH | National Institute of Mental Health (NIMH)
ID : R01MH118233
Informations de copyright
© 2024. This is a U.S. Government work and not under copyright protection in the US; foreign copyright protection may apply.
Références
Carvalho AF, Firth J, Vieta E. Bipolar disorder. N Engl J Med. 2020. https://doi.org/10.1056/NEJMra1906193 .
Vigo D, Thornicroft G, Atun R. Estimating the true global burden of mental illness. Lancet Psychiatry. 2016;3:171–8. https://doi.org/10.1016/S2215-0366(15)00505-2 .
doi: 10.1016/S2215-0366(15)00505-2
pubmed: 26851330
Plans L, Barrot C, Nieto E, Rios J, Schulze TG, Papiol S, et al. Association between completed suicide and bipolar disorder: a systematic review of the literature. J Affect Disord. 2019;242:111–22. https://doi.org/10.1016/j.jad.2018.08.054 .
doi: 10.1016/j.jad.2018.08.054
pubmed: 30173059
Weye N, Momen NC, Christensen MK, Iburg KM, Dalsgaard S, Laursen TM, et al. Association of specific mental disorders with premature mortality in the Danish population using alternative measurement methods. JAMA Netw Open. 2020;3:e206646. https://doi.org/10.1001/jamanetworkopen.2020.6646 .
doi: 10.1001/jamanetworkopen.2020.6646
pubmed: 32492163
pmcid: 7272122
Drancourt N, Etain B, Lajnef M, Henry C, Raust A, Cochet B, et al. Duration of untreated bipolar disorder: missed opportunities on the long road to optimal treatment. Acta Psychiatr Scand. 2013;127:136–44. https://doi.org/10.1111/j.1600-0447.2012.01917.x .
doi: 10.1111/j.1600-0447.2012.01917.x
pubmed: 22901015
Fritz K, Russell AMT, Allwang C, Kuiper S, Lampe L, Malhi GS. Is a delay in the diagnosis of bipolar disorder inevitable? Bipolar Disord. 2017;19:396–400. https://doi.org/10.1111/bdi.12499 .
doi: 10.1111/bdi.12499
pubmed: 28544121
Dagani J, Signorini G, Nielssen O, Bani M, de Pastore A, et al. Meta-analysis of the interval between the onset and management of bipolar disorder. Can J Psychiatry. 2017;62:247–58. https://doi.org/10.1177/0706743716656607 .
doi: 10.1177/0706743716656607
pubmed: 27462036
Bowden CL. Strategies to reduce misdiagnosis of bipolar depression. Psychiatr Serv. 2001;52:51–55. https://doi.org/10.1176/appi.ps.52.1.51 .
doi: 10.1176/appi.ps.52.1.51
pubmed: 11141528
Altamura AC, Dell’Osso B, Berlin HA, Buoli M, Bassetti R, Mundo E. Duration of untreated illness and suicide in bipolar disorder: a naturalistic study. Eur Arch Psychiatry Clin Neurosci. 2010;260:385–91. https://doi.org/10.1007/s00406-009-0085-2 .
doi: 10.1007/s00406-009-0085-2
pubmed: 19911248
Altamura AC, Buoli M, Caldiroli A, Caron L, Melter CC, Dobrea C, et al. Misdiagnosis, duration of untreated illness (DUI) and outcome in bipolar patients with psychotic symptoms: a naturalistic study. J Affect Disord. 2015;182:70–75. https://doi.org/10.1016/j.jad.2015.04.024 .
doi: 10.1016/j.jad.2015.04.024
pubmed: 25978716
Saraf G, Moazen-Zadeh E, Pinto JV, Ziafat K, Torres IJ, Kesavan M, et al. Early intervention for people at high risk of developing bipolar disorder: a systematic review of clinical trials. Lancet Psychiatry. 2020. https://doi.org/10.1016/S2215-0366(20)30188-7 .
Yang J, Visscher PM, Wray NR. Sporadic cases are the norm for complex disease. Eur J Hum Genet. 2010;18:1039–43. https://doi.org/10.1038/ejhg.2009.177 .
doi: 10.1038/ejhg.2009.177
pubmed: 19826454
Chen C-Y, Lee PH, Castro VM, Minnier J, Charney AW, Stahl EA, et al. Genetic validation of bipolar disorder identified by automated phenotyping using electronic health records. Transl Psychiatry. 2018;8:86 https://doi.org/10.1038/s41398-018-0133-7 .
doi: 10.1038/s41398-018-0133-7
pubmed: 29666432
pmcid: 5904248
Castro VM, Minnier J, Murphy SN, Kohane I, Churchill SE, Gainer V, et al. Validation of electronic health record phenotyping of bipolar disorder cases and controls. Am J Psychiatry. 2015;172:363–72. https://doi.org/10.1176/appi.ajp.2014.14030423 .
doi: 10.1176/appi.ajp.2014.14030423
pubmed: 25827034
Nelson SJ, Zeng K, Kilbourne J, Powell T, Moore R. Normalized names for clinical drugs: RxNorm at 6years. J Am Med Inf Assoc. 2011;18:441–8. https://doi.org/10.1136/amiajnl-2011-000116 .
doi: 10.1136/amiajnl-2011-000116
Healthcare Cost and Utilization Project (HCUP). HCUP Clinical Classifications Software (CCS) for ICD-9-CM. Agency for Healthcare Research and Quality; 2006. www.hcup-us.ahrq.gov/toolssoftware/ccs/ccs.jsp . Accessed 1 May 2017.
Hoerl AE, Kennard RW. Ridge regression: biased estimation for nonorthogonal problems. Technometrics. 1970;12:55–67. https://doi.org/10.1080/00401706.1970.10488634 .
doi: 10.1080/00401706.1970.10488634
Simon N, Friedman J, Hastie T, Tibshirani R. Regularization paths for Cox’s proportional hazards model via coordinate descent. J Stat Softw. 2011;39:1–13. https://doi.org/10.18637/jss.v039.i05 .
doi: 10.18637/jss.v039.i05
pubmed: 27065756
pmcid: 4824408
Friedman AJ, Hastie T, Simon N, Tibshirani R, Hastie MT. Package ‘ glmnet ’. 2015. https://glmnet.stanford.edu/ .
Friedman J, Hastie T, Tibshirani R. Regularization paths for generalized linear models via coordinate descent. J Stat Softw. 2010;33:1–22.
doi: 10.18637/jss.v033.i01
pubmed: 20808728
pmcid: 2929880
Wright MN, Ziegler A. ranger: a fast implementation of random forests for high dimensional data in C++ and R. 2015. http://arxiv.org/abs/1508.04409 .
Ke G, Meng Q, Finley T, Wang T, Chen W, Ma W, et al. LightGBM: a highly efficient gradient boosting decision tree. Adv Neural Inf Process Syst. 2017;30:3146–54.
Rembold CM. Number needed to screen: development of a statistic for disease screening. BMJ. 1998;317:307–312.
doi: 10.1136/bmj.317.7154.307
pubmed: 9685274
pmcid: 28622
Steyerberg E. Clinical prediction models: a practical approach to development, validation, and updating. Springer-Verlag; 2009. www.springer.com/us/book/9780387772431 . Accessed October 30, 2018.
Smoller JW, Finn CT. Family, twin, and adoption studies of bipolar disorder. Am J Med Genet C Semin Med Genet. 2003;123C:48–58. https://doi.org/10.1002/ajmg.c.20013 .
doi: 10.1002/ajmg.c.20013
pubmed: 14601036
Gilman SE, Ni MY, Dunn EC, Breslau J, McLaughlin KA, Smoller JW, et al. Contributions of the social environment to first-onset and recurrent mania. Mol Psychiatry. 2015;20:329–336. https://doi.org/10.1038/mp.2014.36 .
doi: 10.1038/mp.2014.36
pubmed: 24751965
Pradier MF, Hughes MC, McCoy TH, Barroilhet SA, Doshi-Velez F, Perlis RH. Predicting change in diagnosis from major depression to bipolar disorder after antidepressant initiation. Neuropsychopharmacology. 2021;46:455–461. https://doi.org/10.1038/s41386-020-00838-x .
doi: 10.1038/s41386-020-00838-x
pubmed: 32927464
Rabelo-da-Ponte FD, Feiten JG, Mwangi B, Barros FC, Wehrmeister FC, Menezes AM, et al. Early identification of bipolar disorder among young adults – a 22-year community birth cohort. Acta Psychiatr Scand. 2020;142:476–485. https://doi.org/10.1111/acps.13233 .
doi: 10.1111/acps.13233
pubmed: 32936930
Walsh CG, Chaudhry B, Dua P, Goodman KW, Kaplan B, Kavuluru R, et al. Stigma, biomarkers, and algorithmic bias: recommendations for precision behavioral health with artificial intelligence. Jamia Open. 2020;3:9–15. https://doi.org/10.1093/jamiaopen/ooz054 .
doi: 10.1093/jamiaopen/ooz054
pubmed: 32607482
pmcid: 7309258
Obermeyer Z, Powers B, Vogeli C, Mullainathan S. Dissecting racial bias in an algorithm used to manage the health of populations. Science. 2019;366:447–453. https://doi.org/10.1126/science.aax2342 .
doi: 10.1126/science.aax2342
pubmed: 31649194
Makadia R, Ryan PB. Transforming the premier perspective hospital database into the observational medical outcomes partnership (OMOP) common data model. EGEMS. 2014;2:1110 https://doi.org/10.13063/2327-9214.1110 .
doi: 10.13063/2327-9214.1110
pubmed: 25848597
pmcid: 4371500
Sugiyama M, Kawanabe M. Machine learning in non-stationary environments: Introduction to Covariate Shift Adaptation. The MIT Press; 2012. https://doi.org/10.7551/mitpress/9780262017091.001.0001 .