Untapped potential of multicenter studies: a review of cardiovascular risk prediction models revealed inappropriate analyses and wide variation in reporting.
Cardiovascular disease
Clinical prediction model
Multicenter
Journal
Diagnostic and prognostic research
ISSN: 2397-7523
Titre abrégé: Diagn Progn Res
Pays: England
ID NLM: 101718985
Informations de publication
Date de publication:
2019
2019
Historique:
received:
13
09
2018
accepted:
03
01
2019
entrez:
17
5
2019
pubmed:
17
5
2019
medline:
17
5
2019
Statut:
epublish
Résumé
Clinical prediction models are often constructed using multicenter databases. Such a data structure poses additional challenges for statistical analysis (clustered data) but offers opportunities for model generalizability to a broad range of centers. The purpose of this study was to describe properties, analysis, and reporting of multicenter studies in the Tufts PACE Clinical Prediction Model Registry and to illustrate consequences of common design and analyses choices. Fifty randomly selected studies that are included in the Tufts registry as multicenter and published after 2000 underwent full-text screening. Simulated examples illustrate some key concepts relevant to multicenter prediction research. Multicenter studies differed widely in the number of participating centers (range 2 to 5473). Thirty-nine of 50 studies ignored the multicenter nature of data in the statistical analysis. In the others, clustering was resolved by developing the model on only one center, using mixed effects or stratified regression, or by using center-level characteristics as predictors. Twenty-three of 50 studies did not describe the clinical settings or type of centers from which data was obtained. Four of 50 studies discussed neither generalizability nor external validity of the developed model. Regression methods and validation strategies tailored to multicenter studies are underutilized. Reporting on generalizability and potential external validity of the model lacks transparency. Hence, multicenter prediction research has untapped potential. This review was not registered.
Sections du résumé
BACKGROUND
BACKGROUND
Clinical prediction models are often constructed using multicenter databases. Such a data structure poses additional challenges for statistical analysis (clustered data) but offers opportunities for model generalizability to a broad range of centers. The purpose of this study was to describe properties, analysis, and reporting of multicenter studies in the Tufts PACE Clinical Prediction Model Registry and to illustrate consequences of common design and analyses choices.
METHODS
METHODS
Fifty randomly selected studies that are included in the Tufts registry as multicenter and published after 2000 underwent full-text screening. Simulated examples illustrate some key concepts relevant to multicenter prediction research.
RESULTS
RESULTS
Multicenter studies differed widely in the number of participating centers (range 2 to 5473). Thirty-nine of 50 studies ignored the multicenter nature of data in the statistical analysis. In the others, clustering was resolved by developing the model on only one center, using mixed effects or stratified regression, or by using center-level characteristics as predictors. Twenty-three of 50 studies did not describe the clinical settings or type of centers from which data was obtained. Four of 50 studies discussed neither generalizability nor external validity of the developed model.
CONCLUSIONS
CONCLUSIONS
Regression methods and validation strategies tailored to multicenter studies are underutilized. Reporting on generalizability and potential external validity of the model lacks transparency. Hence, multicenter prediction research has untapped potential.
REGISTRATION
BACKGROUND
This review was not registered.
Identifiants
pubmed: 31093576
doi: 10.1186/s41512-019-0046-9
pii: 46
pmc: PMC6460661
doi:
Types de publication
Journal Article
Review
Langues
eng
Pagination
6Déclaration de conflit d'intérêts
Not applicableNot applicableThe authors declare that they have no competing interests.Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Références
Ann Intern Med. 2015 Jan 6;162(1):W1-73
pubmed: 25560730
Thromb Haemost. 2012 Sep;108(3):543-53
pubmed: 22871801
J Am Coll Cardiol. 2013 Sep 24;62(13):1144-53
pubmed: 23916938
J Am Heart Assoc. 2013 Mar 18;2(2):e000102
pubmed: 23537808
J Am Heart Assoc. 2013 Jun 21;2(3):e000250
pubmed: 23782923
Stat Med. 2013 Aug 15;32(18):3158-80
pubmed: 23307585
J Vasc Surg. 2013 Oct;58(4):1014-20.e1
pubmed: 23683384
J Am Coll Cardiol. 2013 Nov 12;62(20):1866-76
pubmed: 23850903
Am J Cardiol. 2012 Sep 1;110(5):628-35
pubmed: 22608950
J Am Coll Cardiol. 2014 Mar 4;63(8):788-96
pubmed: 24333491
Eur J Vasc Endovasc Surg. 2012 Sep;44(3):318-24
pubmed: 22682012
J Am Coll Cardiol. 2013 Oct 8;62(15):1384-92
pubmed: 23906859
J Vasc Surg. 2012 Aug;56(2):372-9
pubmed: 22632800
Circulation. 2013 Jan 15;127(2):224-32
pubmed: 23212720
J Clin Epidemiol. 2015 Aug;68(8):847-54
pubmed: 25479685
Heart. 2014 Jul;100(13):1016-23
pubmed: 24740804
PLoS Med. 2009 Jul 21;6(7):e1000097
pubmed: 19621072
Int J Stroke. 2014 Apr;9(3):278-83
pubmed: 23834262
Clin Cancer Res. 2017 Sep 1;23(17):5082-5090
pubmed: 28512173
J Bone Joint Surg Am. 2009 May;91 Suppl 3:80-6
pubmed: 19411504
Stat Med. 2018 May 30;37(12):2034-2052
pubmed: 29575170
Eur J Vasc Endovasc Surg. 2013 May;45(5):488-96
pubmed: 23434110
J Clin Epidemiol. 2018 Nov;103:131-133
pubmed: 30063954
CMAJ. 2013 Aug 6;185(11):E537-44
pubmed: 23798453
J Am Heart Assoc. 2014 Jan 02;3(1):e000467
pubmed: 24385451
BMJ. 2014 Oct 15;349:g5920
pubmed: 25320247
Epidemiology. 2007 Nov;18(6):805-35
pubmed: 18049195
BMJ. 2015 Jan 07;350:g7594
pubmed: 25569120
Acad Emerg Med. 2013 Jan;20(1):17-26
pubmed: 23570474
J Clin Epidemiol. 1999 Oct;52(10):935-42
pubmed: 10513756
Stat Med. 2018 Aug 30;37(19):2902-2906
pubmed: 34349337
J Dig Dis. 2014 Apr;15(4):195-201
pubmed: 24373542
BMJ. 2009 Jun 04;338:b606
pubmed: 19502216
Stat Med. 2004 Mar 30;23(6):907-26
pubmed: 15027080
BMC Med. 2010 Mar 30;8:21
pubmed: 20353579
J Clin Epidemiol. 2008 Apr;61(4):331-43
pubmed: 18313557
JACC Cardiovasc Interv. 2013 Jun;6(6):614-22
pubmed: 23787234
J Card Fail. 2004 Dec;10(6):460-6
pubmed: 15599835
Am J Emerg Med. 2014 May;32(5):412-6
pubmed: 24560394
Heart Rhythm. 2012 Jan;9(1):42-6
pubmed: 21893137
PLoS Med. 2012;9(5):1-12
pubmed: 22629234
Diagn Progn Res. 2017;1:12
pubmed: 29350215
Am J Obstet Gynecol. 2016 Jan;214(1):79-90.e36
pubmed: 26070707
Stat Med. 2017 Sep 10;36(20):3257-3277
pubmed: 28543517
J Neurointerv Surg. 2014 Jun;6(5):342-8
pubmed: 23828326
Biometrics. 1988 Dec;44(4):1049-60
pubmed: 3233245
Cardiology. 2013;124(3):153-60
pubmed: 23485798
CNS Neurosci Ther. 2014 May;20(5):403-10
pubmed: 24612485
J Clin Epidemiol. 2015 Dec;68(12):1504-11
pubmed: 25985892
Br J Surg. 2013 Apr;100(5):645-53
pubmed: 23338659
Cardiol J. 2012;19(6):578-85
pubmed: 23224919
Circ Heart Fail. 2012 Mar 1;5(2):183-90
pubmed: 22361079
Stroke. 2014 Jan;45(1):92-100
pubmed: 24262323
Stat Methods Med Res. 1992;1(3):249-73
pubmed: 1341660
J Am Med Inform Assoc. 2018 Aug 1;25(8):924-930
pubmed: 29444283
Neurocrit Care. 2013 Feb;18(1):143-53
pubmed: 23138544
Chest. 2014 Mar 1;145(3):567-573
pubmed: 24091567
Stroke. 2013 Jun;44(6):1606-8
pubmed: 23549132
BMJ Open. 2017 Sep 27;7(9):e016591
pubmed: 28963291
J Clin Epidemiol. 2015 Mar;68(3):279-89
pubmed: 25179855
Stat Med. 2015 Jun 15;34(13):2081-103
pubmed: 25800943
Stat Med. 2007 Dec 30;26(30):5499-511
pubmed: 17907249
Stroke. 2009 Feb;40(2):350-4
pubmed: 19023098
J Am Coll Cardiol. 2014 Jul 1;63(25 Pt B):2935-2959
pubmed: 24239921
J Clin Epidemiol. 2014 May;67(5):538-46
pubmed: 24447592
Am J Med. 2012 Jul;125(7):695-703.e1
pubmed: 22727237
Circ Cardiovasc Qual Outcomes. 2015 Jul;8(4):368-75
pubmed: 26152680
J Clin Epidemiol. 2013 Mar;66(3):268-77
pubmed: 23116690
Stat Methods Med Res. 2018 Jun;27(6):1723-1736
pubmed: 27647815
Eur J Heart Fail. 2013 Aug;15(8):868-76
pubmed: 23512095
Am J Med. 2015 Jan;128(1):90.e9-15
pubmed: 25242230
Stat Med. 2016 Oct 15;35(23):4136-52
pubmed: 27251001
Chest. 2013 Jan;143(1):138-145
pubmed: 22814859
Plast Reconstr Surg. 2013 Jun;131(6):1213-1222
pubmed: 23714787
Int J Cardiol. 2013 Apr 15;164(3):318-22
pubmed: 21802750
BMJ. 2009 May 28;338:b605
pubmed: 19477892
Emerg Med Australas. 2014 Feb;26(1):34-44
pubmed: 24428678
BMC Med Res Methodol. 2015 Aug 05;15:59
pubmed: 26242875
Stroke. 2013 Sep;44(9):2441-5
pubmed: 23868275
J Heart Lung Transplant. 2012 May;31(5):467-77
pubmed: 22221678
Front Med. 2013 Dec;7(4):477-85
pubmed: 24048813
Am J Obstet Gynecol. 2016 Apr;214(4):424-437
pubmed: 26800772
BMC Infect Dis. 2014 Apr 29;14:230
pubmed: 24779617
Stat Methods Med Res. 2019 Apr;28(4):969-985
pubmed: 29157119
Diagn Progn Res. 2017 Dec 21;1:20
pubmed: 31093549
BMC Med. 2011 Sep 08;9:103
pubmed: 21902820
J Am Heart Assoc. 2013 May 24;2(3):e000174
pubmed: 23709565
JACC Cardiovasc Interv. 2012 Nov;5(11):1108-16
pubmed: 23174634
Eur Heart J. 2012 Dec;33(24):3098-104
pubmed: 23048195
JACC Cardiovasc Interv. 2013 Sep;6(9):897-904
pubmed: 24050858
BMC Med Res Methodol. 2013 Feb 15;13:19
pubmed: 23414436
Circulation. 2014 Sep 9;130(11 Suppl 1):S45-50
pubmed: 25200055
Am J Epidemiol. 2010 Oct 15;172(8):971-80
pubmed: 20807737
J Am Heart Assoc. 2013 Jan 29;2(1):e003269
pubmed: 23525425
Stroke. 2012 Dec;43(12):3218-24
pubmed: 23127975