Comparing the performance of machine learning and conventional models for predicting atherosclerotic cardiovascular disease in a general Chinese population.


Journal

BMC medical informatics and decision making
ISSN: 1472-6947
Titre abrégé: BMC Med Inform Decis Mak
Pays: England
ID NLM: 101088682

Informations de publication

Date de publication:
24 07 2023
Historique:
received: 20 01 2023
accepted: 13 07 2023
medline: 27 7 2023
pubmed: 25 7 2023
entrez: 24 7 2023
Statut: epublish

Résumé

Accurately predicting the risk of atherosclerotic cardiovascular disease (ASCVD) is crucial for implementing individualized prevention strategies and improving patient outcomes. Our objective is to develop machine learning (ML)-based models for predicting ASCVD risk in a prospective Chinese population and compare their performance with conventional regression models. A hybrid dataset consisting of 551 features was used, including 98 demographic, behavioral, and psychological features, 444 Electrocardiograph (ECG) features, and 9 Echocardiography (Echo) features. Seven machine learning (ML)-based models were trained, validated, and tested after selecting the 30 most informative features. We compared the discrimination, calibration, net benefit, and net reclassification improvement (NRI) of the ML models with those of conventional ASCVD risk calculators, such as the Pooled Cohort Equations (PCE) and Prediction for ASCVD Risk in China (China-PAR). The study included 9,609 participants (mean age 53.4 ± 10.4 years, 53.7% female), and during a median follow-up of 4.7 years, 431 (4.5%) participants developed ASCVD. In the testing set, the final ML-based ANN model outperformed PCE, China-PAR, recalibrated PCE, and recalibrated China-PAR in predicting ASCVD. This was demonstrated by the model's higher area under the curve (AUC) of 0.800, compared to 0.777, 0.780, 0.779, and 0.779 for the other models, respectively. Additionally, the model had a lower Hosmer-Lemeshow χ2 of 9.1, compared to 37.3, 67.6, 126.6, and 18.6 for the other models. The net benefit at a threshold of 5% was also higher for the ML-based ANN model at 0.017, compared to 0.016, 0.013, 0.017, and 0.016 for the other models, respectively. Furthermore, the NRI was 0.089 for the ML-based ANN model, while it was 0.355, 0.098, and 0.088 for PCE, China-PAR, and recalibrated PCE, respectively. Compared to conventional regression ASCVD risk calculators, such as PCE and China-PAR, the ANN prediction model may help optimize identification of individuals at heightened cardiovascular risk by flexibly incorporating a wider range of potential predictors. The findings may help guide clinical decision-making and ultimately contribute to ASCVD prevention and management.

Sections du résumé

BACKGROUND
Accurately predicting the risk of atherosclerotic cardiovascular disease (ASCVD) is crucial for implementing individualized prevention strategies and improving patient outcomes. Our objective is to develop machine learning (ML)-based models for predicting ASCVD risk in a prospective Chinese population and compare their performance with conventional regression models.
METHODS
A hybrid dataset consisting of 551 features was used, including 98 demographic, behavioral, and psychological features, 444 Electrocardiograph (ECG) features, and 9 Echocardiography (Echo) features. Seven machine learning (ML)-based models were trained, validated, and tested after selecting the 30 most informative features. We compared the discrimination, calibration, net benefit, and net reclassification improvement (NRI) of the ML models with those of conventional ASCVD risk calculators, such as the Pooled Cohort Equations (PCE) and Prediction for ASCVD Risk in China (China-PAR).
RESULTS
The study included 9,609 participants (mean age 53.4 ± 10.4 years, 53.7% female), and during a median follow-up of 4.7 years, 431 (4.5%) participants developed ASCVD. In the testing set, the final ML-based ANN model outperformed PCE, China-PAR, recalibrated PCE, and recalibrated China-PAR in predicting ASCVD. This was demonstrated by the model's higher area under the curve (AUC) of 0.800, compared to 0.777, 0.780, 0.779, and 0.779 for the other models, respectively. Additionally, the model had a lower Hosmer-Lemeshow χ2 of 9.1, compared to 37.3, 67.6, 126.6, and 18.6 for the other models. The net benefit at a threshold of 5% was also higher for the ML-based ANN model at 0.017, compared to 0.016, 0.013, 0.017, and 0.016 for the other models, respectively. Furthermore, the NRI was 0.089 for the ML-based ANN model, while it was 0.355, 0.098, and 0.088 for PCE, China-PAR, and recalibrated PCE, respectively.
CONCLUSIONS
Compared to conventional regression ASCVD risk calculators, such as PCE and China-PAR, the ANN prediction model may help optimize identification of individuals at heightened cardiovascular risk by flexibly incorporating a wider range of potential predictors. The findings may help guide clinical decision-making and ultimately contribute to ASCVD prevention and management.

Identifiants

pubmed: 37488520
doi: 10.1186/s12911-023-02242-z
pii: 10.1186/s12911-023-02242-z
pmc: PMC10367272
doi:

Types de publication

Comparative Study Journal Article Research Support, Non-U.S. Gov't

Langues

eng

Sous-ensembles de citation

IM

Pagination

134

Informations de copyright

© 2023. The Author(s).

Références

J Am Heart Assoc. 2018 Apr 12;7(8):
pubmed: 29650709
JACC Asia. 2022 Jan 04;2(1):33-43
pubmed: 36340248
Circulation. 2016 Nov 8;134(19):1430-1440
pubmed: 27682885
J Am Heart Assoc. 2014 Nov 21;3(6):e001387
pubmed: 25416036
Am J Epidemiol. 1982 Jan;115(1):92-106
pubmed: 7055134
Circ Cardiovasc Qual Outcomes. 2019 Sep;12(9):e005289
pubmed: 31525078
Biom J. 2008 Aug;50(4):457-79
pubmed: 18663757
Eur Heart J. 2017 Feb 14;38(7):500-507
pubmed: 27252451
JACC Cardiovasc Imaging. 2021 Mar;14(3):615-625
pubmed: 33129741
Nat Rev Cardiol. 2019 Apr;16(4):203-212
pubmed: 30467329
JAMA. 2007 Mar 7;297(9):978-85
pubmed: 17341712
Lancet Public Health. 2020 Dec;5(12):e672-e681
pubmed: 33271080
Diabetologia. 2019 Dec;62(12):2354-2364
pubmed: 31664481
Lancet. 2018 Nov 10;392(10159):1736-1788
pubmed: 30496103
Heart. 2018 Jul;104(14):1156-1164
pubmed: 29352006
Circulation. 2019 Jun 18;139(25):e1082-e1143
pubmed: 30586774
Zhonghua Yu Fang Yi Xue Za Zhi. 2019 Jan 6;53(1):13-35
pubmed: 30605960
Am Heart J. 2019 May;211:34-44
pubmed: 30831332
Circulation. 2018 Jul 3;138(1):e1-e34
pubmed: 29794080
J Am Coll Cardiol. 2017 May 30;69(21):2657-2664
pubmed: 28545640
Atherosclerosis. 2015 Sep;242(1):367-75
pubmed: 26255683
Lancet. 2021 Jan 16;397(10270):199-207
pubmed: 33453782
BMC Genomics. 2020 Jan 2;21(1):6
pubmed: 31898477
Sci Rep. 2021 Apr 26;11(1):8886
pubmed: 33903629
Med Decis Making. 2006 Nov-Dec;26(6):565-74
pubmed: 17099194
Comput Biol Med. 2021 Dec;139:104983
pubmed: 34749095
Cell. 2020 Apr 2;181(1):92-101
pubmed: 32243801
JAMA Cardiol. 2016 Oct 1;1(7):779-786
pubmed: 27487404
Circulation. 2020 Feb 18;141(7):530-539
pubmed: 32065775
Eur Heart J. 2019 Jun 21;40(24):1975-1986
pubmed: 30060039
Eur Heart J. 2014 Feb;35(5):290-8
pubmed: 23839541
JACC Cardiovasc Imaging. 2019 Apr;12(4):681-689
pubmed: 29909114
Lancet. 2016 Jan 16;387(10015):251-72
pubmed: 26510778
Eur Heart J. 2018 Sep 1;39(33):3021-3104
pubmed: 30165516
NPJ Digit Med. 2020 Sep 23;3:125
pubmed: 33043149
Circ Cardiovasc Qual Outcomes. 2016 Nov;9(6):629-640
pubmed: 28263938
J Am Coll Cardiol. 2014 Sep 2;64(9):898-906
pubmed: 25169175
JAMA Cardiol. 2019 Sep 1;4(9):918-927
pubmed: 31411652
Hypertension. 2018 Jun;71(6):1269-1324
pubmed: 29133354
Circulation. 2021 Aug 10;144(6):410-422
pubmed: 34247495
Front Cardiovasc Med. 2022 Oct 05;9:976769
pubmed: 36277775
Biometrics. 1988 Sep;44(3):837-45
pubmed: 3203132

Auteurs

Zihao Fan (Z)

Department of Cardiology, The First Hospital of China Medical University, No. 155, Nanjing Bei Street, Shenyang, 110001, China.
Guangdong Provincial People's Hospital (Guangdong Academy of Medical Sciences), Southern Medical University, 106 Zhongshan 2nd Road, Guangzhou, 510080, China.

Zhi Du (Z)

Department of Cardiology, The First Affiliated Hospital, Zhejiang University School of Medicine, Hangzhou, China.

Jinrong Fu (J)

Department of Endocrinology and Metabolism, The First Hospital of China Medical University, No. 155, Nanjing Bei Street, Shenyang, 110001, China.

Ying Zhou (Y)

Department of Cardiology, The First Hospital of China Medical University, No. 155, Nanjing Bei Street, Shenyang, 110001, China.

Pengyu Zhang (P)

Department of Cardiology, The First Hospital of China Medical University, No. 155, Nanjing Bei Street, Shenyang, 110001, China.

Chuning Shi (C)

Department of Cardiology, The First Hospital of China Medical University, No. 155, Nanjing Bei Street, Shenyang, 110001, China.

Yingxian Sun (Y)

Department of Cardiology, The First Hospital of China Medical University, No. 155, Nanjing Bei Street, Shenyang, 110001, China. yxsun@cmu.edu.cn.

Articles similaires

[Redispensing of expensive oral anticancer medicines: a practical application].

Lisanne N van Merendonk, Kübra Akgöl, Bastiaan Nuijen
1.00
Humans Antineoplastic Agents Administration, Oral Drug Costs Counterfeit Drugs

Smoking Cessation and Incident Cardiovascular Disease.

Jun Hwan Cho, Seung Yong Shin, Hoseob Kim et al.
1.00
Humans Male Smoking Cessation Cardiovascular Diseases Female
Humans United States Aged Cross-Sectional Studies Medicare Part C
1.00
Humans Yoga Low Back Pain Female Male

Classifications MeSH