Diagnostic Performance of Machine Learning-Derived OSA Prediction Tools in Large Clinical and Community-Based Samples.
OSA
artificial neural network
electronic medical record
kernel support vector machine
machine learning
prediction model
random forest
Journal
Chest
ISSN: 1931-3543
Titre abrégé: Chest
Pays: United States
ID NLM: 0231335
Informations de publication
Date de publication:
03 2022
03 2022
Historique:
received:
15
04
2021
revised:
14
09
2021
accepted:
10
10
2021
pubmed:
1
11
2021
medline:
8
4
2022
entrez:
31
10
2021
Statut:
ppublish
Résumé
Prediction tools without patient-reported symptoms could facilitate widespread identification of OSA. What is the diagnostic performance of OSA prediction tools derived from machine learning using readily available data without patient responses to questionnaires? Also, how do they compare with STOP-BANG, an OSA prediction tool, in clinical and community-based samples? Logistic regression and machine learning techniques, including artificial neural network (ANN), random forests (RF), and kernel support vector machine, were used to determine the ability of age, sex, BMI, and race to predict OSA status. A retrospective cohort of 17,448 subjects from sleep clinics within the international Sleep Apnea Global Interdisciplinary Consortium (SAGIC) were randomly split into training (n = 10,469) and validation (n = 6,979) sets. Model comparisons were performed by using the area under the receiver-operating curve (AUC). Trained models were compared with the STOP-BANG questionnaire in two prospective testing datasets: an independent clinic-based sample from SAGIC (n = 1,613) and a community-based sample from the Sleep Heart Health Study (n = 5,599). The AUCs (95% CI) of the machine learning models were significantly higher than logistic regression (0.61 [0.60-0.62]) in both the training and validation datasets (ANN, 0.68 [0.66-0.69]; RF, 0.68 [0.67-0.70]; and kernel support vector machine, 0.66 [0.65-0.67]). In the SAGIC testing sample, the ANN (0.70 [0.68-0.72]) and RF (0.70 [0.68-0.73]) models had AUCs similar to those of the STOP-BANG (0.71 [0.68-0.72]). In the Sleep Heart Health Study testing sample, the ANN (0.72 [0.71-0.74]) had AUCs similar to those of STOP-BANG (0.72 [0.70-0.73]). OSA prediction tools using machine learning without patient-reported symptoms provide better diagnostic performance than logistic regression. In clinical and community-based samples, the symptomless ANN tool has diagnostic performance similar to that of a widely used prediction tool that includes patient symptoms. Machine learning-derived algorithms may have utility for widespread identification of OSA.
Sections du résumé
BACKGROUND
Prediction tools without patient-reported symptoms could facilitate widespread identification of OSA.
RESEARCH QUESTION
What is the diagnostic performance of OSA prediction tools derived from machine learning using readily available data without patient responses to questionnaires? Also, how do they compare with STOP-BANG, an OSA prediction tool, in clinical and community-based samples?
STUDY DESIGN AND METHODS
Logistic regression and machine learning techniques, including artificial neural network (ANN), random forests (RF), and kernel support vector machine, were used to determine the ability of age, sex, BMI, and race to predict OSA status. A retrospective cohort of 17,448 subjects from sleep clinics within the international Sleep Apnea Global Interdisciplinary Consortium (SAGIC) were randomly split into training (n = 10,469) and validation (n = 6,979) sets. Model comparisons were performed by using the area under the receiver-operating curve (AUC). Trained models were compared with the STOP-BANG questionnaire in two prospective testing datasets: an independent clinic-based sample from SAGIC (n = 1,613) and a community-based sample from the Sleep Heart Health Study (n = 5,599).
RESULTS
The AUCs (95% CI) of the machine learning models were significantly higher than logistic regression (0.61 [0.60-0.62]) in both the training and validation datasets (ANN, 0.68 [0.66-0.69]; RF, 0.68 [0.67-0.70]; and kernel support vector machine, 0.66 [0.65-0.67]). In the SAGIC testing sample, the ANN (0.70 [0.68-0.72]) and RF (0.70 [0.68-0.73]) models had AUCs similar to those of the STOP-BANG (0.71 [0.68-0.72]). In the Sleep Heart Health Study testing sample, the ANN (0.72 [0.71-0.74]) had AUCs similar to those of STOP-BANG (0.72 [0.70-0.73]).
INTERPRETATION
OSA prediction tools using machine learning without patient-reported symptoms provide better diagnostic performance than logistic regression. In clinical and community-based samples, the symptomless ANN tool has diagnostic performance similar to that of a widely used prediction tool that includes patient symptoms. Machine learning-derived algorithms may have utility for widespread identification of OSA.
Identifiants
pubmed: 34717928
pii: S0012-3692(21)04248-3
doi: 10.1016/j.chest.2021.10.023
pmc: PMC8941600
pii:
doi:
Types de publication
Journal Article
Research Support, Non-U.S. Gov't
Research Support, N.I.H., Extramural
Langues
eng
Sous-ensembles de citation
IM
Pagination
807-817Subventions
Organisme : NCATS NIH HHS
ID : UL1 TR001070
Pays : United States
Organisme : NHLBI NIH HHS
ID : U01 HL053916
Pays : United States
Organisme : NHLBI NIH HHS
ID : P01 HL094307
Pays : United States
Organisme : NHLBI NIH HHS
ID : U01 HL053938
Pays : United States
Organisme : NHLBI NIH HHS
ID : U01 HL064360
Pays : United States
Organisme : NHLBI NIH HHS
ID : U01 HL053941
Pays : United States
Organisme : NHLBI NIH HHS
ID : U01 HL053934
Pays : United States
Organisme : NHLBI NIH HHS
ID : R24 HL114473
Pays : United States
Organisme : NHLBI NIH HHS
ID : U01 HL053937
Pays : United States
Organisme : NHLBI NIH HHS
ID : U01 HL053931
Pays : United States
Informations de copyright
Copyright © 2021 American College of Chest Physicians. Published by Elsevier Inc. All rights reserved.
Références
Chest. 2020 Sep;158(3):1187-1197
pubmed: 32304773
IEEE Trans Neural Netw. 1992;3(4):602-11
pubmed: 18276460
Sleep. 1997 Dec;20(12):1077-85
pubmed: 9493915
Clin Trials. 2006;3(3):272-9
pubmed: 16895044
Ups J Med Sci. 1987;92(2):193-203
pubmed: 3660504
Lancet Respir Med. 2019 Aug;7(8):687-698
pubmed: 31300334
BMC Bioinformatics. 2011 Mar 17;12:77
pubmed: 21414208
J Clin Epidemiol. 1988;41(6):571-6
pubmed: 3385458
Bioinformatics. 2011 Dec 15;27(24):3439-40
pubmed: 21998157
N Engl J Med. 2017 Jun 29;376(26):2507-2509
pubmed: 28657867
J Clin Sleep Med. 2016 Jan;12(1):71-7
pubmed: 26350603
BMJ. 2016 Jan 25;352:i6
pubmed: 26810254
Am J Respir Crit Care Med. 2001 Jan;163(1):19-25
pubmed: 11208620
J Clin Sleep Med. 2017 May 15;13(5):665-666
pubmed: 28416048
Sleep. 2020 May 12;43(5):
pubmed: 31735957
Respirology. 2020 Jul;25(7):690-702
pubmed: 32436658
Stat Med. 2007 Sep 30;26(22):4179-201
pubmed: 17357992
J Clin Sleep Med. 2019 Apr 15;15(4):629-639
pubmed: 30952214
Sleep. 1998 Nov 1;21(7):759-67
pubmed: 11300121
Br J Surg. 2015 Feb;102(3):148-58
pubmed: 25627261
Sleep. 2013 Apr 01;36(4):591-6
pubmed: 23565005
Circulation. 2015 Nov 17;132(20):1920-30
pubmed: 26572668
J Clin Sleep Med. 2011 Oct 15;7(5):467-72
pubmed: 22003341
Am J Respir Crit Care Med. 1994 Nov;150(5 Pt 1):1279-85
pubmed: 7952553
N Engl J Med. 2016 Sep 29;375(13):1216-9
pubmed: 27682033
J Clin Epidemiol. 1996 Nov;49(11):1225-31
pubmed: 8892489
Sleep. 2018 Mar 1;41(3):
pubmed: 29315434
Sleep. 1995 Apr;18(3):158-66
pubmed: 7610311
Pacing Clin Electrophysiol. 2012 Oct;35(10):1199-208
pubmed: 22827606
J Clin Sleep Med. 2019 Jan 15;15(1):23-32
pubmed: 30621825
Br J Anaesth. 2012 May;108(5):768-75
pubmed: 22401881
Anesthesiology. 2008 May;108(5):812-21
pubmed: 18431116
Clin Exp Otorhinolaryngol. 2016 Mar;9(1):1-7
pubmed: 26976019
BMC Med. 2019 Dec 16;17(1):230
pubmed: 31842878
Ann Am Thorac Soc. 2014 Sep;11(7):1064-74
pubmed: 25068704
Clin Rev Allergy Immunol. 2013 Aug;45(1):109-16
pubmed: 23345025
JAMA. 2017 Jan 24;317(4):407-414
pubmed: 28118461
J Occup Environ Med. 2013 Sep;55(9):1035-40
pubmed: 23969501
Accid Anal Prev. 2008 Jan;40(1):104-15
pubmed: 18215538
Sleep. 2017 Mar 01;40(3):
pubmed: 28364424