Unveiling the sound of the cognitive status: Machine Learning-based speech analysis in the Alzheimer's disease spectrum.
Alzheimer’s disease
Automated pattern recognition
Early diagnosis
Machine Learning
Mild cognitive impairment
Neuropsychological tests
Speech acoustics
Journal
Alzheimer's research & therapy
ISSN: 1758-9193
Titre abrégé: Alzheimers Res Ther
Pays: England
ID NLM: 101511643
Informations de publication
Date de publication:
02 Feb 2024
02 Feb 2024
Historique:
received:
07
11
2023
accepted:
18
01
2024
medline:
3
2
2024
pubmed:
3
2
2024
entrez:
3
2
2024
Statut:
epublish
Résumé
Advancement in screening tools accessible to the general population for the early detection of Alzheimer's disease (AD) and prediction of its progression is essential for achieving timely therapeutic interventions and conducting decentralized clinical trials. This study delves into the application of Machine Learning (ML) techniques by leveraging paralinguistic features extracted directly from a brief spontaneous speech (SS) protocol. We aimed to explore the capability of ML techniques to discriminate between different degrees of cognitive impairment based on SS. Furthermore, for the first time, this study investigates the relationship between paralinguistic features from SS and cognitive function within the AD spectrum. Physical-acoustic features were extracted from voice recordings of patients evaluated in a memory unit who underwent a SS protocol. We implemented several ML models evaluated via cross-validation to identify individuals without cognitive impairment (subjective cognitive decline, SCD), with mild cognitive impairment (MCI), and with dementia due to AD (ADD). In addition, we established models capable of predicting cognitive domain performance based on a comprehensive neuropsychological battery from Fundació Ace (NBACE) using SS-derived information. The results of this study showed that, based on a paralinguistic analysis of sound, it is possible to identify individuals with ADD (F1 = 0.92) and MCI (F1 = 0.84). Furthermore, our models, based on physical acoustic information, exhibited correlations greater than 0.5 for predicting the cognitive domains of attention, memory, executive functions, language, and visuospatial ability. In this study, we show the potential of a brief and cost-effective SS protocol in distinguishing between different degrees of cognitive impairment and forecasting performance in cognitive domains commonly affected within the AD spectrum. Our results demonstrate a high correspondence with protocols traditionally used to assess cognitive function. Overall, it opens up novel prospects for developing screening tools and remote disease monitoring.
Sections du résumé
BACKGROUND
BACKGROUND
Advancement in screening tools accessible to the general population for the early detection of Alzheimer's disease (AD) and prediction of its progression is essential for achieving timely therapeutic interventions and conducting decentralized clinical trials. This study delves into the application of Machine Learning (ML) techniques by leveraging paralinguistic features extracted directly from a brief spontaneous speech (SS) protocol. We aimed to explore the capability of ML techniques to discriminate between different degrees of cognitive impairment based on SS. Furthermore, for the first time, this study investigates the relationship between paralinguistic features from SS and cognitive function within the AD spectrum.
METHODS
METHODS
Physical-acoustic features were extracted from voice recordings of patients evaluated in a memory unit who underwent a SS protocol. We implemented several ML models evaluated via cross-validation to identify individuals without cognitive impairment (subjective cognitive decline, SCD), with mild cognitive impairment (MCI), and with dementia due to AD (ADD). In addition, we established models capable of predicting cognitive domain performance based on a comprehensive neuropsychological battery from Fundació Ace (NBACE) using SS-derived information.
RESULTS
RESULTS
The results of this study showed that, based on a paralinguistic analysis of sound, it is possible to identify individuals with ADD (F1 = 0.92) and MCI (F1 = 0.84). Furthermore, our models, based on physical acoustic information, exhibited correlations greater than 0.5 for predicting the cognitive domains of attention, memory, executive functions, language, and visuospatial ability.
CONCLUSIONS
CONCLUSIONS
In this study, we show the potential of a brief and cost-effective SS protocol in distinguishing between different degrees of cognitive impairment and forecasting performance in cognitive domains commonly affected within the AD spectrum. Our results demonstrate a high correspondence with protocols traditionally used to assess cognitive function. Overall, it opens up novel prospects for developing screening tools and remote disease monitoring.
Identifiants
pubmed: 38308366
doi: 10.1186/s13195-024-01394-y
pii: 10.1186/s13195-024-01394-y
doi:
Types de publication
Journal Article
Langues
eng
Sous-ensembles de citation
IM
Pagination
26Subventions
Organisme : Next Generation EU
ID : MIA.2021.M02.0005
Organisme : Next Generation EU
ID : MIA.2021.M02.0005
Organisme : Next Generation EU
ID : MIA.2021.M02.0005
Organisme : Next Generation EU
ID : MIA.2021.M02.0005
Informations de copyright
© 2024. The Author(s).
Références
Mok VC, Pendlebury S, Wong A, Alladi S, Au L, Bath PM, et al. Tackling challenges in care of Alzheimer’s disease and other dementias amid the COVID-19 pandemic, now and in the future. Alzheimers Dement. 2020;16(11):1571–81.
pubmed: 32789951
pmcid: 7436526
doi: 10.1002/alz.12143
Alzheimer’s & Dementia. 2023 Alzheimer’s disease facts and figures. Alzheimers Dement. 2023;19(4):1598–695.
doi: 10.1002/alz.13016
Rafii MS, Aisen PS. Detection and treatment of Alzheimer’s disease in its preclinical stage. Nat Aging. 2023;3(5):520–31.
pubmed: 37202518
doi: 10.1038/s43587-023-00410-4
Scheltens P, De Strooper B, Kivipelto M, Holstege H, Chételat G, Teunissen CE, et al. Alzheimer’s disease. Lancet. 2021;397(10284):1577–90.
pubmed: 33667416
pmcid: 8354300
doi: 10.1016/S0140-6736(20)32205-4
DelEtoile J, Adeli H. Graph theory and brain connectivity in Alzheimer’s disease. Neuroscientist. 2017;23(6):616–26.
pubmed: 28406055
doi: 10.1177/1073858417702621
Sperling RA, Aisen PS, Beckett LA, Bennett DA, Craft S, Fagan AM, et al. Toward defining the preclinical stages of Alzheimer’s disease: recommendations from the National Institute on Aging-Alzheimer’s Association workgroups on diagnostic guidelines for Alzheimer’s disease. Alzheimers Dement. 2011;7(3):280–92.
pubmed: 21514248
pmcid: 3220946
doi: 10.1016/j.jalz.2011.03.003
Alegret M, Espinosa A, Vinyes-Junqué G, Valero S, Hernández I, Tárraga L, et al. Normative data of a brief neuropsychological battery for Spanish individuals older than 49. J Clin Exp Neuropsychol. 2012;34(2):209–19.
pubmed: 22149440
doi: 10.1080/13803395.2011.630652
Espinosa A, Alegret M, Valero S, Vinyes-Junqué G, Hernández I, Mauleón A, et al. A longitudinal follow-up of 550 mild cognitive impairment patients: evidence for large conversion to dementia rates and detection of major risk factors involved. J Alzheimers Dis. 2013;34(3):769–80.
pubmed: 23271318
doi: 10.3233/JAD-122002
Alegret M, García-Gutiérrez F, Muñoz N, Espinosa A, Ortega G, Lleonart N, et al. FACEmemory®, an innovative online platform for episodic memory pre-screening: findings from the first 3,000 participants. J Alzheimers Dis. 2024; Pre-press:1–15. https://doi.org/10.3233/JAD-230983 .
Martínez-Nicolás I, Llorente TE, Martínez-Sánchez F, Meilán JJG. Ten years of research on automatic voice and speech analysis of people with Alzheimer’s disease and mild cognitive impairment: a systematic review article. Front Psychol. 2021;12:620251.
pubmed: 33833713
pmcid: 8021952
doi: 10.3389/fpsyg.2021.620251
Sperling RA, Karlawish J, Johnson KA. Preclinical Alzheimer disease-the challenges ahead. Nat Rev Neurol. 2013;9(1):54–8.
pubmed: 23183885
doi: 10.1038/nrneurol.2012.241
Szatloczki G, Hoffmann I, Vincze V, Kalman J, Pakaski M. Speaking in Alzheimer’s disease, is that an early sign? Importance of changes in language abilities in Alzheimer’s disease. Front Aging Neurosci. 2015;7:195.
pubmed: 26539107
pmcid: 4611852
doi: 10.3389/fnagi.2015.00195
Qiu X, Sun T, Xu Y, Shao Y, Dai N, Huang X. Pre-trained models for natural language processing: a survey. Sci China Technol Sci. 2020;63(10):1872–97.
doi: 10.1007/s11431-020-1647-3
Sharma G, Umapathy K, Krishnan S. Trends in audio signal feature extraction methods. Appl Acoust. 2020;158:107020.
doi: 10.1016/j.apacoust.2019.107020
Eyben F, Scherer KR, Schuller BW, Sundberg J, André E, Busso C, et al. The Geneva minimalistic acoustic parameter set (GeMAPS) for voice research and affective computing. IEEE Trans Affect Comput. 2015;7(2):190–202.
doi: 10.1109/TAFFC.2015.2457417
Schuller B, Steidl S, Batliner A, Hirschberg J, Burgoon JK, Baird A, et al. The interspeech 2016 computational paralinguistics challenge: deception, sincerity & native language. In: 17TH Annual Conference of the International Speech Communication Association (Interspeech 2016), ISCA; 2016; Vols 1-5. vol. 8; p. 2001–5. https://doi.org/10.21437/Interspeech.2016-129 .
Lindsay H, Tröger J, König A. Language impairment in Alzheimer’s disease-robust and explainable evidence for ad-related deterioration of spontaneous speech through multilingual machine learning. Front Aging Neurosci. 2021;13:642033.
pubmed: 34093165
pmcid: 8170097
doi: 10.3389/fnagi.2021.642033
Xue C, Karjadi C, Paschalidis IC, Au R, Kolachalama VB. Detection of dementia on voice recordings using deep learning: a Framingham Heart Study. Alzheimers Res Ther. 2021;13:1–15.
doi: 10.1186/s13195-021-00888-3
Mahajan P, Baths V. Acoustic and language based deep learning approaches for Alzheimer’s dementia detection from spontaneous speech. Front Aging Neurosci. 2021;13:623607.
pubmed: 33613269
pmcid: 7893079
doi: 10.3389/fnagi.2021.623607
Chen J, Ye J, Tang F, Zhou J. Automatic detection of alzheimer’s disease using spontaneous speech only. Interspeech, 2021;3830–4. https://doi.org/10.21437/interspeech.2021-2002 .
Amini S, Hao B, Zhang L, Song M, Gupta A, Karjadi C, et al. Automated detection of mild cognitive impairment and dementia from voice recordings: a natural language processing approach. Alzheimers Dement. 2023;19(3):946–55.
doi: 10.1002/alz.12721
He R, Chapin K, Al-Tamimi J, Bel N, Marquié M, Rosende-Roca M, et al. Automated classification of cognitive decline and probable Alzheimer’s dementia across multiple speech and language domains. Am J Speech Lang Pathol. 2023;32(5):2075–86.
pubmed: 37486774
doi: 10.1044/2023_AJSLP-22-00403
Asgari M, Kaye J, Dodge H. Predicting mild cognitive impairment from spontaneous spoken utterances. Alzheimers Dement Transl Res Clin Interv. 2017;3(2):219–28.
doi: 10.1016/j.trci.2017.01.006
Themistocleous C, Eckerström M, Kokkinakis D. Identification of mild cognitive impairment from speech in Swedish using deep sequential neural networks. Front Neurol. 2018;9:975.
pubmed: 30498472
pmcid: 6250092
doi: 10.3389/fneur.2018.00975
Gosztolya G, Balogh R, Imre N, Egas-Lopez JV, Hoffmann I, Vincze V, et al. Cross-lingual detection of mild cognitive impairment based on temporal parameters of spontaneous speech. Comput Speech Lang. 2021;69:101215.
doi: 10.1016/j.csl.2021.101215
Hajjar I, Okafor M, Choi JD, Moore E, Abrol A, Calhoun VD, et al. Development of digital voice biomarkers and associations with cognition, cerebrospinal biomarkers, and neural representation in early Alzheimer’s disease. Alzheimers Dement Diagn Assess Dis Monit. 2023;15(1):e12393.
García-Gutiérrez F, Marquié M, Muñoz N, Alegret M, Cano A, De Rojas I, et al. Harnessing acoustic speech parameters to decipher amyloid status in individuals with mild cognitive impairment. Front Neurosci. 2023;17.
Yang Q, Li X, Ding X, Xu F, Ling Z. Deep learning-based speech analysis for Alzheimer’s disease detection: a literature review. Alzheimers Res Ther. 2022;14(1):1–16.
doi: 10.1186/s13195-022-01131-3
Liu Z, Proctor L, Collier PN, Zhao X. Automatic diagnosis and prediction of cognitive decline associated with Alzheimer’s dementia through spontaneous speech. In: 2021 ieee international conference on signal and image processing applications (icsipa). IEEE; 2021. p. 39–43. https://doi.org/10.1109/ICSIPA52582.2021.9576784 .
Haulcy R, Glass J. Classifying Alzheimer’s disease using audio and text-based representations of speech. Front Psychol. 2021;11:624137.
pubmed: 33519651
pmcid: 7845557
doi: 10.3389/fpsyg.2020.624137
Meghanani A, Anoop C, Ramakrishnan A. An exploration of log-mel spectrogram and MFCC features for Alzheimer’s dementia recognition from spontaneous speech. In: 2021 IEEE spoken language technology workshop (SLT). IEEE; 2021. p. 670–7. https://doi.org/10.1109/SLT48900.2021.9383491 .
Luz S, Haide F, Fuente S. d. l., Fromm D, MacWhinney B. Alzheimer's Dementia Recognition through Spontaneous Speech: The ADReSS Challenge. Proc Interspeech. 2020;2172–6. https://doi.org/10.21437/Interspeech.2021-2002 .
Abdelnour C, Rodríguez-Gómez O, Alegret M, Valero S, Moreno-Grau S, Sanabria Á, et al. Impact of recruitment methods in subjective cognitive decline. J Alzheimers Dis. 2017;57(2):625–32.
pubmed: 28269773
doi: 10.3233/JAD-160915
Folstein MF, Robins LN, Helzer JE. The mini-mental state examination. Arch Gen Psychiatr. 1983;40(7):812.
pubmed: 6860082
doi: 10.1001/archpsyc.1983.01790060110016
Del Ser T, Sánchez-Sánchez F, de Yébenes MJG, Otero A, Munoz DG. Validation of the seven-minute screen neurocognitive battery for the diagnosis of dementia in a Spanish population-based sample. Dement Geriatr Cogn Disord. 2006;22(5–6):454–64.
pubmed: 16988506
Boada M, Tárraga L, Modinos G, López O, Cummings J. Neuropsychiatric inventory-nursing home version (NPI-NH): Spanish validation. Neurologia (Barcelona, Spain). 2005;20(10):665–73.
pubmed: 16317587
Hachinski VC, Lassen NA, Marshall J. Multi-infarct dementia: a cause of mental deterioration in the elderly. Lancet. 1974;304(7874):207–9.
doi: 10.1016/S0140-6736(74)91496-2
Blessed G, Tomlinson BE, Roth M. The association between quantitative measures of dementia and of senile change in the cerebral grey matter of elderly subjects. Brit J Psychiatry. 1968;114(512):797–811.
doi: 10.1192/bjp.114.512.797
Morris JC. The Clinical Dementia Rating (CDR): current version and scoring rules. Neurology. 1993;43(11):2412–2412.
Boada M, Tárraga L, Hernández I, Valero S, Alegret M, Ruiz A, et al. Design of a comprehensive Alzheimer’s disease clinic and research center in Spain to meet critical patient and family needs. Alzheimers Dement. 2014;10(3):409–15.
pubmed: 24035148
doi: 10.1016/j.jalz.2013.03.006
Jessen F, Amariglio RE, Van Boxtel M, Breteler M, Ceccaldi M, Chételat G, et al. A conceptual framework for research on subjective cognitive decline in preclinical Alzheimer’s disease. Alzheimers Dement. 2014;10(6):844–52.
pubmed: 24798886
pmcid: 4317324
doi: 10.1016/j.jalz.2014.01.001
Petersen RC. Mild cognitive impairment as a diagnostic entity. J Intern Med. 2004;256(3):183–94.
pubmed: 15324362
doi: 10.1111/j.1365-2796.2004.01388.x
McKhann GM, Knopman DS, Chertkow H, Hyman BT, Jack CR Jr, Kawas CH, et al. The diagnosis of dementia due to Alzheimer’s disease: recommendations from the National Institute on Aging-Alzheimer’s Association workgroups on diagnostic guidelines for Alzheimer’s disease. Alzheimers Dement. 2011;7(3):263–9.
pubmed: 21514250
pmcid: 3312024
doi: 10.1016/j.jalz.2011.03.005
Cummings L. Describing the cookie theft picture: sources of breakdown in Alzheimer’s dementia. Pragmat Soc. 2019;10(2):153–76.
doi: 10.1075/ps.17011.cum
Défossez A, Synnaeve G, Adi Y. Real Time Speech Enhancement in the Waveform Domain. Proc Interspeech. 2020;3291–5. https://doi.org/10.21437/Interspeech.2020-2409 .
Eyben F, Wöllmer M, Schuller B. Opensmile: the Munich versatile and fast open-source audio feature extractor. In: Proceedings of the 18th ACM international conference on Multimedia. Assoc Comput Machinery. 2010;4:1459–62. https://doi.org/10.1145/1873951.1874246 .
Hair JF, Hult GTM, Ringle CM, Sarstedt M, Danks NP, Ray S. An Introduction to Structural Equation Modeling. In: Partial Least Squares Structural Equation Modeling (PLS-SEM) Using R. Cham: Springer; 2021; p. 1–29.
Dowling NM, Hermann B, La Rue A, Sager MA. Latent structure and factorial invariance of a neuropsychological test battery for the study of preclinical Alzheimer’s disease. Neuropsychology. 2010;24(6):742.
pubmed: 21038965
pmcid: 3057903
doi: 10.1037/a0020176
Park LQ, Gross AL, McLaren DG, Pa J, Johnson JK, Mitchell M, et al. Confirmatory factor analysis of the ADNI neuropsychological battery. Brain Imaging Behav. 2012;6:528–39.
pubmed: 22777078
pmcid: 3538867
doi: 10.1007/s11682-012-9190-3
Rosseel Y. lavaan: an R package for structural equation modeling. J Stat Softw. 2012;48:1–36.
doi: 10.18637/jss.v048.i02
Wechsler D. WMS-III: Wechsler memory scale administration and scoring manual. Psychological Corp; 1997.
Wechsler D. Technical manual. Giunti, OS Organizzazioni Speciali; 2002.
Pillon B, Dubois B, Bonnet A, Esteguy M, Guimaraes J, Vigouret J, et al. Cognitive slowing in Parkinson’s disease fails to respond to levodopa treatment: the 15-objects test. Neurology. 1989;39(6):762.
pubmed: 2725868
doi: 10.1212/WNL.39.6.762
Sala SD, Laiacona M, Trivelli C, Spinnler H. Poppelreuter-Ghent’s overlapping figures test: its sensitivity to age, and its clinical use. Arch Clin Neuropsychol. 1995;10(6):511–34.
pubmed: 14588906
doi: 10.1093/arclin/10.6.511
Golden CJ. In reply to Adams’s “In search of Luria’s battery: A false start.”. J Consult Clin Psychol. 1980;48(4):511–6.
Artiola L, Hermosillo D, Heaton R, Pardee R. Manual de normas y procedimientos para la batería neuropsicológica en español. Tucson: mPress. 1999.
Goodglass H, Kaplan E. The assessment of aphasia and related disorders. Lea & Febiger; 1972.
Erzigkeit H. The SKT-a short cognitive performance test as an instrument for the assessment of clinical efficacy of cognition enhancers. In: Diagnosis and treatment of senile dementia. Springer; 1989. p. 164–174.
Kaplan E, Goodglass H, Weintraub S, et al. Boston naming test. Encycl Clin Neuropsychol. 2001. https://doi.org/10.1007/978-0-387-79948-3_869 .
Zebari R, Abdulazeez A, Zeebaree D, Zebari D, Saeed J. A comprehensive review of dimensionality reduction techniques for feature selection and feature extraction. J Appl Sci Technol Trends. 2020;1(2):56–70.
doi: 10.38094/jastt1224
Xue B, Zhang M, Browne WN, Yao X. A survey on evolutionary computation approaches to feature selection. IEEE Trans Evol Comput. 2015;20(4):606–26.
doi: 10.1109/TEVC.2015.2504420
Akiba T, Sano S, Yanase T, Ohta T, Koyama M. Optuna: a next-generation hyperparameter optimization framework. In: Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining. Assoc Comput Machinery. 2019. p. 2623–31. https://doi.org/10.1145/3292500.3330701 .
Bergstra J, Bardenet R, Bengio Y, Kégl B. Algorithms for hyper-parameter optimization. Adv Neural Inf Process Syst. 2011;24:2546–54.
Fernández A, Garcia S, Herrera F, Chawla NV. SMOTE for learning from imbalanced data: progress and challenges, marking the 15-year anniversary. J Artif Intell Res. 2018;61:863–905.
doi: 10.1613/jair.1.11192
Buitinck L, Louppe G, Blondel M, Pedregosa F, Mueller A, Grisel O, et al. API design for machine learning software: experiences from the scikit-learn project. 2013. arXiv preprint arXiv:13090238.
Chen T, Guestrin C. Xgboost: A scalable tree boosting system. In: Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining. Assoc Comput Machinery. 2016. p. 785–94.
Bucks RS, Singh S, Cuerden JM, Wilcock GK. Analysis of spontaneous, conversational speech in dementia of Alzheimer type: evaluation of an objective technique for analysing lexical performance. Aphasiology. 2000;14(1):71–91.
doi: 10.1080/026870300401603
Jarrold W, Peintner B, Wilkins D, Vergryi D, Richey C, Gorno-Tempini ML, et al. Aided diagnosis of dementia type through computer-based analysis of spontaneous speech. In: Proceedings of the Workshop on Computational Linguistics and Clinical Psychology: From Linguistic Signal to Clinical Reality. Assoc Comput Linguist. 2014. p. 27–37. https://doi.org/10.3115/v1/W14-3204 .
Meilán JJG, Martínez-Sánchez F, Carro J, López DE, Millian-Morell L, Arana JM. Speech in Alzheimer’s disease: can temporal and acoustic parameters discriminate dementia? Dement Geriatr Cogn Disord. 2014;37(5–6):327–34.
pubmed: 24481220
doi: 10.1159/000356726
Fraser KC, Meltzer JA, Rudzicz F. Linguistic features identify Alzheimer’s disease in narrative speech. J Alzheimers Dis. 2016;49(2):407–22.
pubmed: 26484921
doi: 10.3233/JAD-150520
Liu L, Zhao S, Chen H, Wang A. A new machine learning method for identifying Alzheimer’s disease. Simul Model Pract Theory. 2020;99:102023.
doi: 10.1016/j.simpat.2019.102023
Shwartz-Ziv R, Armon A. Tabular data: deep learning is not all you need. Inf Fusion. 2022;81:84–90.
doi: 10.1016/j.inffus.2021.11.011
Pulido MLB, Hernández JBA, Ballester MÁF, González CMT, Mekyska J, Smékal Z. Alzheimer’s disease and automatic speech analysis: a review. Expert Syst Appl. 2020;150:113213.
doi: 10.1016/j.eswa.2020.113213