Development and validation of an electronic Symbol-Digit Modalities Test for remote monitoring of people with multiple sclerosis.
cognition
digital health
multiple sclerosis
telemedicine
Journal
European journal of neurology
ISSN: 1468-1331
Titre abrégé: Eur J Neurol
Pays: England
ID NLM: 9506311
Informations de publication
Date de publication:
04 Sep 2024
04 Sep 2024
Historique:
revised:
25
07
2024
received:
29
05
2024
accepted:
10
08
2024
medline:
5
9
2024
pubmed:
5
9
2024
entrez:
5
9
2024
Statut:
aheadofprint
Résumé
Computerized cognitive tests may extend the reach of cognitive screening and monitoring to those with mobility issues or living in remote areas. Moreover, it could enable frequent and autonomous remote cognitive assessments in people with multiple sclerosis (pwMS) on account of its reduced economic and organizational costs. This may further improve our understanding of longitudinal trends and significantly improve the standard of care for pwMS living in remote areas or with mobility limitations. We aimed to evaluate the psychometric properties of an electronic Symbol-Digit Modalities Test (eSDMT) designed to allow pwMS to perform a rapid cognitive assessment independently from home using their own PC/laptop. Sixty-two participants underwent a neuropsychological evaluation, and then performed the eSDMT in the clinic. Forty-two participants also repeated the eSDMT at home. We assessed concurrent validity (eSDMT vs. oral SDMT), test-retest reliability (in the clinic vs. at home), discriminant validity (pwMS with/without cognitive impairment), and other psychometric characteristics of the eSDMT (effect of age, sex, and education on test scores). We observed good-to-excellent concurrent validity (r ≥ 0.84, all p < 0.0001) and test-retest reliability (intraclass correlation coefficients [ICCs]>0.87, all p < 0.0001). Discriminant validity was excellent (area under the curves [AUCs] >0.84, all p < 0.0001). eSDMT scores were only slightly influenced by demographic characteristics (all R We provided evidence which supports the use of our eSDMT as a feasible, valid, and reliable remote assessment of cognitive function in pwMS. Future studies will investigate long-term reliability and predictive power.
Sections du résumé
BACKGROUND
BACKGROUND
Computerized cognitive tests may extend the reach of cognitive screening and monitoring to those with mobility issues or living in remote areas. Moreover, it could enable frequent and autonomous remote cognitive assessments in people with multiple sclerosis (pwMS) on account of its reduced economic and organizational costs. This may further improve our understanding of longitudinal trends and significantly improve the standard of care for pwMS living in remote areas or with mobility limitations. We aimed to evaluate the psychometric properties of an electronic Symbol-Digit Modalities Test (eSDMT) designed to allow pwMS to perform a rapid cognitive assessment independently from home using their own PC/laptop.
METHODS
METHODS
Sixty-two participants underwent a neuropsychological evaluation, and then performed the eSDMT in the clinic. Forty-two participants also repeated the eSDMT at home. We assessed concurrent validity (eSDMT vs. oral SDMT), test-retest reliability (in the clinic vs. at home), discriminant validity (pwMS with/without cognitive impairment), and other psychometric characteristics of the eSDMT (effect of age, sex, and education on test scores).
RESULTS
RESULTS
We observed good-to-excellent concurrent validity (r ≥ 0.84, all p < 0.0001) and test-retest reliability (intraclass correlation coefficients [ICCs]>0.87, all p < 0.0001). Discriminant validity was excellent (area under the curves [AUCs] >0.84, all p < 0.0001). eSDMT scores were only slightly influenced by demographic characteristics (all R
CONCLUSIONS
CONCLUSIONS
We provided evidence which supports the use of our eSDMT as a feasible, valid, and reliable remote assessment of cognitive function in pwMS. Future studies will investigate long-term reliability and predictive power.
Types de publication
Journal Article
Langues
eng
Sous-ensembles de citation
IM
Pagination
e16454Informations de copyright
© 2024 The Author(s). European Journal of Neurology published by John Wiley & Sons Ltd on behalf of European Academy of Neurology.
Références
Mitchell AJ, Benito‐León J, González J‐MM, Rivera‐Navarro J. Quality of life and its assessment in multiple sclerosis: integrating physical and psychological components of wellbeing. Lancet Neurol. 2005;4(9):556‐566. doi:10.1016/S1474-4422(05)70166-6
Ozakbas S, Cagiran I, Ormeci B, Idiman E. Correlations between multiple sclerosis functional composite, expanded disability status scale and health‐related quality of life during and after treatment of relapses in patients with multiple sclerosis. J Neurol Sci. 2004;218(1–2):3‐7. doi:10.1016/j.jns.2003.09.015
Labiano‐Fontcuberta A, Martínez‐Ginés ML, Aladro Y, et al. A comparison study of cognitive deficits in radiologically and clinically isolated syndromes. Mult Scler. 2016;22(2):250‐253. doi:10.1177/1352458515591072
Sumowski JF, Benedict R, Enzinger C, et al. Cognition in multiple sclerosis. Neurology. 2018;90(6):278‐288. doi:10.1212/WNL.0000000000004977
Kalb R, Beier M, Benedict RHB, et al. Recommendations for cognitive screening and management in multiple sclerosis care. Mult Scler J. 2018;24(13):1665‐1680. doi:10.1177/1352458518803785
Benedict RHB, Amato MP, DeLuca J, Geurts JJG. Cognitive impairment in multiple sclerosis: clinical management, MRI, and therapeutic avenues. Lancet Neurol. 2020;19(10):860‐871. doi:10.1016/S1474-4422(20)30277-5
Benedict RHB, Deluca J, Phillips G, LaRocca N, Hudson LD, Rudick R. Validity of the Symbol Digit Modalities Test as a cognition performance outcome measure for multiple sclerosis. Mult Scler. 2017;23(5):721‐733. doi:10.1177/1352458517690821
Strober L, DeLuca J, Benedict RHB, et al. Symbol Digit Modalities Test: a valid clinical trial endpoint for measuring cognition in multiple sclerosis. Mult Scler. 2019;25(13):1781–1790. doi:10.1177/1352458518808204
Dillenseger A, Weidemann ML, Trentzsch K, et al. Digital biomarkers in multiple sclerosis. Brain Sci. 2021;11(11):1‐26. doi:10.3390/brainsci11111519
Marziniak M, Brichetto G, Feys P, Meyding‐Lamadé U, Vernon K, Meuth SG. The use of digital and remote communication technologies as a tool for multiple sclerosis management: narrative review. JMIR Rehabil Assist Technol. 2018;20(4):1‐21. doi:10.2196/rehab.7805
Organisation for Economic Co‐operation and Development (OECD). Internet access (indicator). 2024. doi:10.1787/69c2b997-en
Organisation for Economic Co‐operation and Development (OECD). Access to computers from home (indicator). 2024. doi:10.1787/a70b8a9f-en
Miller JB, Barr WB. The technology crisis in neuropsychology. Arch Clin Neuropsychol. 2017;32(5):541‐554. doi:10.1093/arclin/acx050
Thompson AJ, Banwell BL, Barkhof F, et al. Diagnosis of multiple sclerosis: 2017 revisions of the McDonald criteria. Lancet Neurol. 2018;17(2):162‐173. doi:10.1016/S1474-4422(17)30470-2
Henninger F, Shevchenko Y, Mertens UK, Kieslich PJ, Hilbig BE. Lab.Js: a free, open, online study builder. Behav Res Methods. 2022;54(2):556‐573. doi:10.3758/s13428-019-01283-5
Lange K, Kühn S, Filevich E. “Just another tool for online studies” (JATOS): an easy solution for setup and management of web servers supporting online studies. PLoS One. 2015;10(6):1‐14. doi:10.1371/journal.pone.0130834
Nasreddine ZS, Phillips NA, Bédirian V, et al. The Montreal Cognitive Assessment, MoCA: a brief screening tool for mild cognitive impairment. J Am Geriatr Soc. 2005;53(4):695‐699. doi:10.1111/j.1532-5415.2005.53221.x
Langdon DW, Amato MP, Boringa J, et al. Recommendations for a Brief International Cognitive Assessment for Multiple Sclerosis (BICAMS). Mult Scler J. 2012;18(6):891‐898. doi:10.1177/1352458511431076
Charest K, Tremblay A, Langlois R, Roger É, Duquette P, Rouleau I. Detecting subtle cognitive impairment in multiple sclerosis with the Montreal Cognitive Assessment. Can J Neurol Sci. 2020;47(5):620‐626. doi:10.1017/CJN.2020.97
Benedict RHB, Schretlen D, Groninger L, Dobraski M, Shpritz B. Revision of the Brief Visuospatial Memory Test: studies of normal performance, reliability, and validity. Psychol Assess. 1996;8(2):145‐153. doi:10.1037/1040-3590.8.2.145
Kim H‐Y. Statistical notes for clinical researchers: assessing normal distribution (2) using skewness and kurtosis. Restor Dent Endod. 2013;38(1):52‐54. doi:10.5395/rde.2013.38.1.52
Tukey JW. Exploratory Data Analysis: Past, Present and Future. Defense Technical Information Center; 1993.
Conti S, Bonazzi S, Laiacona M, Masina M, Coralli MV. Montreal Cognitive Assessment (MoCA)‐Italian version: regression based norms and equivalent scores. Neurol Sci. 2015;36(2):209‐214. doi:10.1007/s10072-014-1921-3
Goretti B, Niccolai C, Hakiki B, et al. The Brief International Cognitive Assessment for Multiple Sclerosis (BICAMS): normative values with gender, age and education corrections in the Italian population. BMC Neurol. 2014;14(1):171. doi:10.1186/s12883-014-0171-6
McKinney W. Data Structures for Statistical Computing in Python. Proceedings of the 9th Python in Science Conference. 2010; 1 (scipy):56‐61. doi:10.25080/majora-92bf1922-00a
Vallat R. Pingouin: statistics in Python. J Open Source Softw. 2018;3(31):1026. doi:10.21105/joss.01026
Harris CR, Millman KJ, van der Walt SJ, et al. Array programming with NumPy. Nature. 2020;585(7825):357‐362. doi:10.1038/s41586-020-2649-2
Seabold S, Perktold J. Statsmodels: Econometric and Statistical Modeling with Python. Proceedings of the 9th Python Science Conference. 2010;(Scipy):92‐96. doi:10.25080/majora-92bf1922-011
Virtanen P, Gommers R, Oliphant TE, et al. SciPy 1.0: fundamental algorithms for scientific computing in python. Nat Methods. 2020;17(3):261‐272. doi:10.1038/s41592-019-0686-2
Lam K, van Oirschot P, den Teuling B, et al. Reliability, construct and concurrent validity of a smartphone‐based cognition test in multiple sclerosis. Mult Scler J. 2022;28(2):300‐308. doi:10.1177/13524585211018103
van Dongen L, Westerik B, van der Hiele K, et al. Introducing Multiple Screener: an unsupervised digital screening tool for cognitive deficits in MS. Mult Scler Relat Disord. 2019;2020(38):101479. doi:10.1016/j.msard.2019.101479
Woelfle T, Pless S, Reyes O, et al. Reliability and acceptance of dreaMS, a software application for people with multiple sclerosis: a feasibility study. J Neurol. 2023;270(1):262‐271. doi:10.1007/s00415-022-11306-5
Khaligh‐Razavi SM, Sadeghi M, Khanbagi M, Kalafatis C, Nabavi SM. A self‐administered, artificial intelligence (AI) platform for cognitive assessment in multiple sclerosis (MS). BMC Neurol. 2020;20(1):1‐13. doi:10.1186/s12883-020-01736-x
Hsu W‐Y, Rowles W, Anguera JA, et al. Application of an adaptive, digital, game‐based approach for cognitive assessment in multiple sclerosis: observational study. J Med Internet Res. 2021;23(1):e24356. doi:10.2196/24356
Montalban X, Graves J, Midaglia L, et al. A smartphone sensor‐based digital outcome assessment of multiple sclerosis. Mult Scler J. 2022;28(4):654‐664. doi:10.1177/13524585211028561
Maillart E, Labauge P, Cohen M, et al. MSCopilot, a new multiple sclerosis self‐assessment digital solution: results of a comparative study versus standard tests. Eur J Neurol. 2020;27(3):429‐436. doi:10.1111/ene.14091
Pham L, Harris T, Varosanec M, Morgan V, Kosa P, Bielekova B. Smartphone‐based Symbol‐Digit Modalities Test reliably captures brain damage in multiple sclerosis. Npj Digit Med. 2021;4(1):1‐13. doi:10.1038/s41746-021-00401-y
Patel VP, Shen L, Rose J, Feinstein A. Taking the tester out of the SDMT: a proof of concept fully automated approach to assessing processing speed in people with MS. Mult Scler J. 2019;25(11):1506‐1513. doi:10.1177/1352458518792772
Rao SM, Losinski G, Mourany L, et al. Processing speed test: validation of a self‐administered, iPad®‐based tool for screening cognitive dysfunction in a clinic setting. Mult Scler. 2017;23(14):1929‐1937. doi:10.1177/1352458516688955
Middleton RM, Pearson OR, Ingram G, et al. A rapid electronic cognitive assessment measure for multiple sclerosis: validation of cognitive reaction, an electronic version of the Symbol Digit Modalities Test. J Med Internet Res. 2020;22(9):1‐14. doi:10.2196/18234
Scaramozza M, Chiesa PA, Zajac L, et al. Konectom™ cognitive processing speed test enables reliable remote, unsupervised cognitive assessment in people with multiple sclerosis: exploring the use of substitution time as a novel digital outcome measure. Mult Scler J. 2024;1‐12. doi:10.1177/13524585241259650
Benedict RHB, Cookfair D, Gavett R, et al. Validity of the minimal assessment of cognitive function in multiple sclerosis (MACFIMS). J Int Neuropsychol Soc. 2006;12(4):549‐558. doi:10.1017/s1355617706060723
Waskowiak PT, Ruitenberg MF, Hulst HE. Neuropsychological assessment in MS is outdated and is in need for innovation: yes. Mult Scler J. 2024;30:150‐151. doi:10.1177/13524585241230184
Elbulok‐Charcape MM, Rabin LA, Spadaccini AT, Barr WB. Trends in the neuropsychological assessment of ethnic/racial minorities: a survey of clinical neuropsychologists in the United States and Canada. Cultur Divers Ethnic Minor Psychol. 2014;20(3):353‐361. doi:10.1037/a0035023
Merlo D, Kalincik T, Zhu C, et al. Subjective versus objective performance in people with multiple sclerosis using the MSReactor computerised cognitive tests. Mult Scler Relat Disord. 2022;58:103393. doi:10.1016/j.msard.2021.103393
Lublin FD, Reingold SC, Cohen JA, et al. Defining the clinical course of multiple sclerosis: the 2013 revisions. Neurology. 2014;83(3):278‐286. doi:10.1212/WNL.0000000000000560