The Ubiquitous Cognitive Assessment Tool for Smartwatches: Design, Implementation, and Evaluation Study.
Stroop task
attention
cognition
memory
mobile phone
response time
wearable devices
Journal
JMIR mHealth and uHealth
ISSN: 2291-5222
Titre abrégé: JMIR Mhealth Uhealth
Pays: Canada
ID NLM: 101624439
Informations de publication
Date de publication:
01 06 2020
01 06 2020
Historique:
received:
17
12
2019
accepted:
31
03
2020
revised:
10
03
2020
entrez:
2
6
2020
pubmed:
2
6
2020
medline:
19
3
2021
Statut:
epublish
Résumé
Cognitive functioning plays a significant role in individuals' mental health, since fluctuations in memory, attention, and executive functions influence their daily task performance. Existing digital cognitive assessment tools cannot be administered in the wild and their test sets are not brief enough to capture frequent fluctuations throughout the day. The ubiquitous availability of mobile and wearable devices may allow their incorporation into a suitable platform for real-world cognitive assessment. The aims of this study were threefold: (1) to evaluate a smartwatch-based tool for the assessment of cognitive performance, (2) to investigate the usability of this tool, and (3) to understand participants' perceptions regarding the application of a smartwatch in cognitive assessment. We built the Ubiquitous Cognitive Assessment Tool (UbiCAT) on a smartwatch-based platform. UbiCAT implements three cognitive tests-an Arrow test, a Letter test, and a Color test-adapted from the two-choice reaction-time, N-back, and Stroop tests, respectively. These tests were designed together with domain experts. We evaluated the UbiCAT test measures against standard computer-based tests with 21 healthy adults by applying statistical analyses significant at the 95% level. Usability testing for each UbiCAT app was performed using the Mobile App Rating Scale (MARS) questionnaire. The NASA-TLX (Task Load Index) questionnaire was used to measure cognitive workload during the N-back test. Participants rated perceived discomfort of wearing a smartwatch during the tests using a 7-point Likert scale. Upon finishing the experiment, an interview was conducted with each participant. The interviews were transcribed and semantic analysis was performed to group the findings. Pearson correlation analysis between the total correct responses obtained from the UbiCAT and the computer-based tests revealed a significant strong correlation (r=.78, P<.001). One-way analysis of variance (ANOVA) showed a significant effect of the N-back difficulty level on the participants' performance measures. The study also demonstrated usability ratings above 4 out of 5 in terms of aesthetics, functionality, and information. Low discomfort (<3 out of 7) was reported by our participants after using the UbiCAT. Seven themes were extracted from the transcripts of the interviews conducted with our participants. UbiCAT is a smartwatch-based tool that assesses three key cognitive domains. Usability ratings showed that participants were engaged with the UbiCAT tests and did not feel any discomfort. The majority of the participants were interested in using the UbiCAT, although some preferred computer-based tests, which might be due to the widespread use of personal computers. The UbiCAT can be administered in the wild with mentally ill patients to assess their attention, working memory, and executive function.
Sections du résumé
BACKGROUND
Cognitive functioning plays a significant role in individuals' mental health, since fluctuations in memory, attention, and executive functions influence their daily task performance. Existing digital cognitive assessment tools cannot be administered in the wild and their test sets are not brief enough to capture frequent fluctuations throughout the day. The ubiquitous availability of mobile and wearable devices may allow their incorporation into a suitable platform for real-world cognitive assessment.
OBJECTIVE
The aims of this study were threefold: (1) to evaluate a smartwatch-based tool for the assessment of cognitive performance, (2) to investigate the usability of this tool, and (3) to understand participants' perceptions regarding the application of a smartwatch in cognitive assessment.
METHODS
We built the Ubiquitous Cognitive Assessment Tool (UbiCAT) on a smartwatch-based platform. UbiCAT implements three cognitive tests-an Arrow test, a Letter test, and a Color test-adapted from the two-choice reaction-time, N-back, and Stroop tests, respectively. These tests were designed together with domain experts. We evaluated the UbiCAT test measures against standard computer-based tests with 21 healthy adults by applying statistical analyses significant at the 95% level. Usability testing for each UbiCAT app was performed using the Mobile App Rating Scale (MARS) questionnaire. The NASA-TLX (Task Load Index) questionnaire was used to measure cognitive workload during the N-back test. Participants rated perceived discomfort of wearing a smartwatch during the tests using a 7-point Likert scale. Upon finishing the experiment, an interview was conducted with each participant. The interviews were transcribed and semantic analysis was performed to group the findings.
RESULTS
Pearson correlation analysis between the total correct responses obtained from the UbiCAT and the computer-based tests revealed a significant strong correlation (r=.78, P<.001). One-way analysis of variance (ANOVA) showed a significant effect of the N-back difficulty level on the participants' performance measures. The study also demonstrated usability ratings above 4 out of 5 in terms of aesthetics, functionality, and information. Low discomfort (<3 out of 7) was reported by our participants after using the UbiCAT. Seven themes were extracted from the transcripts of the interviews conducted with our participants.
CONCLUSIONS
UbiCAT is a smartwatch-based tool that assesses three key cognitive domains. Usability ratings showed that participants were engaged with the UbiCAT tests and did not feel any discomfort. The majority of the participants were interested in using the UbiCAT, although some preferred computer-based tests, which might be due to the widespread use of personal computers. The UbiCAT can be administered in the wild with mentally ill patients to assess their attention, working memory, and executive function.
Identifiants
pubmed: 32478664
pii: v8i6e17506
doi: 10.2196/17506
pmc: PMC7296405
doi:
Types de publication
Journal Article
Research Support, Non-U.S. Gov't
Langues
eng
Sous-ensembles de citation
IM
Pagination
e17506Informations de copyright
©Pegah Hafiz, Jakob Eyvind Bardram. Originally published in JMIR mHealth and uHealth (http://mhealth.jmir.org), 01.06.2020.
Références
Neuropsychology. 2015 Jan;29(1):45-58
pubmed: 24933488
Int J Methods Psychiatr Res. 2019 Sep;28(3):e1775
pubmed: 30761648
JMIR Form Res. 2019 Jul 26;3(3):e13898
pubmed: 31350840
Psychol Assess. 2009 Dec;21(4):457-62
pubmed: 19947780
J Exp Psychol. 1958 Apr;55(4):352-8
pubmed: 13539317
Appl Neuropsychol Adult. 2014;21(2):136-42
pubmed: 24826507
Clin Psychol (New York). 2009 Jun;16(2):256-277
pubmed: 22493520
Acta Astronaut. 2011 Dec 1;69(11-12):949-959
pubmed: 22025811
Int J Geriatr Psychiatry. 2012 Oct;27(10):989-98
pubmed: 22278997
JMIR Mhealth Uhealth. 2015 Mar 11;3(1):e27
pubmed: 25760773
Psychol Med. 2017 Dec;47(16):2753-2766
pubmed: 28585513
Acta Psychol (Amst). 1969;30:412-31
pubmed: 5811531
Schizophr Res. 2004 Jun 1;68(2-3):283-97
pubmed: 15099610
Sleep. 1999 May 1;22 Suppl 2:S347-53
pubmed: 10394606
PLoS One. 2014 Feb 19;9(2):e88820
pubmed: 24586404
JMIR Ment Health. 2019 Nov 18;6(11):e12814
pubmed: 31738172
Behav Res Methods. 2010 Nov;42(4):1096-104
pubmed: 21139177
Cogn Neuropsychol. 2007 Oct;24(7):755-89
pubmed: 18066734
Crit Rev Neurobiol. 2000;14(1):1-21
pubmed: 11253953
PLoS One. 2013 Jun 11;8(6):e65925
pubmed: 23776570
JMIR Mhealth Uhealth. 2017 May 25;5(5):e68
pubmed: 28546139
Neurosci Biobehav Rev. 2009 Feb;33(2):81-8
pubmed: 18771686
Bipolar Disord. 2020 Feb;22(1):28-37
pubmed: 31541587
Alcohol Clin Exp Res. 2009 Dec;33(12):2094-102
pubmed: 19740132
J Psychiatr Res. 2010 Feb;44(3):168-76
pubmed: 19700172
J Healthc Inform Res. 2018 Jun;2(1-2):1-24
pubmed: 30035250
Int Psychogeriatr. 2015 Aug;27(8):1251-62
pubmed: 25742756
Arch Clin Neuropsychol. 2009 Mar;24(2):165-78
pubmed: 19395350
Int J Methods Psychiatr Res. 2018 Sep;27(3):e1736
pubmed: 30088298
Int J Methods Psychiatr Res. 2017 Dec;26(4):
pubmed: 28370881
J Psychoactive Drugs. 2016 Sep-Oct;48(4):288-94
pubmed: 27260123