Moving towards intelligent telemedicine: Computer vision measurement of human movement.
Computer vision
DeepLabCut
Finger tapping
Motor control
Telemedicine
Journal
Computers in biology and medicine
ISSN: 1879-0534
Titre abrégé: Comput Biol Med
Pays: United States
ID NLM: 1250250
Informations de publication
Date de publication:
08 2022
08 2022
Historique:
received:
01
04
2022
revised:
31
05
2022
accepted:
19
06
2022
pubmed:
4
7
2022
medline:
14
7
2022
entrez:
3
7
2022
Statut:
ppublish
Résumé
Telemedicine video consultations are rapidly increasing globally, accelerated by the COVID-19 pandemic. This presents opportunities to use computer vision technologies to augment clinician visual judgement because video cameras are so ubiquitous in personal devices and new techniques, such as DeepLabCut (DLC) can precisely measure human movement from smartphone videos. However, the accuracy of DLC to track human movements in videos obtained from laptop cameras, which have a much lower FPS, has never been investigated; this is a critical gap because patients use laptops for most telemedicine consultations. To determine the validity and reliability of DLC applied to laptop videos to measure finger tapping, a validated test of human movement. Sixteen adults completed finger-tapping tests at 0.5 Hz, 1 Hz, 2 Hz, 3 Hz and at maximal speed. Hand movements were recorded simultaneously by a laptop camera at 30 frames per second (FPS) and by Optotrak, a 3D motion analysis system at 250 FPS. Eight DLC neural network architectures (ResNet50, ResNet101, ResNet152, MobileNetV1, MobileNetV2, EfficientNetB0, EfficientNetB3, EfficientNetB6) were applied to the laptop video and extracted movement features were compared to the ground truth Optotrak motion tracking. Over 96% (529/552) of DLC measures were within +/-0.5 Hz of the Optotrak measures. At tapping frequencies >4 Hz, there was progressive decline in accuracy, attributed to motion blur associated with the laptop camera's low FPS. Computer vision methods hold potential for moving us towards intelligent telemedicine by providing human movement analysis during consultations. However, further developments are required to accurately measure the fastest movements.
Sections du résumé
BACKGROUND
Telemedicine video consultations are rapidly increasing globally, accelerated by the COVID-19 pandemic. This presents opportunities to use computer vision technologies to augment clinician visual judgement because video cameras are so ubiquitous in personal devices and new techniques, such as DeepLabCut (DLC) can precisely measure human movement from smartphone videos. However, the accuracy of DLC to track human movements in videos obtained from laptop cameras, which have a much lower FPS, has never been investigated; this is a critical gap because patients use laptops for most telemedicine consultations.
OBJECTIVES
To determine the validity and reliability of DLC applied to laptop videos to measure finger tapping, a validated test of human movement.
METHOD
Sixteen adults completed finger-tapping tests at 0.5 Hz, 1 Hz, 2 Hz, 3 Hz and at maximal speed. Hand movements were recorded simultaneously by a laptop camera at 30 frames per second (FPS) and by Optotrak, a 3D motion analysis system at 250 FPS. Eight DLC neural network architectures (ResNet50, ResNet101, ResNet152, MobileNetV1, MobileNetV2, EfficientNetB0, EfficientNetB3, EfficientNetB6) were applied to the laptop video and extracted movement features were compared to the ground truth Optotrak motion tracking.
RESULTS
Over 96% (529/552) of DLC measures were within +/-0.5 Hz of the Optotrak measures. At tapping frequencies >4 Hz, there was progressive decline in accuracy, attributed to motion blur associated with the laptop camera's low FPS. Computer vision methods hold potential for moving us towards intelligent telemedicine by providing human movement analysis during consultations. However, further developments are required to accurately measure the fastest movements.
Identifiants
pubmed: 35780600
pii: S0010-4825(22)00541-8
doi: 10.1016/j.compbiomed.2022.105776
pmc: PMC9428734
pii:
doi:
Types de publication
Journal Article
Research Support, Non-U.S. Gov't
Langues
eng
Sous-ensembles de citation
IM
Pagination
105776Informations de copyright
Copyright © 2022 Elsevier Ltd. All rights reserved.
Références
J Med Eng Technol. 2006 Jan-Feb;30(1):11-6
pubmed: 16393848
IEEE Int Conf Rehabil Robot. 2017 Jul;2017:1556-1561
pubmed: 28814041
J Gastrointest Surg. 2020 Jul;24(7):1692-1697
pubmed: 32385614
Mov Disord Clin Pract. 2016 Oct 24;4(3):368-373
pubmed: 30363378
Telemed J E Health. 2018 Jul;24(7):471-480
pubmed: 29252110
Artif Intell Med. 2014 Jan;60(1):27-40
pubmed: 24332155
J Biomed Inform. 2022 Mar;127:104030
pubmed: 35183766
Arch Neurol. 1991 Jun;48(6):644-8
pubmed: 2039388
Nat Neurosci. 2018 Sep;21(9):1281-1289
pubmed: 30127430
Sensors (Basel). 2021 Nov 28;21(23):
pubmed: 34883949
Mov Disord. 2003 May;18(5):560-5
pubmed: 12722170
J Neurol Sci. 2020 Jul 15;414:116930
pubmed: 32460041
J Neural Eng. 2013 Oct;10(5):056020
pubmed: 24045504
J Neurol Sci. 2020 Sep 15;416:117003
pubmed: 32645513
Sensors (Basel). 2009;9(3):2187-201
pubmed: 22574008
Sensors (Basel). 2017 Jan 25;17(2):
pubmed: 28125051
Ann Clin Transl Neurol. 2021 Feb;8(2):440-447
pubmed: 33377609
Entropy (Basel). 2020 Mar 24;22(3):
pubmed: 33286143
Annu Int Conf IEEE Eng Med Biol Soc. 2011;2011:7123-6
pubmed: 22255980
Artif Intell Med. 2020 Nov;110:101966
pubmed: 33250146
BMJ. 2020 Mar 25;368:m1182
pubmed: 32213507
Nat Protoc. 2019 Jul;14(7):2152-2176
pubmed: 31227823
Brain. 1986 Aug;109 ( Pt 4):739-57
pubmed: 3730813
Biochem Med (Zagreb). 2015 Jun 05;25(2):141-51
pubmed: 26110027
Mov Disord Clin Pract. 2020 Dec 28;8(1):69-75
pubmed: 34853806