Computer vision for kinematic metrics of the drinking task in a pilot study of neurotypical participants.
Biomechanics
Human pose estimation
Machine learning
Markerless motion capture
Neurorehabilitation
Upper extremity
Journal
Scientific reports
ISSN: 2045-2322
Titre abrégé: Sci Rep
Pays: England
ID NLM: 101563288
Informations de publication
Date de publication:
05 Sep 2024
05 Sep 2024
Historique:
received:
13
03
2024
accepted:
28
08
2024
medline:
6
9
2024
pubmed:
6
9
2024
entrez:
5
9
2024
Statut:
epublish
Résumé
Assessment of the upper limb is critical to guiding the rehabilitation cycle. Drawbacks of observation-based assessment include subjectivity and coarse resolution of ordinal scales. Kinematic assessment gives rise to objective quantitative metrics, but uptake is encumbered by costly and impractical setups. Our objective was to investigate feasibility and accuracy of computer vision (CV) for acquiring kinematic metrics of the drinking task, which are recommended in stroke rehabilitation research. We implemented CV for upper limb kinematic assessment using modest cameras and an open-source machine learning solution. To explore feasibility, 10 neurotypical participants were recruited for repeated kinematic measures during the drinking task. To investigate accuracy, a simultaneous marker-based motion capture system was used, and error was quantified for the following kinematic metrics: Number of Movement Units (NMU), Trunk Displacement (TD), and Movement Time (MT). Across all participant trials, kinematic metrics of the drinking task were successfully acquired using CV. Compared to marker-based motion capture, no significant difference was observed for group mean values of kinematic metrics. Mean error for NMU, TD, and MT were - 0.12 units, 3.4 mm, and 0.15 s, respectively. Bland-Altman analysis revealed no bias. Kinematic metrics of the drinking task can be measured using CV, and preliminary findings support accuracy. Further study in neurodivergent populations is needed to determine validity of CV for kinematic assessment of the post-stroke upper limb.
Identifiants
pubmed: 39237646
doi: 10.1038/s41598-024-71470-8
pii: 10.1038/s41598-024-71470-8
doi:
Types de publication
Journal Article
Langues
eng
Sous-ensembles de citation
IM
Pagination
20668Subventions
Organisme : NIH HHS
ID : UL1TR001998
Pays : United States
Informations de copyright
© 2024. The Author(s).
Références
Kwakkel, G., Kollen, B. J., van der Grond, J. & Prevo, A. J. Probability of regaining dexterity in the flaccid upper limb: Impact of severity of paresis and time since onset in acute stroke. Stroke 34(9), 2181–2186 (2003).
pubmed: 12907818
doi: 10.1161/01.STR.0000087172.16305.CD
Tsao, C. W. et al. Heart disease and stroke statistics—2022 update: A report from the American Heart Association. Circulation 145(8), e153–e639 (2022).
pubmed: 35078371
doi: 10.1161/CIR.0000000000001052
World Health Organization. World Report on Disability 2011 (World Health Organization, 2011).
World Health Organization. International Classification of Functioning, Disability, and Health: ICF 2001 (World Health Organization, 2001).
Moore, J. L., Raad, J., Ehrlich-Jones, L. & Heinemann, A. W. Development and use of a knowledge translation tool: The rehabilitation measures database. Arch. Phys. Med. Rehabil. 95(1), 197–202 (2014).
pubmed: 24076083
doi: 10.1016/j.apmr.2013.09.010
Alt Murphy, M., Sunnerhagen, K. S., Johnels, B. & Willén, C. Three-dimensional kinematic motion analysis of a daily activity drinking from a glass: A pilot study. J. Neuroeng. Rehabil. 3, 18 (2006).
pubmed: 16914057
doi: 10.1186/1743-0003-3-18
Schwarz, A., Kanzler, C. M., Lambercy, O., Luft, A. R. & Veerbeek, J. M. Systematic review on kinematic assessments of upper limb movements after stroke. Stroke (1970) 50(3), 718–727 (2019).
doi: 10.1161/STROKEAHA.118.023531
Alt Murphy, M. (eds) Development and validation of upper extremity kinematic movement analysis for people with stroke. Reaching and drinking from a glass (2013).
Thrane, G., Sunnerhagen, K. S. & Murphy, M. A. Upper limb kinematics during the first year after stroke: The stroke arm longitudinal study at the University of Gothenburg (SALGOT). J. Neuroeng. Rehabil. 17(1), 1–12 (2020).
doi: 10.1186/s12984-020-00705-2
Levin, M. F., Kleim, J. A. & Wolf, S. L. What do motor “recovery” and “compensation” mean in patients following stroke?. Neurorehabil. Neural Repair 23(4), 313–319 (2009).
pubmed: 19118128
doi: 10.1177/1545968308328727
Alt Murphy, M., Murphy, S., Persson, H. C., Bergström, U. B. & Sunnerhagen, K. S. Kinematic analysis using 3D motion capture of drinking task in people with and without upper-extremity impairments. J. Vis. Exp. https://doi.org/10.3791/57228-v (2018).
doi: 10.3791/57228-v
pubmed: 29658937
pmcid: 5933268
Ellis, M. D., Lan, Y., Yao, J. & Dewald, J. P. A. Robotic quantification of upper extremity loss of independent joint control or flexion synergy in individuals with hemiparetic stroke: A review of paradigms addressing the effects of shoulder abduction loading. J. Neuroeng. Rehabil. 13(1), 95 (2016).
pubmed: 27794362
pmcid: 5086410
doi: 10.1186/s12984-016-0203-0
Xu, J. et al. Separable systems for recovery of finger strength and control after stroke. J. Neurophysiol. 118(2), 1151–1163 (2017).
pubmed: 28566461
pmcid: 5547267
doi: 10.1152/jn.00123.2017
Kwakkel, G. et al. Standardized measurement of quality of upper limb movement after stroke: Consensus-based core recommendations from the second stroke recovery and rehabilitation roundtable. Int. J. Stroke 14(8), 783–791 (2019).
pubmed: 31510885
doi: 10.1177/1747493019873519
Krakauer, J. W., Carmichael, S. T., Corbett, D. & Wittenberg, G. F. Getting neurorehabilitation right: What can be learned from animal models?. Neurorehabil. Neural Repair 26(8), 923–931 (2012).
pubmed: 22466792
pmcid: 4554531
doi: 10.1177/1545968312440745
Kanko, R. M., Laende, E. K., Davis, E. M., Selbie, W. S. & Deluzio, K. J. Concurrent assessment of gait kinematics using marker-based and markerless motion capture. J. Biomech. 127, 110665 (2021).
pubmed: 34380101
doi: 10.1016/j.jbiomech.2021.110665
Nakano, N. et al. Evaluation of 3D Markerless motion capture accuracy using OpenPose with multiple video cameras. Front. Sports Act. Living https://doi.org/10.3389/fspor.2020.00050 (2020).
doi: 10.3389/fspor.2020.00050
pubmed: 33345042
pmcid: 7739760
Needham, L. et al. The development and evaluation of a fully automated markerless motion capture workflow. J. Biomech. 144, 111338 (2022).
pubmed: 36252308
doi: 10.1016/j.jbiomech.2022.111338
Song, K., Hullfish, T. J., Silva, R. S., Silbernagel, K. G. & Baxter, J. R. Markerless motion capture estimates of lower extremity kinematics and kinetics are comparable to marker-based across 8 movements. bioRxiv. 2023.02.21.526496 (2023).
Uhlrich, S. D., Falisse, A., Kidziński, Ł., Muccini, J., Ko, M., Chaudhari, A. S. et al. OpenCap: 3D human movement dynamics from smartphone videos. bioRxiv. 2022.07.07.499061 (2022).
Prince, S. Computer Vision: Models, Learning, and Inference (Cambridge University Press, 2012).
doi: 10.1017/CBO9780511996504
Cao, Z., Hidalgo, G., Simon, T., Wei, S. E. & Sheikh, Y. OpenPose: Realtime multi-person 2D pose estimation using part affinity fields. IEEE Trans. Pattern Anal. Mach. Intell. 43(1), 172–186 (2021).
pubmed: 31331883
doi: 10.1109/TPAMI.2019.2929257
Lugaresi, C., Tang, J., Nash, H., McClanahan, C., Uboweja, E., Hays, M. et al. MediaPipe: A Framework for Building Perception Pipelines. https://doi.org/10.48550/arXiv.1906.08172 . Available from: https://ui.adsabs.harvard.edu/abs/2019arXiv190608172L (2019).
Kendall, A., Grimes, M. & Cipolla, R. (eds) PoseNet: A convolutional network for real-time 6-DOF camera relocalization. In 2015 IEEE International Conference on Computer Vision (ICCV) 7–13 (2015).
Bajpai, R. & Joshi, D. MoveNet: A deep neural network for joint profile prediction across variable walking speeds and slopes. IEEE Trans. Instrum. Meas. 70, 1–11 (2021).
Chung, J.-L., Ong, L.-Y. & Chew, L. Comparative analysis of skeleton-based human pose estimation. Future Internet 14, 380 (2022).
doi: 10.3390/fi14120380
Kim, W., Sung, J., Saakes, D., Huang, C. & Xiong, S. Ergonomic postural assessment using a new open-source human pose estimation technology (OpenPose). Int. J. Ind. Ergon. 84, 103164 (2021).
doi: 10.1016/j.ergon.2021.103164
Jiaming, C., Ong, L.-Y. & Chew, L. Telehealth Using posenet-based system for in-home rehabilitation. Future Internet 13, 173 (2021).
doi: 10.3390/fi13070173
Kidziński, Ł et al. Deep neural networks enable quantitative movement analysis using single-camera videos. Nat. Commun. 11(1), 4054 (2020).
pubmed: 32792511
pmcid: 7426855
doi: 10.1038/s41467-020-17807-z
Baptista, R. et al. Home self-training: Visual feedback for assisting physical activity for stroke survivors. Comput. Methods Progr. Biomed. 176, 111–120 (2019).
doi: 10.1016/j.cmpb.2019.04.019
Capecci, M. et al. The KIMORE dataset: Kinematic assessment of movement and clinical scores for remote monitoring of physical rehabilitation. IEEE Trans. Neural Syst. Rehabilit. Eng. 27(7), 1436–1448 (2019).
doi: 10.1109/TNSRE.2019.2923060
Cary, F., Postolache, O. & Girão, P. S. (eds) Kinect based system and Artificial Neural Networks classifiers for physiotherapy assessment. In 2014 IEEE International Symposium on Medical Measurements and Applications (MeMeA) 11–12 (2014).
Eichler, N., Hel-Or, H., Shmishoni, I., Itah, D., Gross, B. & Raz, S. (eds). Non-invasive motion analysis for stroke rehabilitation using off the shelf 3D sensors. In 2018 International Joint Conference on Neural Networks (IJCNN) 8–13 (2018).
Pogrzeba, L., Neumann, T., Wacker, M. & Jung, B. Analysis and quantification of repetitive motion in long-term rehabilitation. IEEE J. Biomed. Health Inform. 23(3), 1075–1085 (2019).
pubmed: 29994665
doi: 10.1109/JBHI.2018.2848103
Lee, T., Jeon, E. T., Jung, J. M. & Lee, M. Deep-learning-based stroke screening using skeleton data from neurological examination videos. J. Pers. Med. 12(10), 1691 (2022).
pubmed: 36294830
pmcid: 9604814
doi: 10.3390/jpm12101691
Lin, H. P., Zhao, L., Woolley, D., Zhang, X., Cheng, H. J., Liang, W. et al. Exploring the feasibility of computer vision for detecting post-stroke compensatory movements. In IEEE International Conference on Rehabilitation Robotics, Vol. 2023 1–6 (2023).
Zestas, O. N., Soumis, D. N., Kyriakou, K. D., Seklou, K. & Tselikas, N. D. A computer-vision based hand rehabilitation assessment suite. AEU Int. J. Electron. Commun. 169, 154762 (2023).
doi: 10.1016/j.aeue.2023.154762
Kim, B. & Neville, C. Accuracy and feasibility of a novel fine hand motor skill assessment using computer vision object tracking. Sci. Rep. 13(1), 1813 (2023).
pubmed: 36725905
pmcid: 9892571
doi: 10.1038/s41598-023-29091-0
Debnath, B., O’Brien, M., Yamaguchi, M. & Behera, A. A review of computer vision-based approaches for physical rehabilitation and assessment. Multimed. Syst. 28(1), 209–239 (2022).
doi: 10.1007/s00530-021-00815-4
Alt Murphy, M., Willén, C. & Sunnerhagen, K. S. Kinematic variables quantifying upper-extremity performance after stroke during reaching and drinking from a glass. Neurorehabil. Neural Repair 25(1), 71–80 (2011).
pubmed: 20829411
doi: 10.1177/1545968310370748
Alt Murphy, M., Willén, C. & Sunnerhagen, K. S. Movement kinematics during a drinking task are associated with the activity capacity level after stroke. Neurorehabil. Neural Repair 26(9), 1106–1115 (2012).
pubmed: 22647879
doi: 10.1177/1545968312448234
Shapiro, R. Direct linear transformation method for three-dimensional cinematography. Res. Q. 49(2), 197–205 (1978).
pubmed: 725286
Alt Murphy, M., Willén, C. & Sunnerhagen, K. S. Responsiveness of upper extremity kinematic measures and clinical improvement during the first three months after stroke. Neurorehabil. Neural Repair 27(9), 844–853 (2013).
pubmed: 23764883
doi: 10.1177/1545968313491008
Cerveri, P., Pedotti, A. & Ferrigno, G. Robust recovery of human motion from video using Kalman filters and virtual humans. Hum. Mov. Sci. 22(3), 377–404 (2003).
pubmed: 12967764
doi: 10.1016/S0167-9457(03)00004-6
Crenna, F., Rossi, G. B. & Berardengo, M. Filtering biomechanical signals in movement analysis. Sensors (Basel) 21(13), 4580 (2021).
pubmed: 34283131
doi: 10.3390/s21134580
Youngjoo, K. & Hyochoong, B. Introduction to Kalman filter and its applications. In Ch. 2 Introduction and Implementations of the Kalman Filter (ed. Felix, G.) (IntechOpen, 2018).
Kabsch, W. A solution for the best rotation to relate two sets of vectors. Acta Crystallogr. Sect. A 32(5), 922–923 (1976).
doi: 10.1107/S0567739476001873
Needham, L., Evans, M., Cosker, D. P. & Colyer, S. L. Can Markerless pose estimation algorithms estimate 3D mass centre positions and velocities during linear sprinting activities?. Sensors 21(8), 2889 (2021).
pubmed: 33924266
pmcid: 8074772
doi: 10.3390/s21082889
Bland, J. M. & Altman, D. G. Agreement between methods of measurement with multiple observations per individual. J. Biopharm. Stat. 17(4), 571–582 (2007).
pubmed: 17613642
doi: 10.1080/10543400701329422
Sivan, M., O’Connor, R. J., Makower, S., Levesley, M. & Bhakta, B. Systematic review of outcome measures used in the evaluation of robot-assisted upper limb exercise in stroke. J. Rehabilit. Med. 43(3), 181–189 (2011).
doi: 10.2340/16501977-0674
Yang, S. et al. Markerless motion capture systems for tracking of persons in forensic biomechanics: An overview. Comput. Methods Biomech. Biomed. Eng. 2, 46–65 (2013).
Urbanczyk, C. A., Bonfiglio, A., McGregor, A. H. & Bull, A. M. J. Comparing optical and electromagnetic tracking systems to facilitate compatibility in sports kinematics data. Int. Biomech. 8(1), 75–84 (2021).
pubmed: 34806553
pmcid: 8635616
doi: 10.1080/23335432.2021.2003719
Hindle, B. R., Keogh, J. W. L. & Lorimer, A. V. Inertial-based human motion capture: A technical summary of current processing methodologies for spatiotemporal and kinematic measures. Appl. Bionics Biomech. 2021, 6628320 (2021).
pubmed: 33859720
pmcid: 8024877
doi: 10.1155/2021/6628320
Cutti, A. G., Giovanardi, A., Rocchi, L., Davalli, A. & Sacchetti, R. Ambulatory measurement of shoulder and elbow kinematics through inertial and magnetic sensors. Med. Biol. Eng. Comput. 46(2), 169–178 (2008).
pubmed: 18087742
doi: 10.1007/s11517-007-0296-5
El-Gohary, M. & McNames, J. Shoulder and elbow joint angle tracking with inertial sensors. IEEE Trans. Biomed. Eng. 59(9), 2635–2641 (2012).
pubmed: 22911538
doi: 10.1109/TBME.2012.2208750
Schmitz, A. et al. The measurement of in vivo joint angles during a squat using a single camera markerless motion capture system as compared to a marker based system. Gait Posture 41(2), 694–698 (2015).
pubmed: 25708833
doi: 10.1016/j.gaitpost.2015.01.028
Natarajan, S. K., Wang, X., Spranger, M. & Gräser, A. (eds) Reha@Home—A vision based markerless gait analysis system for rehabilitation at home. In 2017 13th IASTED International Conference on Biomedical Engineering (BioMed) 20–21 (2017).
Metcalf, C. D. et al. Markerless motion capture and measurement of hand kinematics: Validation and application to home-based upper limb rehabilitation. IEEE Trans. Biomed. Eng. 60(8), 2184–2192 (2013).
pubmed: 23475333
doi: 10.1109/TBME.2013.2250286
Scott, B., Chadwick, E., McInnes, M. & Blana, D. Assessing single camera markerless motion capture during upper limb activities of daily living. Gait Posture 106, S184 (2023).
doi: 10.1016/j.gaitpost.2023.07.222
Sandau, M. et al. Markerless motion capture can provide reliable 3D gait kinematics in the sagittal and frontal plane. Med. Eng. Phys. 36(9), 1168–1175 (2014).
pubmed: 25085672
doi: 10.1016/j.medengphy.2014.07.007
Rab, G., Petuskey, K. & Bagley, A. A method for determination of upper extremity kinematics. Gait Posture 15(2), 113–119 (2002).
pubmed: 11869904
doi: 10.1016/S0966-6362(01)00155-2
Benoit, D. L. et al. Effect of skin movement artifact on knee kinematics during gait and cutting motions measured in vivo. Gait Posture 24(2), 152–164 (2006).
pubmed: 16260140
doi: 10.1016/j.gaitpost.2005.04.012
Kanko, R. M., Laende, E., Selbie, W. S. & Deluzio, K. J. Inter-session repeatability of markerless motion capture gait kinematics. J. Biomech. 121, 110422 (2021).
pubmed: 33873117
doi: 10.1016/j.jbiomech.2021.110422