A Deep Learning Approach for Human Action Recognition Using Skeletal Information.
Activities of daily living
Convolutional neural networks
Human action recognition
Journal
Advances in experimental medicine and biology
ISSN: 0065-2598
Titre abrégé: Adv Exp Med Biol
Pays: United States
ID NLM: 0121103
Informations de publication
Date de publication:
2020
2020
Historique:
entrez:
30
5
2020
pubmed:
30
5
2020
medline:
9
9
2020
Statut:
ppublish
Résumé
In this paper we present an approach toward human action detection for activities of daily living (ADLs) that uses a convolutional neural network (CNN). The network is trained on discrete Fourier transform (DFT) images that result from raw sensor readings, i.e., each human action is ultimately described by an image. More specifically, we work using 3D skeletal positions of human joints, which originate from processing of raw RGB sequences enhanced by depth information. The motion of each joint may be described by a combination of three 1D signals, representing its coefficients into a 3D Euclidean space. All such signals from a set of human joints are concatenated to form an image, which is then transformed by DFT and is used for training and evaluation of a CNN. We evaluate our approach using a publicly available challenging dataset of human actions that may involve one or more body parts simultaneously and for two sets of actions which resemble common ADLs.
Identifiants
pubmed: 32468527
doi: 10.1007/978-3-030-32622-7_9
doi:
Types de publication
Journal Article
Langues
eng
Sous-ensembles de citation
IM
Pagination
105-114Références
Abadi M et al (2016) TensorFlow: a system for large-scale machine learning. In: Proceedings of the USENIX symposium on operating systems design and implementation (OSDI)
Berretti S, Daoudi M, Turaga P, Basu A (2018) Representation, analysis, and recognition of 3D humans: a survey. ACM Trans Multim Comput Commun Appl (TOMM) 14(1s):16
Chollet F (2015) Keras. https://github.com/fchollet/keras
Du Y, Fu Y, Wang L (2015) Skeleton based action recognition with convolutional neural network. In: Proceedings of 3rd IAPR Asian conference on pattern recognition (ACPR). IEEE
Graves A, Mohamed AR, Hinton G (2013) Speech recognition with deep recurrent neural networks. In: Proceedings of IEEE international conference of acoustics, speech and signal processing (ICASSP)
Hou Y, Li Z, Wang P, Li W (2018) Skeleton optical spectra-based action recognition using convolutional neural networks. IEEE Trans Circuits Syst Video Technol 28(3):807–811
doi: 10.1109/TCSVT.2016.2628339
Jiang W, Yin Z (2015) Human activity recognition using wearable sensors by deep convolutional neural networks. In: Proceedings of ACM international conference on multimedia (MM)
Ke Q, An S, Bennamoun M, Sohel F, Boussaid F (2017) Skeletonnet: mining deep part features for 3-d action recognition. IEEE Signal Process Lett 24(6):731–735
doi: 10.1109/LSP.2017.2690339
Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: Advances in neural information processing systems, pp 1097–1105
Lawton MP, Brody EM (1969) Assessment of older people: self-maintaining and instrumental activities of daily living. Gerontologist 9(3 Part 1):179–186
doi: 10.1093/geront/9.3_Part_1.179
LeCun Y, Bottou L, Bengio Y, Haner P (1998) Gradient-based learning applied to document recognition. Proc IEEE 86(11):2278–2324
doi: 10.1109/5.726791
Li C, Hou Y, Wang P, Li W (2017) Joint distance maps based action recognition with convolutional neural networks. IEEE Signal Process Lett 24(5):624–628
doi: 10.1109/LSP.2017.2678539
Liu C, Hu Y, Li Y, Song S, Liu J (2017a) PKU-MMD: a large scale benchmark for continuous multi-modal human action understanding. In: Proceedings of ACM multimedia workshop (MM)
Liu M, Liu H, Chen C (2017b) Enhanced skeleton visualization for view invariant human action recognition. Pattern Recogn 68:346–362
doi: 10.1016/j.patcog.2017.02.030
Mathe E, Mitsou A, Spyrou E, Mylonas P (2018) Arm gesture recognition using a convolutional neural network. In: Proceedings of international workshop on semantic and social media adaptation and personalization (SMAP)
Srivastava N, Hinton G, Krizhevsky A, Sutskever I, Salakhutdinov R (2014) Dropout: a simple way to prevent neural networks from overfitting. J Mach Learn Res 15(1):1929–1958
Wang P, Li W, Ogunbona P, Wan J, Escalera S (2018a) RGB-D-based human motion recognition with deep learning: a survey. Comput Vis Image Underst 171:118–139
doi: 10.1016/j.cviu.2018.04.007
Wang P, Li W, Li C, Hou Y (2018b) Action recognition based on joint trajectory maps with convolutional neural networks. Knowl-Based Syst 158:43–53
doi: 10.1016/j.knosys.2018.05.029
Zhang Z (2012) Microsoft Kinect sensor and its effect. IEEE Multim 19(2):4–10
doi: 10.1109/MMUL.2012.24