Automated anomalous child repetitive head movement identification through transformer networks.
Atypical head movements
Head gesture classification
Head pose estimation
Movement analysis
Non-deterministic finite automata (NFA)
Periodicity
Repetitive behavior analysis
Transfer learning
Transformer
Journal
Physical and engineering sciences in medicine
ISSN: 2662-4737
Titre abrégé: Phys Eng Sci Med
Pays: Switzerland
ID NLM: 101760671
Informations de publication
Date de publication:
Dec 2023
Dec 2023
Historique:
received:
01
02
2023
accepted:
24
07
2023
medline:
11
12
2023
pubmed:
10
10
2023
entrez:
9
10
2023
Statut:
ppublish
Résumé
The increasing prevalence of behavioral disorders in children is of growing concern within the medical community. Recognising the significance of early identification and intervention for atypical behaviors, there is a consensus on their pivotal role in improving outcomes. Due to inadequate facilities and a shortage of medical professionals with specialized expertise, traditional diagnostic methods have been unable to effectively address the rising incidence of behavioral disorders. Hence, there is a need to develop automated approaches for the diagnosis of behavioral disorders in children, to overcome the challenges with traditional methods. The purpose of this study is to develop an automated model capable of analyzing videos to differentiate between typical and atypical repetitive head movements in. To address problems resulting from the limited availability of child datasets, various learning methods are employed to mitigate these issues. In this work, we present a fusion of transformer networks, and Non-deterministic Finite Automata (NFA) techniques, which classify repetitive head movements of a child as typical or atypical based on an analysis of gender, age, and type of repetitive head movement, along with count, duration, and frequency of each repetitive head movement. Experimentation was carried out with different transfer learning methods to enhance the performance of the model. The experimental results on five datasets: NIR face dataset, Bosphorus 3D face dataset, ASD dataset, SSBD dataset, and the Head Movements in the Wild dataset, indicate that our proposed model has outperformed many state-of-the-art frameworks when distinguishing typical and atypical repetitive head movements in children.
Identifiants
pubmed: 37814077
doi: 10.1007/s13246-023-01309-5
pii: 10.1007/s13246-023-01309-5
doi:
Types de publication
Journal Article
Langues
eng
Sous-ensembles de citation
IM
Pagination
1427-1445Informations de copyright
© 2023. Australasian College of Physical Scientists and Engineers in Medicine.
Références
Goldman S, Wang C, Salgado MW, Greene PE, Kim M, Rapin I (2009) Motor stereotypies in children with autism and other developmental disorders. Dev Med Child Neurol 51(1):30–38
pubmed: 19087102
doi: 10.1111/j.1469-8749.2008.03178.x
Edition F et al (2013) Diagnostic and statistical manual of mental disorders. Am Psychiatric Assoc 21(21):591–643
Borjon JI, Abney DH, Yu C, Smith LB (2021) Head and eyes: looking behavior in 12-to 24-month-old infants. J Vis 21(8):18–18
pubmed: 34403460
pmcid: 8375006
doi: 10.1167/jov.21.8.18
Martin KB, Hammal Z, Ren G, Cohn JF, Cassell J, Ogihara M, Britton JC, Gutierrez A, Messinger DS (2018) Objective measurement of head movement differences in children with and without autism spectrum disorder. Mol Autism 9(1):1–10
doi: 10.1186/s13229-018-0198-4
Peiris H, Wickramarachchi DC, Samarasinghe P, Vance P, Dahanayake D, Kulasekara V, Nadeeshani M (2022) Diagnosing autism in low-income countries: clinical record-based analysis in Sri Lanka. Autism Res 15(7):1358–1367
pubmed: 35707912
doi: 10.1002/aur.2765
Otsuka K, Tsumori M (2020) Analyzing multifunctionality of head movements in face-to-face conversations using deep convolutional neural networks. IEEE Access 8:217169–217195. https://doi.org/10.1109/access.2020.3041672
doi: 10.1109/access.2020.3041672
Zhao Z, Zhu Z, Zhang X, Tang H, Xing J, Hu X, Lu J, Qu X (2022) Identifying autism with head movement features by implementing machine learning algorithms. J Autism Dev Disord 52(7):3038–3049
pubmed: 34250557
doi: 10.1007/s10803-021-05179-2
Kumano S, Otsuka K, Mikami D, Matsuda M, Yamato J (2015) Analyzing interpersonal empathy via collective impressions. IEEE Trans Affect Comput 6(4):324–336
doi: 10.1109/TAFFC.2015.2417561
Tan W, Rong G (2003) A real-time head nod and shake detector using HMMs. Expert Syst Appl 25(3):461–466
doi: 10.1016/S0957-4174(03)00088-5
Happy S, Dasgupta A, George A, Routray A (2012) A video database of human faces under near infra-red illumination for human computer interaction applications. In: 2012 4th International Conference on Intelligent Human Computer Interaction (IHCI), pp. 1–4. IEEE
Savran A, Alyüz N, Dibeklioğlu H, Çeliktutan O, Gökberk B, Sankur B, Akarun L (2008) Bosphorus database for 3D face analysis. European Workshop on Biometrics and Identity Management. Springer, Cham, pp 47–56
doi: 10.1007/978-3-540-89991-4_6
Rajagopalan S, Dhall A, Goecke R (2013) Self-stimulatory behaviours in the wild for autism diagnosis. In: Proceedings of the IEEE International Conference on Computer Vision Workshops, pp. 755–761
Wei P, Ahmedt-Aristizabal D, Gammulle H, Denman S, Armin MA (2022) Vision-based activity recognition in children with autism-related behaviors. Heliyon 9:e16763
doi: 10.1016/j.heliyon.2023.e16763
Negin F, Ozyer B, Agahian S, Kacdioglu S, Ozyer GT (2021) Vision-assisted recognition of stereotype behaviors for early diagnosis of autism spectrum disorders. Neurocomputing 446:145–155
doi: 10.1016/j.neucom.2021.03.004
Krishnappa Babu PR, Di Martino JM, Chang Z, Perochon S, Aiello R, Carpenter KL, Compton S, Davis N, Franz L, Espinosa S et al (2022) Complexity analysis of head movements in autistic toddlers. J Child Psychol Psychiatr. https://doi.org/10.1111/jcpp.13681
doi: 10.1111/jcpp.13681
Anzalone SM, Xavier J, Boucenna S, Billeci L, Narzisi A, Muratori F, Cohen D, Chetouani M (2019) Quantifying patterns of joint attention during human-robot interactions: an application for autism spectrum disorder assessment. Pattern Recogn Lett 118:42–50
doi: 10.1016/j.patrec.2018.03.007
Sani-Bozkurt S, Bozkus-Genc G (2021) Social robots for joint attention development in autism spectrum disorder: a systematic review. Int J Disabil Dev Educ. https://doi.org/10.1080/1034912X.2021.1905153
doi: 10.1080/1034912X.2021.1905153
Washington P, Kline A, Mutlu OC, Leblanc E, Hou C, Stockham N, Paskov K, Chrisman B, Wall D (2021) Activity recognition with moving cameras and few training examples: applications for detection of autism-related headbanging. In: Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems, pp. 1–7
Leo M, Bernava GM, Carcagnì P, Distante C (2022) Video-based automatic baby motion analysis for early neurological disorder diagnosis: state of the art and future directions. Sensors 22(3):866
pubmed: 35161612
pmcid: 8839211
doi: 10.3390/s22030866
Campbell K, Carpenter KL, Hashemi J, Espinosa S, Marsan S, Borg JS, Chang Z, Qiu Q, Vermeer S, Adler E et al (2019) Computer vision analysis captures atypical attention in toddlers with autism. Autism 23(3):619–628
pubmed: 29595333
doi: 10.1177/1362361318766247
Sharma M, Ahmetovic D, Jeni LA, Kitani KM (2018) Recognizing visual signatures of spontaneous head gestures. In: 2018 IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 400–408. IEEE
Fan H, Xiong B, Mangalam K, Li Y, Yan Z, Malik J, Feichtenhofer C (2021) Multiscale vision transformers. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 6824–6835
Arnab A, Dehghani M, Heigold G, Sun C, Lučić M, Schmid C (2021) Vivit: A video vision transformer. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 6836–6846
Dwibedi D, Aytar Y, Tompson J, Sermanet P, Zisserman A (2020) Counting out time: Class agnostic video repetition counting in the wild. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10387–10396
Noble WS (2006) What is a support vector machine? Nat Biotechnol 24(12):1565–1567
pubmed: 17160063
doi: 10.1038/nbt1206-1565
LaValley MP (2008) Logistic regression. Circulation 117(18):2395–2399
pubmed: 18458181
doi: 10.1161/CIRCULATIONAHA.106.682658
Bottou L, Chapelle O, Decoste D, Weston J (2007) Large-scale kernel machines
Maenner MJ, Schieve LA, Rice CE, Cunniff C, Giarelli E, Kirby RS, Lee L-C, Nicholas JS, Wingate MS, Durkin MS (2013) Frequency and pattern of documented diagnostic features and the age of autism identification. J Am Acad Child Adolesc Psychiatr 52(4):401–413
doi: 10.1016/j.jaac.2013.01.014
Anagnostou E, Chaplin W, Watner D, Silverman JM, Smith CJ, Zagursky K, Kryzak LA, Corwin TE, Feirsen N, Tanel N et al (2011) Factor analysis of repetitive behaviors in autism as measured by the y-bocs. J Neuropsychiatry Clin Neurosci 23(3):332–339
pubmed: 21948895
doi: 10.1176/jnp.23.3.jnp332
Grossi E, Caminada E, Goffredo M, Vescovo B, Castrignano T, Piscitelli D, Valagussa G, Franceschini M, Vanzulli F (2021) Patterns of restricted and repetitive behaviors in autism spectrum disorders: a cross-sectional video recording study. Preliminary report. Brain Sci 11(6):678
pubmed: 34064245
pmcid: 8224357
doi: 10.3390/brainsci11060678
Leekam SR, Prior MR, Uljarevic M (2011) Restricted and repetitive behaviors in autism spectrum disorders: a review of research in the last decade. Psychol Bull 137(4):562
pubmed: 21574682
doi: 10.1037/a0023341
South M, Ozonoff S, McMahon WM (2005) Repetitive behavior profiles in asperger syndrome and high-functioning autism. J Autism Dev Disord 35(2):145–158
pubmed: 15909401
doi: 10.1007/s10803-004-1992-8
Lugaresi C, Tang J, Nash H, McClanahan C, Uboweja E, Hays M, Zhang F, Chang CL, Yong MG, Lee J, et al. (2019) Mediapipe: A framework for building perception pipelines
Palais B, Palais R, Rodi S (2009) A disorienting look at euler’s theorem on the axis of a rotation. Am Math Mon 116(10):892–909
doi: 10.4169/000298909X477014
Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser Ł, Polosukhin I (2017) Attention is all you need, vol. 30
Zhuang F, Qi Z, Duan K, Xi D, Zhu Y, Zhu H, Xiong H, He Q (2020) A comprehensive survey on transfer learning. Proc IEEE 109(1):43–76
doi: 10.1109/JPROC.2020.3004555
Zhang J, Li W, Ogunbona P, Xu D (2019) Recent advances in transfer learning for cross-dataset visual recognition: A problem-oriented perspective. ACM Computing Surveys (CSUR) 52(1):1–38
doi: 10.1145/3158369
Sharif Razavian A, Azizpour H, Sullivan J, Carlsson S (2014) Cnn features off-the-shelf: an astounding baseline for recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 806–813
Wang M, Deng W (2018) Deep visual domain adaptation: a survey. Neurocomputing 312:135–153
doi: 10.1016/j.neucom.2018.05.083
Weiss K, Khoshgoftaar TM, Wang D (2016) A survey of transfer learning. J Big Data 3(1):1–40
doi: 10.1186/s40537-016-0043-6
Tan C, Sun F, Kong T, Zhang W, Yang C, Liu C (2018) A survey on deep transfer learning. International Conference on Artificial Neural Networks. Springer, Cham, pp 270–279
Mohottala S, Samarasinghe P, Kasthurirathna D, Abhayaratne C (2022) Graph neural network based child activity recognition. 2022 IEEE International Conference on Industrial Technology (ICIT). https://doi.org/10.1109/ICIT48603.2022.10002799
doi: 10.1109/ICIT48603.2022.10002799
Liu F, Xu X, Qiu S, Qing C, Tao D (2015) Simple to complex transfer learning for action recognition. IEEE Trans Image Process 25(2):949–960
pubmed: 26841395
doi: 10.1109/TIP.2015.2512107
Sargano AB, Wang X, Angelov P, Habib Z (2017) Human action recognition using transfer learning with deep representations. In 2017 International Joint Conference on Neural Networks (IJCNN), 463–469. IEEE
Wang T, Chen Y, Zhang M, Chen J, Snoussi H (2017) Internal transfer learning for improving performance in human action recognition for small datasets. IEEE Access 5:17627–17633
doi: 10.1109/ACCESS.2017.2746095
Szatmari P, Georgiades S, Bryson S, Zwaigenbaum L, Roberts W, Mahoney W, Goldberg J, Tuff L (2006) Investigating the structure of the restricted, repetitive behaviours and interests domain of autism. J Child Psychol Psychiatr 47(6):582–590
doi: 10.1111/j.1469-7610.2005.01537.x
Al Banna M, Ghosh T, Taher KA, Kaiser MS, Mahmud M et al (2020) A monitoring system for patients of autism spectrum disorder using artificial intelligence. In: Banna A (ed) International Conference on Brain Informatics 251–262. Springer
Choudhury A, et al. (2018) Prognosticating autism spectrum disorder using artificial neural network: Levenberg-marquardt algorithm
Song D-Y, Kim SY, Bong G, Kim JM, Yoo HJ (2019) The use of artificial intelligence in screening and diagnosis of autism spectrum disorder: a literature review. J Kor Acad Child Adolesc Psychiatr 30(4):145
doi: 10.5765/jkacap.190027
Wall DP, Dally R, Luyster R, Jung J-Y, DeLuca TF (2012) Use of artificial intelligence to shorten the behavioral diagnosis of autism. Plos ONE. https://doi.org/10.1371/journal.pone.0043855
doi: 10.1371/journal.pone.0043855
pubmed: 23300743
pmcid: 3530443
Kameda T, Weiner P (1970) On the state minimization of nondeterministic finite automata. IEEE Trans Comput 100(7):617–627
doi: 10.1109/T-C.1970.222994
Chang L, Toner BB, Fukudo S, Guthrie E, Locke GR, Norton NJ, Sperber AD (2006) Gender, age, society, culture, and the patient’s perspective in the functional gastrointestinal disorders. Gastroenterology 130(5):1435–1446
pubmed: 16678557
doi: 10.1053/j.gastro.2005.09.071
Afifi M (2007) Gender differences in mental health. Singapore Med J 48(5):385
pubmed: 17453094
Locker D, Poulton R, Thomson W (2001) Psychological disorders and dental anxiety in a young adult population. Commun Dent Oral Epidemiol 29(6):456–463
doi: 10.1034/j.1600-0528.2001.290607.x
Piccinelli M, Wilkinson G (2000) Gender differences in depression: critical review. Br J Psychiatry 177(6):486–492
pubmed: 11102321
doi: 10.1192/bjp.177.6.486
Association AP, et al. (2015) Depressive disorders: Dsm-5® selections
Quan H, Sundararajan V, Halfon P, Fong A, Burnand B, Luthi J-C, Saunders LD, Beck CA, Feasby TE, Ghali WA (2005) Coding algorithms for defining comorbidities in ICD-9-CM and ICD-10 administrative data. Med Care 43:1130–1139
pubmed: 16224307
doi: 10.1097/01.mlr.0000182534.19832.83
Regier DA, Kuhl EA, Kupfer DJ (2013) The DSM-5: classification and criteria changes. World Psychiatr 12(2):92–98
doi: 10.1002/wps.20050
Trott P (1977) International classification of diseases for oncology. J Clin Pathol 30(8):782
pmcid: 476557
doi: 10.1136/jcp.30.8.782-c
Rajagopalan S, Dhall A, Goecke R (2013) Self-stimulatory behaviours in the wild for autism diagnosis. In: Proceedings of the IEEE International Conference on Computer Vision Workshops 755–761
Di Zio M, Fursova N, Gelsema T, Gießing S, Guarnera U, Petrauskienė J, Quensel-von Kalben L, Scanu M, ten Bosch K, van der Loo M, et al. (2016) Methodology for data validation 1.0
Eren L, Ince T, Kiranyaz S (2019) A generic intelligent bearing fault diagnosis system using compact adaptive 1D cnn classifier. J Signal Process Syst 91(2):179–189
doi: 10.1007/s11265-018-1378-3
Zhao J, Mao X, Chen L (2019) Speech emotion recognition using deep 1D & 2D cnn lstm networks. Biomed Signal Process Control 47:312–323
doi: 10.1016/j.bspc.2018.08.035