Machine learning classification of autism spectrum disorder based on reciprocity in naturalistic social interactions.
Journal
Translational psychiatry
ISSN: 2158-3188
Titre abrégé: Transl Psychiatry
Pays: United States
ID NLM: 101562664
Informations de publication
Date de publication:
03 Feb 2024
03 Feb 2024
Historique:
received:
13
01
2023
accepted:
23
01
2024
revised:
19
01
2024
medline:
4
2
2024
pubmed:
4
2
2024
entrez:
3
2
2024
Statut:
epublish
Résumé
Autism spectrum disorder is characterized by impaired social communication and interaction. As a neurodevelopmental disorder typically diagnosed during childhood, diagnosis in adulthood is preceded by a resource-heavy clinical assessment period. The ongoing developments in digital phenotyping give rise to novel opportunities within the screening and diagnostic process. Our aim was to quantify multiple non-verbal social interaction characteristics in autism and build diagnostic classification models independent of clinical ratings. We analyzed videos of naturalistic social interactions in a sample including 28 autistic and 60 non-autistic adults paired in dyads and engaging in two conversational tasks. We used existing open-source computer vision algorithms for objective annotation to extract information based on the synchrony of movement and facial expression. These were subsequently used as features in a support vector machine learning model to predict whether an individual was part of an autistic or non-autistic interaction dyad. The two prediction models based on reciprocal adaptation in facial movements, as well as individual amounts of head and body motion and facial expressiveness showed the highest precision (balanced accuracies: 79.5% and 68.8%, respectively), followed by models based on reciprocal coordination of head (balanced accuracy: 62.1%) and body (balanced accuracy: 56.7%) motion, as well as intrapersonal coordination processes (balanced accuracy: 44.2%). Combinations of these models did not increase overall predictive performance. Our work highlights the distinctive nature of non-verbal behavior in autism and its utility for digital phenotyping-based classification. Future research needs to both explore the performance of different prediction algorithms to reveal underlying mechanisms and interactions, as well as investigate the prospective generalizability and robustness of these algorithms in routine clinical care.
Identifiants
pubmed: 38310111
doi: 10.1038/s41398-024-02802-5
pii: 10.1038/s41398-024-02802-5
doi:
Types de publication
Journal Article
Langues
eng
Sous-ensembles de citation
IM
Pagination
76Subventions
Organisme : Deutsche Forschungsgemeinschaft (German Research Foundation)
ID : 876/3-1
Organisme : Deutsche Forschungsgemeinschaft (German Research Foundation)
ID : FA 876/5-1
Informations de copyright
© 2024. The Author(s).
Références
American Psychiatric Association. Diagnostic and statistical manual of mental disorders (DSM-5®). 5th ed. Arlington, VA: American Psychiatric Pub; 2013.
Matson JL, Kozlowski AM. The increasing prevalence of autism spectrum disorders. Res Autism Spectr Disord. 2011;5:418–25.
Zwaigenbaum L, Penner M. Autism spectrum disorder: advances in diagnosis and evaluation. BMJ. 2018;361:k1674.
AWMF. Autismus-Spektrum-Störungen im Kindes-, Jugend- und Erwachsenenalter, Teil 1: Diagnostik: Interdisziplinäre S3-Leitlinie der DGKJP und der DGPPN sowie der beteiligten Fachgesellschaften, Berufsverbände und Patientenorganisationen. 2016. https://www.awmf.org/uploads/tx_szleitlinien/028-018l_S3_Autismus-Spektrum-Stoerungen_ASS-Diagnostik_2016-05.pdf .
Autistica. Your questions: shaping future autism research. 2016. https://www.autistica.org.uk/downloads/files/Autism-Top-10-Your-Priorities-for-Autism-Research.pdf .
Tariq Q, Daniels J, Schwartz JN, Washington P, Kalantarian H, Wall DP. Mobile detection of autism through machine learning on home video: A development and prospective validation study. PLoS Med. 2018;15:1–20.
Drimalla H, Scheffer T, Landwehr N, Baskow I, Roepke S, Behnoush B, et al. Towards the automatic detection of social biomarkers in autism spectrum disorder: introducing the simulated interaction task (SIT). NPJ Digit Med. 2020;3:1–10. https://www.nature.com/articles/s41746-020-0227-5?utm_source=researcher_app&utm_medium=referral&utm_campaign=RESR_MRKT_Researcher_inbound .
Robles M, Namdarian N, Otto J, Wassiljew E, Navab N, Falter-Wagner C, et al. A virtual reality based system for the screening and classification of autism. IEEE Trans Vis Comput Graph. 2022;28:2168–78.
pubmed: 35171773
Redcay E, Dodell-Feder D, Mavros PL, Kleiner M, Pearrow MJ, Triantafyllou C, et al. Atypical brain activation patterns during a face-to-face joint attention game in adults with autism spectrum disorder. 2013;34:2511–23. https://onlinelibrary.wiley.com/doi/abs/10.1002/hbm.22086 .
Küpper C, Stroth S, Wolff N, Hauck F, Kliewer N, Schad-Hansjosten T, et al. Identifying predictive features of autism spectrum disorders in a clinical sample of adolescents and adults using machine learning. Sci Rep. 2020;10:1–11.
Kosmicki JA, Sochat V, Duda M, Wall DP. Searching for a minimal set of behaviors for autism detection through feature selection-based machine learning. Transl Psychiatry. 2015;5:e514–e514.
Morrison KE, DeBrabander KM, Jones DR, Faso DJ, Ackerman RA, Sasson NJ. Outcomes of real-world social interaction for autistic adults paired with autistic compared to typically developing partners. Autism. 2019;1362361319892701. https://doi.org/10.1177/1362361319892701 .
de Belen RAJ, Bednarz T, Sowmya A, Del Favero D. Computer vision in autism spectrum disorder research: a systematic review of published studies from 2009 to 2019. Transl Psychiatry. 2020;10:333.
Vabalas A, Gowen E, Poliakoff E, Casson AJ. Applying machine learning to kinematic and eye movement features of a movement imitation task to predict autism diagnosis. Sci Rep. 2020;10:1–13.
Kojovic N, Natraj S, Mohanty SP, Maillart T, Schaer M. Using 2D video-based pose estimation for automated prediction of autism spectrum disorders in young children. Sci Rep. 2021;11:1–10.
Koehne S, Hatri A, Cacioppo JT, Dziobek I. Perceived interpersonal synchrony increases empathy: Insights from autism spectrum disorder. Cognition. 2016;146:8–15. https://doi.org/10.1016/j.cognition.2015.09.007 .
pubmed: 26398860
Hove MJ, Risen JL. It’s all in the timing: Interpersonal synchrony increases affiliation. Soc Cogn. 2009;27:949–60.
Miles LK, Nind LK, Macrae CN. The rhythm of rapport: Interpersonal synchrony and social perception. J Exp Soc Psychol. 2009;45:585–9. May
McNaughton KA, Redcay E. Interpersonal synchrony in autism. Curr Psychiatry Rep. 2020;22:12 https://doi.org/10.1007/s11920-020-1135-8 .
pubmed: 32025922
Zampella CJ, Bennetto L, Herrington JD. Computer vision analysis of reduced interpersonal affect coordination in youth with autism spectrum disorder. Autism Res. 2020;13:2133–42.
pubmed: 32666690
pmcid: 7748996
Koehler JC, Georgescu AL, Weiske J, Spangemacher M, Burghof L, Falkai P, et al. Brief report: specificity of interpersonal synchrony deficits to autism spectrum disorder and its potential for digitally assisted diagnostics. J Autism Dev Disord. 2021. https://doi.org/10.1007/s10803-021-05194-3 .
Georgescu AL, Koeroglu S, De AF, Vogeley K, Falter-Wagner CM, Tschacher W. Reduced nonverbal interpersonal synchrony in autism spectrum disorder independent of partner diagnosis: a motion energy study. Mol Autism. 2020;11:1–14.
McAuliffe D, Pillai AS, Tiedemann A, Mostofsky SH, Ewen JB. Dyspraxia in ASD: impaired coordination of movement elements. Autism Res. 2017;10:648–52.
pubmed: 27653620
Georgescu AL, Koehler JC, Weiske J, Vogeley K, Koutsouleris N, Falter-Wagner C. Machine learning to study social interaction difficulties in ASD. Front Robot AI. 2019;6:1–7.
de Marchena A, Eigsti IM. Conversational gestures in autism spectrum disorders: asynchrony but not decreased frequency. Autism Res. 2010;3:311–22. 2010/12/25
pubmed: 21182208
Stagg SD, Slavny R, Hand C, Cardoso A, Smith P. Does facial expressivity count? How typically developing children respond initially to children with autism. Autism. 2014;18:704–11.
pubmed: 24121180
Cook JL, Blakemore SJ, Press C. Atypical basic movement kinematics in autism spectrum conditions. Brain. 2013;136:2816–24.
pubmed: 23983031
pmcid: 4017873
Wang LAL, Petrulla V, Zampella CJ, Waller R, Schultz RT. Gross motor impairment and its relation to social skills in autism spectrum disorder: a systematic review and two meta-analyses. Psychol Bull. 2022;148:273–300.
pubmed: 35511567
pmcid: 9894569
Redcay E, Schilbach L. Using second-person neuroscience to elucidate the mechanisms of social interaction. Nat Rev Neurosci. 2019;20:495–505. https://doi.org/10.1038/s41583-019-0179-4 .
pubmed: 31138910
pmcid: 6997943
Schuwerk T, Vuori M, Sodian B. Implicit and explicit Theory of Mind reasoning in autism spectrum disorders: the impact of experience. Autism. 2015;19:459–68.
pubmed: 24627427
Peirce J, Gray JR, Simpson S, MacAskill M, Höchenberger R, Sogo H, et al. PsychoPy2: experiments in behavior made easy. Behav Res Methods. 2019;51:195–203.
pubmed: 30734206
pmcid: 6420413
Ramseyer FT. Motion energy analysis (MEA): A primer on the assessment of motion from video. J Couns Psychol. 2020;67:536–49.
pubmed: 32614233
Tschacher W, Rees GM, Ramseyer F. Nonverbal synchrony and affect in dyadic interactions. 2014;5. https://doi.org/10.3389/fpsyg.2014.01323 .
Nadig A, Lee I, Singh L, Bosshart K, Ozonoff S. How does the topic of conversation affect verbal exchange and eye gaze? A comparison between typical development and high-functioning autism. Neuropsychologia. 2010;48:2730–9.
pubmed: 20493890
pmcid: 2935904
Baron-Cohen S, Wheelwright S, Skinner R, Martin J, Clubley E. The autism-spectrum quotient (AQ): evidence from asperger syndrome/high-functioning autism, malesand females, scientists and mathematicians. J Autism Dev Disord. 2001;31:5–17.
pubmed: 11439754
Paulus C. DER SAARBRÜCKER PERSÖNLICHKEITSFRAGEBOGEN SPF(IRI) ZUR MESSUNG VON EMPATHIE: Psychometrische Evaluation der deutschen Version des Interpersonal Reactivity Index. https://doi.org/10.23668/psycharchives.9249 .
Davis MH. A multidimensional approach to individual differences in empathy. JSAS Catalog of Selected Documents in Psychology. 1980;10:85.
Bagby RM, Parker JDA, Taylor GJ. The twenty-item Toronto Alexithymia scale. Item selection and cross-validation of the factor structure. J Psychosom Res. 1994;38:23–32. https://doi.org/10.1016/0022-3999(94)90005-1 .
pubmed: 8126686
Hautzinger M, Bailer M, Worall H, Keller F. BECK-DEPRESSIONS-INVENTAR - Beck Depression Inventory (BDI; Beck AT, Ward CH, Mendelson M, MockJ, Erbaugh J, 1961) - German version [Internet]. Prof. Dr. Martin Hautzinger bei PsychAuthors: https://www.psychauthors.de/psychauthors/index.php?wahl=forschung&uwahl=psychauthors&uuwahl=p00492MH 1994.
Graf A. Eine deutschsprachige Version der Self-Monitoring-Skala. Z fur Arb- und Organisationspsychologie. 2004;48:109–21.
Kirby A, Edwards L, Sugden D, Rosenblum S. The development and standardization of the Adult Developmental Co-ordination Disorders/Dyspraxia Checklist (ADC). Res Dev Disabil. 2010;31:131–9. http://www.sciencedirect.com/science/article/pii/S0891422209001437 .
pubmed: 19819107
Lehrl S, Triebig G, Fischer BANS. Multiple choice vocabulary test MWT as a valid and short test to estimate premorbid intelligence. Acta Neurologica Scandinavica. 1995;91:335–345.
Weiß RH. CFT 20-R: grundintelligenztest skala 2-revision. Hogrefe; 2006. Göttingen.
Drimalla H, Baskow I, Behnia B, Roepke S, Dziobek I. Imitation and recognition of facial emotions in autism: a computer vision approach. Molecular autism. 2021;12:1–15.
Baltrušaitis T, Zadeh A, Lim YC, Morency LP. Openface 2.0: facial behavior analysis toolkit. In: 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018). Xi’an, China: IEEE; 2018. pp. 59–66.
Schoenherr D, Paulick J, Worrack S, Strauss BM, Rubel JA, Schwartz B, et al. Quantification of nonverbal synchrony using linear time series analysis methods: Lack of convergent validity and evidence for facets of synchrony. Behav Res Methods. 2019;51:361–83.
pubmed: 30298266
Wolpert DH. Stacked generalization. Neural networks. 1992;5:241–259.
Koutsouleris, Vetter & Wiegand (2023). Neurominer [Computer software]. Retrieved from https://github.com/neurominer-git/NeuroMiner_1.2 .
The MathWorks Inc. MATLAB version: 9.13.0 (R2022b), Natick, Massachusetts: The MathWorks Inc. https://www.mathworks.com .
Golland P, Fischl B. Permutation Tests for Classification: Towards Statistical Significance in Image-Based Studies. In: Taylor C, Noble JA, editors. Information Processing in Medical Imaging. IPMI 2003. Lecture Notes in Computer Science, vol 2732. Berlin, Heidelberg: Springer.
Benjamin DJ, Berger JO, Johannesson M, Nosek BA, Wagenmakers EJ, Berk R, et al. Redefine statistical significance. Nat Hum Behav. 2018;2:6–10.
pubmed: 30980045
Lakens D, Adolfi FG, Albers CJ, Anvari F, Apps MAJ, Argamon SE, et al. Justify your alpha. Nat Hum Behav. 2018;2:168–71.
Wolpert DH. Stacked generalization. Neural Netw. 1992;5:241–59. https://www.sciencedirect.com/science/article/pii/S0893608005800231 .
Yoshimura S, Sato W, Uono S, Toichi M. Impaired overt facial mimicry in response to dynamic facial expressions in high-functioning autism spectrum disorders. J Autism Dev Disord. 2015;45:1318–28.
pubmed: 25374131
McIntosh DN, Reichmann-Decker A, Winkielman P, Wilbarger JL. When the social mirror breaks: deficits in automatic, but not voluntary, mimicry of emotional facial expressions in autism. Dev Sci. 2006;9:295–302. 2006/05/04
pubmed: 16669800
Drimalla H, Baskow I, Behnia B, Roepke S, Dziobek I. Imitation and recognition of facial emotions in autism: a computer vision approach. Mol Autism. 2021;12:1–15.
Zhao Z, Zhu Z, Zhang X, Tang H, Xing J, Hu X, et al. Atypical Head Movement during face-to-face interaction in children with autism spectrum disorder. Autism Res. 2021;1–12.
Bloch C, Vogeley K, Georgescu AL, Falter-Wagner CM. INTRApersonal synchrony as constituent of INTERpersonal synchrony and its relevance for autism spectrum disorder. Front Robot AI. 2019;6:1–8.
Sagi O, Rokach L. Ensemble learning: A survey. WIREs Data Mining Knowl Discov. 2018;8:e1249.
Liao M, Duan H, Wang G. Application of machine learning techniques to detect the children with autism spectrum disorder. J Healthc Eng. 2022;2022:9340027.
Dwyer D, Krishnadas R. Five points to consider when reading a translational machine-learning paper. Br J Psychiatry. 2022;220:169–71.
Leo M, Carcagnì P, Distante C, Spagnolo P, Mazzeo PL, Rosato AC, et al. Computational assessment of facial expression production in ASD children. Sensors. 2018;18:1–25.
Koehler JC, Falter-Wagner CM. Digitally assisted diagnostics of autism spectrum disorder. Front Psychiatry. 2023;14,1066284.
Riley RD, Ensor J, Snell KIE, Debray TPA, Altman DG, Moons KGM, et al. External validation of clinical prediction models using big datasets from e-health records or IPD meta-analysis: Opportunities and challenges. BMJ. 2016;353:i3140, https://doi.org/10.1136/bmj.i3140 .
Milton DEM. On the ontological status of autism: the ‘double empathy problem’. Disabil Soc. 2012;27:883–7. https://doi.org/10.1080/09687599.2012.710008 .
Bolis D, Balsters J, Wenderoth N, Becchio C, Schilbach L. Beyond autism: introducing the dialectical misattunement hypothesis and a bayesian account of intersubjectivity. Psychopathology. 2018;50:355–72.
Baltrušaitis T, Robinson P, Morency LP. OpenFace: an open source facial behavior analysis toolkit. 2016. https://www.omron.com/ecb/products/mobile/ .
Dwyer DB, Falkai P, Koutsouleris N. Machine learning approaches for clinical psychology and psychiatry. Annu Rev Clin Psychol. 2018;14:91–118. 2018/02/06.
pubmed: 29401044
Benjamini Y, Hochberg Y. Controlling the false discovery rate: a practical and powerful approach to multiple testing. J R Stat Soc. Ser B (Methodological). 1995;57:289–300.