Using machine learning to improve diagnostic assessment of ASD in the light of specific differential and co-occurring diagnoses.
Autism spectrum disorders
Machine learning
diagnosis
Journal
Journal of child psychology and psychiatry, and allied disciplines
ISSN: 1469-7610
Titre abrégé: J Child Psychol Psychiatry
Pays: England
ID NLM: 0375361
Informations de publication
Date de publication:
01 2023
01 2023
Historique:
accepted:
08
05
2022
pubmed:
2
7
2022
medline:
17
12
2022
entrez:
1
7
2022
Statut:
ppublish
Résumé
Diagnostic assessment of ASD requires substantial clinical experience and is particularly difficult in the context of other disorders with behavioral symptoms in the domain of social interaction and communication. Observation measures such as the Autism Diagnostic Observation Schedule (ADOS) do not take into account such co-occurring disorders. We used a well-characterized clinical sample of individuals (n = 1,251) that had received detailed outpatient evaluation for the presence of an ASD diagnosis (n = 481) and covered a range of additional overlapping diagnoses, including anxiety-related disorders (ANX, n = 122), ADHD (n = 439), and conduct disorder (CD, n = 194). We focused on ADOS module 3, covering the age range with particular high prevalence of such differential diagnoses. We used machine learning (ML) and trained random forest models on ADOS single item scores to predict a clinical best-estimate diagnosis of ASD in the context of these differential diagnoses (ASD vs. ANX, ASD vs. ADHD, ASD vs. CD), in the context of co-occurring ADHD, and an unspecific model using all available data. We employed nested cross-validation for an unbiased estimate of classification performance and made available a Webapp to showcase the results and feasibility for translation into clinical practice. We obtained very good overall sensitivity (0.89-0.94) and specificity (0.87-0.89). In particular for individuals with less severe symptoms, our models showed increases of up to 35% in sensitivity or specificity. Furthermore, we analyzed item importance profiles of the ANX, ADHD, and CD models in comparison with the unspecific model revealing distinct patterns of importance for specific ADOS items with respect to differential diagnoses. ML-based diagnostic classification may improve clinical decisions by utilizing the full range of information from detailed diagnostic observation instruments such as the ADOS. Importantly, this strategy might be of particular relevance for older children with less severe symptoms for whom the diagnostic decision is often particularly difficult.
Sections du résumé
BACKGROUND
Diagnostic assessment of ASD requires substantial clinical experience and is particularly difficult in the context of other disorders with behavioral symptoms in the domain of social interaction and communication. Observation measures such as the Autism Diagnostic Observation Schedule (ADOS) do not take into account such co-occurring disorders.
METHOD
We used a well-characterized clinical sample of individuals (n = 1,251) that had received detailed outpatient evaluation for the presence of an ASD diagnosis (n = 481) and covered a range of additional overlapping diagnoses, including anxiety-related disorders (ANX, n = 122), ADHD (n = 439), and conduct disorder (CD, n = 194). We focused on ADOS module 3, covering the age range with particular high prevalence of such differential diagnoses. We used machine learning (ML) and trained random forest models on ADOS single item scores to predict a clinical best-estimate diagnosis of ASD in the context of these differential diagnoses (ASD vs. ANX, ASD vs. ADHD, ASD vs. CD), in the context of co-occurring ADHD, and an unspecific model using all available data. We employed nested cross-validation for an unbiased estimate of classification performance and made available a Webapp to showcase the results and feasibility for translation into clinical practice.
RESULTS
We obtained very good overall sensitivity (0.89-0.94) and specificity (0.87-0.89). In particular for individuals with less severe symptoms, our models showed increases of up to 35% in sensitivity or specificity. Furthermore, we analyzed item importance profiles of the ANX, ADHD, and CD models in comparison with the unspecific model revealing distinct patterns of importance for specific ADOS items with respect to differential diagnoses.
CONCLUSIONS
ML-based diagnostic classification may improve clinical decisions by utilizing the full range of information from detailed diagnostic observation instruments such as the ADOS. Importantly, this strategy might be of particular relevance for older children with less severe symptoms for whom the diagnostic decision is often particularly difficult.
Types de publication
Journal Article
Research Support, Non-U.S. Gov't
Langues
eng
Sous-ensembles de citation
IM
Pagination
16-26Commentaires et corrections
Type : CommentIn
Informations de copyright
© 2022 The Authors. Journal of Child Psychology and Psychiatry published by John Wiley & Sons Ltd on behalf of Association for Child and Adolescent Mental Health.
Références
Abbas, H., Garberson, F., Glover, E., & Wall, D.P. (2018). Machine learning approach for early detection of autism by combining questionnaire and home video screening. Journal of the American Medical Informatics Association, 25, 1000-1007.
American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (DSM-5®) (5th edn). Washington, DC: American Psychiatric Association.
AWMF. (2016). Autismus-Spektrum-Störungen im Kindes-, Jugend- und Erwachsenenalter Teil 1: Diagnostik. Interdisziplinäre S3-Leitlinie der DGKJP und der DGPPN sowie der beteiligten Fachgesellschaften, Berufsverbände und Patientenorganisationen.
Baxter, A.J., Brugha, T.S., Erskine, H.E., Scheurer, R.W., Vos, T., & Scott, J.G. (2015). The epidemiology and global burden of autism spectrum disorders. Psychological Medicine, 45, 601-613.
Bone, D., Bishop, S.L., Black, M.P., Goodwin, M.S., Lord, C., & Narayanan, S.S. (2016). Use of machine learning to improve autism screening and diagnostic instruments: Effectiveness, efficiency, and multi-instrument fusion. Journal of Child Psychology and Psychiatry, 57, 927-937.
Bone, D., Goodwin, M.S., Black, M.P., Lee, C.-C., Audhkhasi, K., & Narayanan, S. (2015). Applying Machine Learning to facilitate autism diagnostics: Pitfalls and promises. Journal of Autism and Developmental Disorders, 45, 1121-1136.
Breiman, L. (2001). Random forests. Machine Learning, 45, 5-32.
Budman, I., Meiri, G., Ilan, M., Faroy, M., Langer, A., Reboh, D., … & Dinstein, I. (2019). Quantifying the social symptoms of autism using motion capture. Scientific Reports, 9, 7712.
Chong, E., Chanda, K., Ye, Z., Southerland, A., Ruiz, N., Jones, R. M., … & Rehg, J. M. (2019). Detecting gaze towards eyes in natural social interactions and its use in child assessment. ArXiv:1902.00607 [Cs]. http://arxiv.org/abs/1902.00607
Davidovitch, M., Levit-Binnun, N., Golan, D., & Manning-Courtney, P. (2015). Late diagnosis of autism spectrum disorder after initial negative assessment by a Multidisciplinary Team. Journal of Developmental & Behavioral Pediatrics, 36, 227-234.
Drimalla, H., Scheffer, T., Landwehr, N., Baskow, I., Roepke, S., Behnia, B., & Dziobek, I. (2020). Towards the automatic detection of social biomarkers in autism spectrum disorder: Introducing the simulated interaction task (SIT). NPJ Digital Medicine, 3, 25.
Duda, M., Daniels, J., & Wall, D.P. (2016). Clinical evaluation of a novel and mobile autism risk assessment. Journal of Autism and Developmental Disorders, 46, 1953-1961.
Duda, M., Kosmicki, J.A., & Wall, D.P. (2014). Testing the accuracy of an observation-based classifier for rapid detection of autism risk. Translational Psychiatry, 4, e424.
Duda, M., Ma, R., Haber, N., & Wall, D.P. (2016). Use of machine learning for behavioral distinction of autism and ADHD. Translational Psychiatry, 6, e732.
Falkmer, T., Anderson, K., Falkmer, M., & Horlin, C. (2013). Diagnostic procedures in autism spectrum disorders: A systematic literature review. European Child & Adolescent Psychiatry, 22, 329-340.
Frässle, S., Yao, Y., Schöbi, D., Aponte, E.A., Heinzle, J., & Stephan, K.E. (2018). Generative models for clinical applications in computational psychiatry. WIREs Cognitive Science, 9, e1460.
Frenette, P., Dodds, L., MacPherson, K., Flowerdew, G., Hennen, B., & Bryson, S. (2013). Factors affecting the age at diagnosis of autism spectrum disorders in Nova Scotia, Canada. Autism, 17, 184-195.
Fusaro, V.A., Daniels, J., Duda, M., DeLuca, T.F., D'Angelo, O., Tamburello, J., … & Wall, D.P. (2014). The potential of accelerating early detection of autism through content analysis of YouTube videos. PLoS ONE, 9, e93533.
Gilmour, J., Hill, B., Place, M., & Skuse, D.H. (2004). Social communication deficits in conduct disorder: A clinical and community survey. Journal of Child Psychology and Psychiatry, 45, 967-978.
Grzadzinski, R., Di Martino, A., Brady, E., Mairena, M.A., O'Neale, M., Petkova, E., … & Castellanos, F.X. (2011). Examining autistic traits in children with ADHD: Does the autism Spectrum extend to ADHD? Journal of Autism and Developmental Disorders, 41, 1178-1191.
Grzadzinski, R., Dick, C., Lord, C., & Bishop, S. (2016). Parent-reported and clinician-observed autism spectrum disorder (ASD) symptoms in children with attention deficit/hyperactivity disorder (ADHD): Implications for practice under DSM-5. Molecular Autism, 7, 7.
Hartley, S.L., & Sikora, D.M. (2009). Which DSM-IV-TR criteria best differentiate high-functioning autism spectrum disorder from ADHD and anxiety disorders in older children? Autism, 13, 485-509.
Hartz, A., Guth, B., Jording, M., Vogeley, K., & Schulte-Rüther, M. (2021). Temporal behavioral parameters of on-going gaze encounters in a virtual environment. Frontiers in Psychology, 12, 982.
Hossain, M.M., Khan, N., Sultana, A., Ma, P., McKyer, E.L.J., Ahmed, H.U., & Purohit, N. (2020). Prevalence of comorbid psychiatric disorders among people with autism spectrum disorder: An umbrella review of systematic reviews and meta-analyses. Psychiatry Research, 287, 922.
Hyde, K.K., Novack, M.N., LaHaye, N., Parlett-Pelleriti, C., Anden, R., Dixon, D.R., & Linstead, E. (2019). Applications of supervised Machine Learning in autism Spectrum disorder research: A review. Review Journal of Autism and Developmental Disorders, 6, 128-146.
Kamp-Becker, I., Albertowski, K., Becker, J., Ghahreman, M., Langmann, A., Mingebach, T., … & Stroth, S. (2018). Diagnostic accuracy of the ADOS and ADOS-2 in clinical practice. European Child & Adolescent Psychiatry, 27, 1193-1207.
Kamp-Becker, I., Poustka, L., Bachmann, C., Ehrlich, S., Hoffmann, F., Kanske, P., … & Wermter, A.-K. (2017). Study protocol of the ASD-Net, the German research consortium for the study of Autism Spectrum Disorder across the lifespan: From a better etiological understanding, through valid diagnosis, to more effective health care. BMC Psychiatry, 17, 206.
Klapwijk, E.T., Aghajani, M., Colins, O.F., Marijnissen, G.M., Popma, A., Lang, N.D.J., … & Vermeiren, R.R.J.M. (2016). Different brain responses during empathy in autism spectrum disorders versus conduct disorder and callous-unemotional traits. Journal of Child Psychology and Psychiatry, 57, 737-747.
Kosmicki, J.A., Sochat, V., Duda, M., & Wall, D.P. (2015). Searching for a minimal set of behaviors for autism detection through feature selection-based machine learning. Translational Psychiatry, 5, e514.
Kruppa, J.A., Reindl, V., Gerloff, C., Oberwelland Weiss, E., Prinz, J., Herpertz-Dahlmann, B., … & Schulte-Rüther, M. (2020). Interpersonal synchrony special issue brain and motor synchrony in children and adolescents with ASD-a fNIRS hyperscanning study. Social Cognitive and Affective Neuroscience, 16, 103-116. https://doi.org/10.1093/scan/nsaa092
Küpper, C., Stroth, S., Wolff, N., Hauck, F., Kliewer, N., Schad-Hansjosten, T., … & Roepke, S. (2020). Identifying predictive features of autism spectrum disorders in a clinical sample of adolescents and adults using machine learning. Scientific Reports, 10, 4805.
Levy, S., Duda, M., Haber, N., & Wall, D.P. (2017). Sparsifying machine learning models identify stable subsets of predictive features for behavioral detection of autism. Molecular Autism, 8, 65.
Lord, C., Rutter, M., DiLavore, P., Risi, S., Gotham, K., & Bishop, S. (2012). Autism diagnostic observation schedule-2nd edition (ADOS-2) (p. 284). Los Angeles: Western Psychological Corporation.
Lord, C., Rutter, M., & Le Couteur, A. (1994). Autism Diagnostic Interview-Revised: A revised version of a diagnostic interview for caregivers of individuals with possible pervasive developmental disorders. Journal of Autism and Developmental Disorders, 24, 659-685.
Milledge, S.V., Cortese, S., Thompson, M., McEwan, F., Rolt, M., Meyer, B., … & Eisenbarth, H. (2019). Peer relationships and prosocial behaviour differences across disruptive behaviours. European Child & Adolescent Psychiatry, 28, 781-793.
Poustka, L., Rühl, D., Feineis-matthews, S., Bölte, S., Poustka, F., & Hartung, M. (2015). ADOS-2 Diagnostische Beobachtungsskale für Autistische Störungen-2. Hogrefe AG: Verlag Hans Huber.
Roessner, V., Rothe, J., Kohls, G., Schomerus, G., Ehrlich, S., & Beste, C. (2021). Taming the chaos?! Using eXplainable artificial intelligence (XAI) to tackle the complexity in mental health research. European Child & Adolescent Psychiatry, 30, 1143-1146.
Ros, R., & Graziano, P.A. (2018). Social functioning in children with or at risk for attention deficit/hyperactivity disorder: A meta-analytic review. Journal of Clinical Child & Adolescent Psychology, 47, 213-235.
Shawe-Taylor, J., & Cristianini, N. (2004). Kernel methods for pattern analysis. Cambridge, UK: University Press.
Tariq, Q., Daniels, J., Schwartz, J.N., Washington, P., Kalantarian, H., & Wall, D.P. (2018). Mobile detection of autism through machine learning on home video: A development and prospective validation study. PLOS Medicine, 15, e1002705.
Thom, R.P., Keary, C.J., Kramer, G., Nowinski, L.A., & McDougle, C.J. (2020). Psychiatric assessment of social impairment across the lifespan. Harvard Review of Psychiatry, 28, 159-178.
Tyson, K.E., & Cruess, D.G. (2012). Differentiating high-functioning autism and social phobia. Journal of Autism and Developmental Disorders, 42, 1477-1490.
Uddin, S., Khan, A., Hossain, M.E., & Moni, M.A. (2019). Comparing different supervised machine learning algorithms for disease prediction. BMC Medical Informatics and Decision Making, 19, 281.
Vabalas, A., Gowen, E., Poliakoff, E., & Casson, A.J. (2019). Machine learning algorithm validation with a limited sample size. PLOS ONE, 14, e0224365.
van Steensel, F.J.A., Bögels, S.M., & Wood, J.J. (2013). Autism spectrum traits in children with anxiety disorders. Journal of Autism and Developmental Disorders, 43, 361-370.
von Polier, G.G., Greimel, E., Konrad, K., Großheinrich, N., Kohls, G., Vloet, T.D., … & Schulte-Rüther, M. (2020). Neural correlates of empathy in boys with early onset conduct disorder. Frontiers in Psychiatry, 11, 178.
Wall, D.P., Kosmicki, J., Deluca, T.F., Harstad, E., & Fusaro, V.A. (2012). Use of machine learning to shorten observation-based screening and diagnosis of autism. Translational Psychiatry, 2, e100.
Wittkopf, S., Stroth, S., Langmann, A., Wolff, N., Roessner, V., Roepke, S., … & Kamp-Becker, I. (2021). Differentiation of autism spectrum disorder and mood or anxiety disorder. Autism, 136236132110396. https://doi.org/10.1177/13623613211039673
Wright, M.N., & Ziegler, A. (2017). ranger: A fast implementation of random forests for high dimensional data in C++ and R. Journal of Statistical Software, 77, 1-17.