Statistical analysis in Small-N Designs: using linear mixed-effects modeling for evaluating intervention effectiveness.
Small-N designs
dysgraphia
mixed-effects
treatment study
tutorial
Journal
Aphasiology
ISSN: 0268-7038
Titre abrégé: Aphasiology
Pays: England
ID NLM: 8708531
Informations de publication
Date de publication:
2019
2019
Historique:
entrez:
5
10
2020
pubmed:
1
1
2019
medline:
1
1
2019
Statut:
ppublish
Résumé
Advances in statistical methods and computing power have led to a renewed interest in addressing the statistical analysis challenges posed by Small-N Designs (SND). Linear mixed-effects modeling (LMEM) is a multiple regression technique that is flexible and suitable for SND and can provide standardized effect sizes and measures of statistical significance. Our primary goals are to: 1) explain LMEM at the conceptual level, situating it in the context of treatment studies, and 2) provide practical guidance for implementing LMEM in repeated measures SND. We illustrate an LMEM analysis, presenting data from a longitudinal training study of five individuals with acquired dysgraphia, analyzing both binomial (accuracy) and continuous (reaction time) repeated measurements. The LMEM analysis reveals that both spelling accuracy and reaction time improved and, for accuracy, improved significantly more quickly under a training schedule with distributed, compared to clustered, practice. We present guidance on obtaining and interpreting various effect sizes and measures of statistical significance from LMEM, and include a simulation study comparing two We provide a strong case for the application of LMEM to the analysis of training studies as a preferable alternative to visual analysis or other statistical techniques. When applied to a treatment dataset, the evidence supports that the approach holds up under the extreme conditions of small numbers of individuals, with repeated measures training data for both continuous (reaction time) and binomially distributed (accuracy) dependent measures. The approach provides standardized measures of effect sizes that are obtained through readily available and well-supported statistical packages, and provides statistically rigorous estimates of the expected average effect size of training effects, taking into account variability across both items and individuals.
Sections du résumé
BACKGROUND
BACKGROUND
Advances in statistical methods and computing power have led to a renewed interest in addressing the statistical analysis challenges posed by Small-N Designs (SND). Linear mixed-effects modeling (LMEM) is a multiple regression technique that is flexible and suitable for SND and can provide standardized effect sizes and measures of statistical significance.
AIMS
OBJECTIVE
Our primary goals are to: 1) explain LMEM at the conceptual level, situating it in the context of treatment studies, and 2) provide practical guidance for implementing LMEM in repeated measures SND.
METHODS & PROCEDURES
METHODS
We illustrate an LMEM analysis, presenting data from a longitudinal training study of five individuals with acquired dysgraphia, analyzing both binomial (accuracy) and continuous (reaction time) repeated measurements.
OUTCOMES & RESULTS
RESULTS
The LMEM analysis reveals that both spelling accuracy and reaction time improved and, for accuracy, improved significantly more quickly under a training schedule with distributed, compared to clustered, practice. We present guidance on obtaining and interpreting various effect sizes and measures of statistical significance from LMEM, and include a simulation study comparing two
CONCLUSION
CONCLUSIONS
We provide a strong case for the application of LMEM to the analysis of training studies as a preferable alternative to visual analysis or other statistical techniques. When applied to a treatment dataset, the evidence supports that the approach holds up under the extreme conditions of small numbers of individuals, with repeated measures training data for both continuous (reaction time) and binomially distributed (accuracy) dependent measures. The approach provides standardized measures of effect sizes that are obtained through readily available and well-supported statistical packages, and provides statistically rigorous estimates of the expected average effect size of training effects, taking into account variability across both items and individuals.
Identifiants
pubmed: 33012945
doi: 10.1080/02687038.2018.1454884
pmc: PMC7531584
mid: NIHMS1506946
doi:
Types de publication
Journal Article
Langues
eng
Pagination
1-30Subventions
Organisme : NIDCD NIH HHS
ID : R01 DC006740
Pays : United States
Déclaration de conflit d'intérêts
Disclosure statement No potential conflict of interest was reported by the authors.
Références
Arch Gen Psychiatry. 2004 Mar;61(3):310-7
pubmed: 14993119
Behav Modif. 2011 Jul;35(4):303-22
pubmed: 21411481
Behav Modif. 2013 Jan;37(1):62-89
pubmed: 22977266
J Sch Psychol. 2014 Apr;52(2):149-78
pubmed: 24606973
Psychol Bull. 2006 May;132(3):354-80
pubmed: 16719566
Cogn Neuropsychol. 2009 Dec;26(8):724-51
pubmed: 20425660
Perspect Psychol Sci. 2006 Sep;1(3):181-210
pubmed: 26151629
Aphasiology. 2015;29(5):570-574
pubmed: 28824217
Behav Res Methods Instrum Comput. 2003 Feb;35(1):1-10
pubmed: 12723775
Res Synth Methods. 2012 Sep;3(3):224-39
pubmed: 26062165
Behav Res Methods. 2009 Nov;41(4):977-90
pubmed: 19897807
Behav Res Methods. 2012 Dec;44(4):1244-54
pubmed: 22648696
Neuropsychologia. 2009 Nov;47(13):2687-9
pubmed: 19383504
J Speech Lang Hear Res. 2016 Oct 1;59(5):1111-1122
pubmed: 27716858
Behav Modif. 2004 Mar;28(2):234-46
pubmed: 14997950
Cortex. 2015 Oct;71:148-59
pubmed: 26218619
Sch Psychol Q. 2015 Mar;30(1):50-63
pubmed: 24884449
Neuropsychol Rehabil. 2014;24(3-4):554-71
pubmed: 24533782
Neuropsychol Rehabil. 2014;24(3-4):305-14
pubmed: 24766415
Neuropsychol Rehabil. 2012;22(2):138-68
pubmed: 22247957
Behav Res Methods. 2012 Sep;44(3):795-805
pubmed: 22180105
Psychol Methods. 2013 Sep;18(3):385-405
pubmed: 23834421
J Mem Lang. 2013 Apr;68(3):
pubmed: 24403724
Cogn Neuropsychol. 2002 Jun 1;19(4):291-9
pubmed: 20957541
Aphasiology. 2012 May 1;26(5):632-655
pubmed: 23188949
Behav Modif. 2014 Sep;38(5):665-704
pubmed: 24902590
Neuropsychologia. 2009 Nov;47(13):2690-5
pubmed: 19383506
J Sch Psychol. 2014 Apr;52(2):109-22
pubmed: 24606971
Neuropsychol Rehabil. 2014;24(3-4):445-63
pubmed: 23883189
Neuropsychol Rehabil. 2011 Jan;21(1):26-41
pubmed: 21181603
Behav Res Methods. 2013 Mar;45(1):65-74
pubmed: 22806706
Neuropsychol Rehabil. 2014;24(3-4):492-506
pubmed: 24641472
J Sch Psychol. 2014 Apr;52(2):191-211
pubmed: 24606975
Neuropsychol Rehabil. 2014;24(3-4):590-606
pubmed: 24047454
Behav Res Methods. 2009 May;41(2):372-84
pubmed: 19363177
Behav Res Methods. 2010 Nov;42(4):930-43
pubmed: 21139160