Are treatment effect assumptions in orthodontic studies overoptimistic?
Journal
European journal of orthodontics
ISSN: 1460-2210
Titre abrégé: Eur J Orthod
Pays: England
ID NLM: 7909010
Informations de publication
Date de publication:
04 10 2021
04 10 2021
Historique:
pubmed:
16
5
2021
medline:
28
10
2021
entrez:
15
5
2021
Statut:
ppublish
Résumé
At the clinical trial design stage, assumptions regarding the treatment effects to be detected should be appropriate so that the required sample size can be calculated. There is evidence in the medical literature that sample size assumption can be overoptimistic. The aim of this study was to compare the distribution of the assumed effects versus that of the observed effects as a proxy for overoptimistic treatment effect assumptions at the study design stage. Systematic reviews (SRs) published between 1 January 2010 and 31 December 2019 containing at least one meta-analysis on continuous outcomes were identified electronically. SR and primary study level characteristics were extracted from the SRs and the individual trials. Details on the sample size calculation process and assumptions and the observed treatment effects were extracted. Eighty-five SRs with meta-analysis containing 347 primary trials were included. The median number of SR authors was 5 (interquartile range: 4-7). At the primary study level, the majority were single centre (78.1%), utilized a parallel design (52%), and rated as an unclear/moderate level of risk of bias (34.3%). A sample size was described in only 31.7% (110/347) of studies. From this cohort of 110 studies, in only 37 studies was the assumed clinical difference that the study was designed to detect reported (37/110). The assumed treatment effect was recalculated for the remaining 73 studies (73/110). The one-sided exact signed rank test showed a significant difference between the assumed and observed treatment effects (P < 0.001) suggesting greater values for the assumed effect sizes. Careful consideration of the assumptions at the design stage of orthodontic studies are necessary in order to reduce the unreliability of clinical study results and research waste.
Sections du résumé
BACKGROUND
At the clinical trial design stage, assumptions regarding the treatment effects to be detected should be appropriate so that the required sample size can be calculated. There is evidence in the medical literature that sample size assumption can be overoptimistic. The aim of this study was to compare the distribution of the assumed effects versus that of the observed effects as a proxy for overoptimistic treatment effect assumptions at the study design stage.
MATERIALS AND METHOD
Systematic reviews (SRs) published between 1 January 2010 and 31 December 2019 containing at least one meta-analysis on continuous outcomes were identified electronically. SR and primary study level characteristics were extracted from the SRs and the individual trials. Details on the sample size calculation process and assumptions and the observed treatment effects were extracted.
RESULTS
Eighty-five SRs with meta-analysis containing 347 primary trials were included. The median number of SR authors was 5 (interquartile range: 4-7). At the primary study level, the majority were single centre (78.1%), utilized a parallel design (52%), and rated as an unclear/moderate level of risk of bias (34.3%). A sample size was described in only 31.7% (110/347) of studies. From this cohort of 110 studies, in only 37 studies was the assumed clinical difference that the study was designed to detect reported (37/110). The assumed treatment effect was recalculated for the remaining 73 studies (73/110). The one-sided exact signed rank test showed a significant difference between the assumed and observed treatment effects (P < 0.001) suggesting greater values for the assumed effect sizes.
CONCLUSIONS
Careful consideration of the assumptions at the design stage of orthodontic studies are necessary in order to reduce the unreliability of clinical study results and research waste.
Identifiants
pubmed: 33991101
pii: 6276220
doi: 10.1093/ejo/cjab018
pmc: PMC8488969
doi:
Types de publication
Journal Article
Meta-Analysis
Research Support, Non-U.S. Gov't
Langues
eng
Sous-ensembles de citation
IM
Pagination
583-587Subventions
Organisme : Department of Health
Pays : United Kingdom
Informations de copyright
© The Author(s) 2021. Published by Oxford University Press on behalf of the European Orthodontic Society.
Références
PLoS One. 2014 Jan 21;9(1):e85949
pubmed: 24465806
Biom J. 2006 Aug;48(4):537-55
pubmed: 16972704
Lancet. 1997 Sep 13;350(9080):804-7
pubmed: 9298015
Crit Care. 2010;14(2):R77
pubmed: 20429873
J Clin Epidemiol. 2003 Aug;56(8):717-20
pubmed: 12954462
Am J Orthod Dentofacial Orthop. 2020 Jul;158(1):150-151
pubmed: 32600753
Community Dent Oral Epidemiol. 2012 Feb;40 Suppl 1:36-42
pubmed: 22369707
Clin Trials. 2013 Aug;10(4):522-9
pubmed: 23794405
J Rheumatol. 2005 Nov;32(11):2083-8
pubmed: 16265683
Eur J Orthod. 2020 Sep 11;42(4):454-459
pubmed: 31414114
J Evid Based Dent Pract. 2011 Jun;11(2):75-83
pubmed: 21605830
Gen Psychiatr. 2019 Aug 8;32(4):e100069
pubmed: 31552383
Can J Anaesth. 2018 Jun;65(6):611-618
pubmed: 29569142
Eur J Orthod. 2019 May 24;41(3):225-230
pubmed: 29992332
Nat Rev Neurosci. 2013 May;14(5):365-76
pubmed: 23571845
Eur J Orthod. 2014 Feb;36(1):67-73
pubmed: 23460731
JAMA. 1994 Jul 13;272(2):122-4
pubmed: 8015121
Psychol Sci. 2011 Nov;22(11):1359-66
pubmed: 22006061
JAMA. 2002 Jul 17;288(3):358-62
pubmed: 12117401
Neuroimage. 2020 Nov 1;221:117164
pubmed: 32679253
Am J Orthod Dentofacial Orthop. 2009 Nov;136(5):632.e1-5; discussion 632-3
pubmed: 19892276
Prog Orthod. 2018 Jun 11;19(1):16
pubmed: 29888375