How Often Do Orthopaedic Matched Case-Control Studies Use Matched Methods? A Review of Methodological Quality.

BACKGROUND

Case-control studies are a common method of analyzing associations between clinical outcomes and potential risk factors. Matching cases to controls based on known confounding variables can decrease bias and allow investigators to assess the association of interest with increased precision. However, the analysis of matched data generally requires matched statistical methods, and failure to use these methods can lead to imprecise or biased results. The appropriate use of matched statistical methods in orthopaedic case-control studies has not been documented.

QUESTIONS/PURPOSES

(1) What proportion of matched orthopaedic case-control studies use the appropriate matched statistical analyses? (2) What study factors are associated with the use of appropriate matched statistical tests?

METHODS

All matched case-control studies published in the top 10 orthopaedic journals according to impact factor from 2007 to 2016 were identified by literature review. Studies using appropriate statistical techniques were identified by two independent evaluators; discrepancies were settled by a third evaluator, all with advanced training in biostatistics. The number of studies using appropriate matched statistical methods was compared with the number of studies reviewed. Logistic regression was used to identify key study factors (including journal, publication year, rank according to impact factor, number of matching factors, number of controls per case, and the inclusion of a biostatistician coauthor) associated with the use of appropriate statistical methods. Three hundred nineteen articles that were initially classified as case-control studies were screened, yielding 83 matched case-control studies. One hundred two of the excluded articles were cohort or cross-sectional studies that were misclassified as case-control studies. The median number of matching factors was 3.0 (range, 1-10) and the median number of controls per case was 1.0 (range, 0.5-6.0). Thirty studies (36%) had a statistician coauthor.

RESULTS

Thirty of the 83 included studies (36%) used appropriately matched methods throughout, 11 (13%) used matched methods for multivariable but not univariable analyses, and 42 (51%) used only unmatched methods, which we considered inappropriate. After controlling for the number of controls per case and publication year, we found that the inclusion of a statistician coauthor (70% versus 38%; odds ratio, 3.6; 95% confidence interval, 1.4-20.3; p = 0.01) and journal were associated with the use of appropriate methods.

CONCLUSIONS

Although matched case-control studies can be statistically more efficient study designs, in that they are capable of generating more precise effect size estimates than other kinds of retrospective research, most orthopaedic case-control studies use inappropriate statistical methods in their analyses. Additionally, the high degree of study misclassification indicates a need to more rigorously define differences among case-control, cohort, and cross-sectional study designs.

CLINICAL RELEVANCE

Failing to use matched statistical tests may lead to imprecise and/or biased effect estimates, which may lead to a tendency to overestimate or underestimate associations between possible risk factors and clinically relevant outcomes. Orthopaedic researchers should be cognizant of the risks and benefits of matching and should consult individuals with biostatistical expertise as needed to ensure that their statistical methods are appropriate and methodologically rigorous.