Distinguishing Clinical from Statistical Significances in Contemporary Comparative Effectiveness Research.
Journal
Annals of surgery
ISSN: 1528-1140
Titre abrégé: Ann Surg
Pays: United States
ID NLM: 0372354
Informations de publication
Date de publication:
23 Feb 2024
23 Feb 2024
Historique:
medline:
23
2
2024
pubmed:
23
2
2024
entrez:
23
2
2024
Statut:
aheadofprint
Résumé
To determine the prevalence of clinical significance reporting in contemporary comparative effectiveness research (CER). In CER, a statistically significant difference between study groups may or may not be clinically significant. Misinterpreting statistically significant results could lead to inappropriate recommendations that increase healthcare costs and treatment toxicity. CER studies from 2022 issues of Annals of Surgery, Journal of the American Medical Association, Journal of Clinical Oncology, Journal of Surgical Research, and Journal of the American College of Surgeons were systematically reviewed by two different investigators. The primary outcome of interest was whether authors specified what they considered to be a clinically significant difference in the Methods. Of 307 reviewed studies, 162 were clinical trials and 145 were observational studies. Authors specified what they considered to be a clinically significant difference in 26 studies (8.5%). Clinical significance was defined using clinically validated standards in 25 studies and subjectively in 1 study. Seven studies (2.3%) recommended a change in clinical decision-making, all with primary outcomes achieving statistical significance. Five (71.4%) of these studies did not have clinical significance defined in their methods. In randomized controlled trials with statistically significant results, sample size was inversely correlated with effect size (r=-0.30, P=0.038). In contemporary CER, most authors do not specify what they consider to be a clinically significant difference in study outcome. Most studies recommending a change in clinical-decision making did so based on statistical significance alone, and clinical significance was usually defined with clinically validated standards.
Sections du résumé
OBJECTIVE
OBJECTIVE
To determine the prevalence of clinical significance reporting in contemporary comparative effectiveness research (CER).
BACKGROUND
BACKGROUND
In CER, a statistically significant difference between study groups may or may not be clinically significant. Misinterpreting statistically significant results could lead to inappropriate recommendations that increase healthcare costs and treatment toxicity.
METHODS
METHODS
CER studies from 2022 issues of Annals of Surgery, Journal of the American Medical Association, Journal of Clinical Oncology, Journal of Surgical Research, and Journal of the American College of Surgeons were systematically reviewed by two different investigators. The primary outcome of interest was whether authors specified what they considered to be a clinically significant difference in the Methods.
RESULTS
RESULTS
Of 307 reviewed studies, 162 were clinical trials and 145 were observational studies. Authors specified what they considered to be a clinically significant difference in 26 studies (8.5%). Clinical significance was defined using clinically validated standards in 25 studies and subjectively in 1 study. Seven studies (2.3%) recommended a change in clinical decision-making, all with primary outcomes achieving statistical significance. Five (71.4%) of these studies did not have clinical significance defined in their methods. In randomized controlled trials with statistically significant results, sample size was inversely correlated with effect size (r=-0.30, P=0.038).
CONCLUSION
CONCLUSIONS
In contemporary CER, most authors do not specify what they consider to be a clinically significant difference in study outcome. Most studies recommending a change in clinical-decision making did so based on statistical significance alone, and clinical significance was usually defined with clinically validated standards.
Identifiants
pubmed: 38390761
doi: 10.1097/SLA.0000000000006250
pii: 00000658-990000000-00793
doi:
Types de publication
Journal Article
Langues
eng
Sous-ensembles de citation
IM
Informations de copyright
Copyright © 2024 Wolters Kluwer Health, Inc. All rights reserved.
Déclaration de conflit d'intérêts
Conflicts of Interest and Source of Funding: None