Evaluations of statistical methods for outlier detection when benchmarking in clinical registries: a systematic review.
epidemiology
health & safety
quality in health care
Journal
BMJ open
ISSN: 2044-6055
Titre abrégé: BMJ Open
Pays: England
ID NLM: 101552874
Informations de publication
Date de publication:
14 07 2023
14 07 2023
Historique:
medline:
17
7
2023
pubmed:
15
7
2023
entrez:
14
7
2023
Statut:
epublish
Résumé
Benchmarking is common in clinical registries to support the improvement of health outcomes by identifying underperforming clinician or health service providers. Despite the rise in clinical registries and interest in publicly reporting benchmarking results, appropriate methods for benchmarking and outlier detection within clinical registries are not well established, and the current application of methods is inconsistent. The aim of this review was to determine the current statistical methods of outlier detection that have been evaluated in the context of clinical registry benchmarking. A systematic search for studies evaluating the performance of methods to detect outliers when benchmarking in clinical registries was conducted in five databases: EMBASE, ProQuest, Scopus, Web of Science and Google Scholar. A modified healthcare modelling evaluation tool was used to assess quality; data extracted from each study were summarised and presented in a narrative synthesis. Nineteen studies evaluating a variety of statistical methods in 20 clinical registries were included. The majority of studies conducted application studies comparing outliers without statistical performance assessment (79%), while only few studies used simulations to conduct more rigorous evaluations (21%). A common comparison was between random effects and fixed effects regression, which provided mixed results. Registry population coverage, provider case volume minimum and missing data handling were all poorly reported. The optimal methods for detecting outliers when benchmarking clinical registry data remains unclear, and the use of different models may provide vastly different results. Further research is needed to address the unresolved methodological considerations and evaluate methods across a range of registry conditions. CRD42022296520.
Identifiants
pubmed: 37451708
pii: bmjopen-2022-069130
doi: 10.1136/bmjopen-2022-069130
pmc: PMC10351235
doi:
Types de publication
Systematic Review
Journal Article
Research Support, Non-U.S. Gov't
Langues
eng
Sous-ensembles de citation
IM
Pagination
e069130Informations de copyright
© Author(s) (or their employer(s)) 2023. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.
Déclaration de conflit d'intérêts
Competing interests: None declared.
Références
Health Policy. 2016 Oct;120(10):1151-1161
pubmed: 27638232
Croat Med J. 2019 Feb 28;60(1):50-52
pubmed: 30825279
Colorectal Dis. 2010 Aug;12(8):783-91
pubmed: 20041920
Med Decis Making. 2002 Mar-Apr;22(2):163-72
pubmed: 11958498
PLoS One. 2017 Sep 8;12(9):e0183667
pubmed: 28886607
Ann Thorac Surg. 2015 Mar;99(3):757-61
pubmed: 25742812
BMC Health Serv Res. 2018 Dec 29;18(1):1007
pubmed: 30594187
Am J Transplant. 2014 Jun;14(6):1310-7
pubmed: 24786673
Med Decis Making. 2003 Nov-Dec;23(6):526-39
pubmed: 14672113
Circ Cardiovasc Qual Outcomes. 2016 Nov;9(6):659-669
pubmed: 28263941
Ann Surg. 2012 Apr;255(4):703-7
pubmed: 22388108
Hum Vaccin Immunother. 2016 Nov;12(11):2813-2832
pubmed: 27448625
Child Dev. 2014 May-Jun;85(3):842-860
pubmed: 24116396
J Intensive Care. 2021 Feb 15;9(1):18
pubmed: 33588956
JACC Cardiovasc Interv. 2015 Jan;8(1 Pt A):10-6
pubmed: 25499301
J Am Coll Surg. 2015 Nov;221(5):901-13
pubmed: 26363711
PLoS One. 2014 Jul 16;9(7):e102297
pubmed: 25029164
Stat Methods Med Res. 2016 Dec;25(6):2670-2684
pubmed: 24742429
ANZ J Surg. 2022 Jan;92(1-2):23-26
pubmed: 35040551
Med J Aust. 2016 Nov 21;205(10):S27-S29
pubmed: 27852198
BMJ Qual Saf. 2021 Jan;30(1):38-45
pubmed: 32034014
Circ Cardiovasc Qual Outcomes. 2018 Oct;11(10):e004951
pubmed: 30354572
Value Health. 2014 Mar;17(2):174-82
pubmed: 24636375
Stat Med. 2018 Feb 10;37(3):457-472
pubmed: 29034485
Am J Transplant. 2013 Aug;13(8):2044-51
pubmed: 23890285
BMC Med Res Methodol. 2014 Apr 22;14:53
pubmed: 24755369
BMJ Qual Saf. 2013 Oct;22(10):870-6
pubmed: 23832924
Biostatistics. 2014 Oct;15(4):651-64
pubmed: 24812420
J Trauma. 2010 Nov;69(5):1132-7
pubmed: 20404760
Circ Cardiovasc Qual Outcomes. 2015 Jan;8(1):15-22
pubmed: 25424242
EGEMS (Wash DC). 2017 Dec 07;5(1):26
pubmed: 29930965
Paediatr Respir Rev. 2013 Sep;14(3):141-5
pubmed: 23707656
J Clin Epidemiol. 2008 Mar;61(3):232-240
pubmed: 18226745
Health Serv Res. 2010 Dec;45(6 Pt 1):1614-29
pubmed: 20722747
Med J Aust. 2017 Jun 5;206(10):427-429
pubmed: 28566065
J Trauma Nurs. 2019 May/Jun;26(3):121-127
pubmed: 31483767
Health Serv Res. 2014 Jun;49(3):1056-73
pubmed: 24461071
ANZ J Surg. 2016 Nov;86(11):951-952
pubmed: 27784147
Pharmacoepidemiol Drug Saf. 2020 Apr;29(4):369-379
pubmed: 32128936
Lancet. 2013 Nov 16;382(9905):1674-7
pubmed: 23831144
J Eval Clin Pract. 2001 Feb;7(1):35-45
pubmed: 11240838
Med J Aust. 2011 Apr 4;194(7):360-3
pubmed: 21470087
Surgery. 2015 Feb;157(2):381-95
pubmed: 25616951
Stat Med. 2013 Sep 20;32(21):3720-36
pubmed: 23526209
J R Stat Soc Ser C Appl Stat. 2011 Aug;60(4):575-589
pubmed: 21918583
Med J Aust. 2010 Mar 1;192(5):244-5
pubmed: 20201755
Health Care Manag Sci. 2015 Jun;18(2):166-72
pubmed: 24402171
Br J Anaesth. 2014 Aug;113(2):286-94
pubmed: 25038159
Intern Med J. 2011 Jan;41(1a):42-8
pubmed: 19811553
J Public Health Med. 2003 Dec;25(4):325-35
pubmed: 14747592
J Gen Intern Med. 2011 May;26(5):467-73
pubmed: 21104453
Colorectal Dis. 2012 Jul;14(7):e413-21
pubmed: 22321047
Med Care. 2010 May;48(5):458-65
pubmed: 20393365
Qual Saf Health Care. 2005 Oct;14(5):347-51
pubmed: 16195568
Med Decis Making. 2014 Feb;34(2):192-205
pubmed: 23920433
Stat Methods Med Res. 2018 Nov;27(11):3350-3366
pubmed: 28330409
Transplantation. 2017 Jun;101(6):1373-1380
pubmed: 27482960
Stat Med. 2005 Apr 30;24(8):1185-202
pubmed: 15568194