Oversampling and replacement strategies in propensity score matching: a critical review focused on small sample size in clinical settings.
Monte Carlo simulations
Oversampling
Propensity score matching
Replacement
Small samples
Journal
BMC medical research methodology
ISSN: 1471-2288
Titre abrégé: BMC Med Res Methodol
Pays: England
ID NLM: 100968545
Informations de publication
Date de publication:
22 11 2021
22 11 2021
Historique:
received:
14
05
2021
accepted:
26
10
2021
entrez:
23
11
2021
pubmed:
24
11
2021
medline:
27
1
2022
Statut:
epublish
Résumé
Propensity score matching is a statistical method that is often used to make inferences on the treatment effects in observational studies. In recent years, there has been widespread use of the technique in the cardiothoracic surgery literature to evaluate to potential benefits of new surgical therapies or procedures. However, the small sample size and the strong dependence of the treatment assignment on the baseline covariates that often characterize these studies make such an evaluation challenging from a statistical point of view. In such settings, the use of propensity score matching in combination with oversampling and replacement may provide a solution to these issues by increasing the initial sample size of the study and thus improving the statistical power that is needed to detect the effect of interest. In this study, we review the use of propensity score matching in combination with oversampling and replacement in small sample size settings. We performed a series of Monte Carlo simulations to evaluate how the sample size, the proportion of treated, and the assignment mechanism affect the performances of the proposed approaches. We assessed the performances with overall balance, relative bias, root mean squared error and nominal coverage. Moreover, we illustrate the methods using a real case study from the cardiac surgery literature. Matching without replacement produced estimates with lower bias and better nominal coverage than matching with replacement when 1:1 matching was considered. In contrast to that, matching with replacement showed better balance, relative bias, and root mean squared error than matching without replacement for increasing levels of oversampling. The best nominal coverage was obtained by using the estimator that accounts for uncertainty in the matching procedure on sets of units obtained after matching with replacement. The use of replacement provides the most reliable treatment effect estimates and that no more than 1 or 2 units from the control group should be matched to each treated observation. Moreover, the variance estimator that accounts for the uncertainty in the matching procedure should be used to estimate the treatment effect.
Sections du résumé
BACKGROUND
Propensity score matching is a statistical method that is often used to make inferences on the treatment effects in observational studies. In recent years, there has been widespread use of the technique in the cardiothoracic surgery literature to evaluate to potential benefits of new surgical therapies or procedures. However, the small sample size and the strong dependence of the treatment assignment on the baseline covariates that often characterize these studies make such an evaluation challenging from a statistical point of view. In such settings, the use of propensity score matching in combination with oversampling and replacement may provide a solution to these issues by increasing the initial sample size of the study and thus improving the statistical power that is needed to detect the effect of interest. In this study, we review the use of propensity score matching in combination with oversampling and replacement in small sample size settings.
METHODS
We performed a series of Monte Carlo simulations to evaluate how the sample size, the proportion of treated, and the assignment mechanism affect the performances of the proposed approaches. We assessed the performances with overall balance, relative bias, root mean squared error and nominal coverage. Moreover, we illustrate the methods using a real case study from the cardiac surgery literature.
RESULTS
Matching without replacement produced estimates with lower bias and better nominal coverage than matching with replacement when 1:1 matching was considered. In contrast to that, matching with replacement showed better balance, relative bias, and root mean squared error than matching without replacement for increasing levels of oversampling. The best nominal coverage was obtained by using the estimator that accounts for uncertainty in the matching procedure on sets of units obtained after matching with replacement.
CONCLUSIONS
The use of replacement provides the most reliable treatment effect estimates and that no more than 1 or 2 units from the control group should be matched to each treated observation. Moreover, the variance estimator that accounts for the uncertainty in the matching procedure should be used to estimate the treatment effect.
Identifiants
pubmed: 34809559
doi: 10.1186/s12874-021-01454-z
pii: 10.1186/s12874-021-01454-z
pmc: PMC8609749
doi:
Types de publication
Journal Article
Review
Langues
eng
Sous-ensembles de citation
IM
Pagination
256Informations de copyright
© 2021. The Author(s).
Références
J Clin Epidemiol. 2010 Jan;63(1):2-6
pubmed: 19230611
Stat Med. 2009 Nov 10;28(25):3083-107
pubmed: 19757444
Pharmacoepidemiol Drug Saf. 2008 Jun;17(6):546-55
pubmed: 18311848
Stat Med. 2014 May 10;33(10):1685-99
pubmed: 24323618
Stat Med. 2020 Jul 30;39(17):2265-2290
pubmed: 32449222
Stat Med. 2006 Jul 15;25(13):2230-56
pubmed: 16220488
Eur J Epidemiol. 2019 Jan;34(1):23-36
pubmed: 30341708
Am J Epidemiol. 2004 Aug 15;160(4):301-5
pubmed: 15286014
J Am Coll Cardiol. 2010 Jul 6;56(2):117-24
pubmed: 20451344
Stat Med. 2004 Jan 15;23(1):65-75
pubmed: 14695640
Dev Psychol. 2008 Mar;44(2):395-406
pubmed: 18331131
Stat Methods Med Res. 2017 Jun;26(3):1199-1215
pubmed: 25715391
Psychol Methods. 2004 Dec;9(4):403-25
pubmed: 15598095
Pharm Stat. 2011 Mar-Apr;10(2):150-61
pubmed: 20925139
Heart Vessels. 2016 Oct;31(10):1616-24
pubmed: 26577993
Am J Epidemiol. 2006 Jun 15;163(12):1149-56
pubmed: 16624967
Ann Thorac Surg. 2003 Dec;76(6):2000-6
pubmed: 14667630
Stat Med. 2014 Mar 15;33(6):1057-69
pubmed: 24123228
BMC Med Res Methodol. 2017 Dec 6;17(1):162
pubmed: 29207961
Int J Environ Res Public Health. 2021 Jun 22;18(13):
pubmed: 34206234
J Artif Organs. 2018 Mar;21(1):52-60
pubmed: 28988400
Stat Med. 2014 Oct 30;33(24):4306-19
pubmed: 25087884
J Am Coll Cardiol. 2018 Apr 3;71(13):1417-1428
pubmed: 29598861
Stat Methods Med Res. 2019 Jan;28(1):3-19
pubmed: 28573919
Circulation. 2018 Jul 24;138(4):367-376
pubmed: 29514799
J Clin Epidemiol. 1994 Aug;47(8):881-9
pubmed: 7730891
Circulation. 2005 Aug 30;112(9 Suppl):I351-7
pubmed: 16159845
BMC Med Res Methodol. 2020 Jun 26;20(1):168
pubmed: 32586271
Stat Sci. 2010 Feb 1;25(1):1-21
pubmed: 20871802
Multivariate Behav Res. 2011 May;46(3):399-424
pubmed: 21818162
Stat Methods Med Res. 2020 Mar;29(3):644-658
pubmed: 32186264
J Clin Epidemiol. 2010 Aug;63(8):826-33
pubmed: 20630332
Stat Med. 2020 May 20;39(11):1623-1640
pubmed: 32109319
Ann Thorac Surg. 2018 Jul;106(1):145-150
pubmed: 29625102
Biometrics. 2000 Mar;56(1):118-24
pubmed: 10783785
Am J Epidemiol. 2010 Nov 1;172(9):1092-7
pubmed: 20802241
J Thorac Cardiovasc Surg. 2020 Feb;159(2):432-442.e1
pubmed: 31213376
Psychol Methods. 2008 Dec;13(4):279-313
pubmed: 19071996
Interact Cardiovasc Thorac Surg. 2018 Jan 1;26(1):84-90
pubmed: 29049830
J Clin Epidemiol. 2013 Aug;66(8 Suppl):S84-S90.e1
pubmed: 23849158
Stat Med. 2014 Feb 28;33(5):721-37
pubmed: 24114884
BMC Med Res Methodol. 2012 May 30;12:70
pubmed: 22646911
Stat Med. 2019 May 20;38(11):2074-2102
pubmed: 30652356
CMAJ. 2012 Aug 7;184(11):1265-9
pubmed: 22371511
Stat Med. 2010 Feb 10;29(3):337-46
pubmed: 19960510
Biom J. 2009 Feb;51(1):171-84
pubmed: 19197955
Int J Artif Organs. 2017 Oct 16;:0
pubmed: 29048703
Artif Organs. 2019 Mar;43(3):E41-E52
pubmed: 30062796
Stat Med. 2010 Sep 10;29(20):2137-48
pubmed: 20108233
Stat Med. 2004 Oct 15;23(19):2937-60
pubmed: 15351954