Hypothesis testing and sample size considerations for the test-negative design.
Case-control study
Continuity correction
Sample size
Score test
Test-negative design
Vaccines
Journal
BMC medical research methodology
ISSN: 1471-2288
Titre abrégé: BMC Med Res Methodol
Pays: England
ID NLM: 100968545
Informations de publication
Date de publication:
16 Jul 2024
16 Jul 2024
Historique:
received:
20
12
2023
accepted:
05
07
2024
medline:
17
7
2024
pubmed:
17
7
2024
entrez:
16
7
2024
Statut:
epublish
Résumé
The test-negative design (TND) is an observational study design to evaluate vaccine effectiveness (VE) that enrolls individuals receiving diagnostic testing for a target disease as part of routine care. VE is estimated as one minus the adjusted odds ratio of testing positive versus negative comparing vaccinated and unvaccinated patients. Although the TND is related to case-control studies, it is distinct in that the ratio of test-positive cases to test-negative controls is not typically pre-specified. For both types of studies, sparse cells are common when vaccines are highly effective. We consider the implications of these features on power for the TND. We use simulation studies to explore three hypothesis-testing procedures and associated sample size calculations for case-control and TND studies. These tests, all based on a simple logistic regression model, are a standard Wald test, a continuity-corrected Wald test, and a score test. The Wald test performs poorly in both case-control and TND when VE is high because the number of vaccinated test-positive cases can be low or zero. Continuity corrections help to stabilize the variance but induce bias. We observe superior performance with the score test as the variance is pooled under the null hypothesis of no group differences. We recommend using a score-based approach to design and analyze both case-control and TND. We propose a modification to the TND score sample size to account for additional variability in the ratio of controls over cases. This work enhances our understanding of the data generating mechanism in a test-negative design (TND) and how it is distinct from that of a case-control study due to its passive recruitment of controls.
Identifiants
pubmed: 39014324
doi: 10.1186/s12874-024-02277-4
pii: 10.1186/s12874-024-02277-4
doi:
Types de publication
Journal Article
Langues
eng
Sous-ensembles de citation
IM
Pagination
151Informations de copyright
© 2024. The Author(s).
Références
De Serres G, Skowronski DM, Wu XW, Ambrose CS. The test-negative design: validity, accuracy and precision of vaccine efficacy estimates compared to the gold standard of randomised placebo-controlled clinical trials. Euro Surveill. 2013;18(37):1–9. https://doi.org/10.2807/1560-7917.es2013.18.37.20585 .
doi: 10.2807/1560-7917.es2013.18.37.20585
Jackson ML, Nelson JC. The test-negative design for estimating influenza vaccine effectiveness. Vaccine. 2013;31(17):2165–8. https://doi.org/10.1016/j.vaccine.2013.02.053 .
doi: 10.1016/j.vaccine.2013.02.053
pubmed: 23499601
Schwartz LM, Halloran ME, Rowhani-Rahbar A, Neuzil KM, Victor JC. Rotavirus vaccine effectiveness in low-income settings: An evaluation of the test-negative design. Vaccine. 2017;35(1):184–90. https://doi.org/10.1016/j.vaccine.2016.10.077 .
doi: 10.1016/j.vaccine.2016.10.077
pubmed: 27876198
pmcid: 5154240
Azman AS, Parker LA, Rumunu J, et al. Effectiveness of one dose of oral cholera vaccine in response to an outbreak: a case-cohort study. Lancet Glob Heal. 2016;4:e856–63. https://doi.org/10.1016/S2214-109X(16)30211-X .
doi: 10.1016/S2214-109X(16)30211-X
Chua H, Feng S, Lewnard JA, et al. The Use of Test-negative Controls to Monitor Vaccine Effectiveness: A Systematic Review of Methodology. Epidemiology. 2020;31(1):43–64. https://doi.org/10.1097/EDE.0000000000001116 .
doi: 10.1097/EDE.0000000000001116
pubmed: 31609860
pmcid: 6888869
Sullivan SG, Feng S, Cowling BJ. Influenza vaccine effectiveness: potential of the test-negative design. A systematic review. Expert Rev Vaccines. 2014;13(12):1571–91. https://doi.org/10.1586/14760584.2014.966695.Influenza .
doi: 10.1586/14760584.2014.966695.Influenza
pubmed: 25348015
pmcid: 4277796
Bernal JL, Andrews N, Gower C, et al. Early effectiveness of COVID-19 vaccination with BNT162b2 mRNA vaccine and ChAdOx1 adenovirus vector vaccine on symptomatic disease, hospitalisations and mortality in older adults in England. medRxiv. Published online March 2, 2021:2021.03.01.21252652. https://doi.org/10.1101/2021.03.01.21252652 .
Broome CV, Facklam RR, Fraser DW. Pneumococcal disease after pneumococcal vaccination. N Engl J Med. 1980;303(10):549–52. https://doi.org/10.1056/NEJM198009043031003 .
doi: 10.1056/NEJM198009043031003
pubmed: 6995835
Sullivan SG, Tchetgen EJT, Cowling BJ. Theoretical basis of the test-negative study design for assessment of influenza vaccine effectiveness. Pract Epidemiol. 2016;184(5):345–53. https://doi.org/10.1093/aje/kww064 .
doi: 10.1093/aje/kww064
Dean NE, Hogan JW, Schnitzer ME. Covid-19 vaccine effectiveness and the test-negative design. N Engl J Med. 2021;385:1431–3. https://doi.org/10.1056/NEJMe2113151 .
doi: 10.1056/NEJMe2113151
pubmed: 34496195
Cheng AC, Holmes M, Irving LB, et al. Influenza Vaccine Effectiveness against Hospitalisation with Confirmed Influenza in the 2010–11 Seasons: A Test-negative Observational Study. PLoS ONE. 2013;8(7):1–8. https://doi.org/10.1371/journal.pone.0068760 .
doi: 10.1371/journal.pone.0068760
Bateman AC, Kieke BA, Irving SA, Meece JK, Shay DK, Belongia EA. Effectiveness of Monovalent 2009 Pandemic Influenza A Virus Subtype H1N1 and 2010–2011 Trivalent Inactivated Influenza Vaccines in Wisconsin During the 2010–2011 Influenza Season. Published online 2013. https://doi.org/10.1093/infdis/jit020 .
Anders KL, Cutcher Z, Kleinschmidt I, et al. Cluster-randomized test-negative design trials: a novel and efficient method to assess the efficacy of community-level dengue interventions. Am J Epidemiol. 2018;187(9):2021–8. https://doi.org/10.1093/aje/kwy099 .
doi: 10.1093/aje/kwy099
pubmed: 29741576
pmcid: 6118074
Helmeke C, Gräfe L, Irmscher HM, Gottschalk C, Karagiannis I, Oppermann H. Effectiveness of the 2012/13 trivalent live and inactivated influenza vaccines in children and adolescents in Saxony-Anhalt, Germany: a test-negative case-control study. PLoS One. 2015;10(4):1–10. https://doi.org/10.1371/journal.pone.0122910 .
doi: 10.1371/journal.pone.0122910
Griffin MR, Monto AS, Belongia EA, Treanor JJ, Chen Q. Effectiveness of Non-Adjuvanted Pandemic Influenza A Vaccines for Preventing Pandemic Influenza Acute Respiratory Illness Visits in 4 U. PLoS ONE. 2011;6(8): e23085. https://doi.org/10.1371/journal.pone.0023085 .
doi: 10.1371/journal.pone.0023085
pubmed: 21857999
pmcid: 3155536
Eisenberg KW, Szilagyi PG, Fairbrother G, et al. Vaccine effectiveness against laboratory-confirmed influenza in children 6 to 59 months of age during the 2003 2004 and 2004 2005 influenza seasons. Pediatrics. 2008;122(5):911–9. https://doi.org/10.1542/peds.2007-3304 .
doi: 10.1542/peds.2007-3304
pubmed: 18977968
Cowling BJ, Chan KH, Feng S, et al. The effectiveness of influenza vaccination in preventing hospitalizations in children in Hong Kong, 2009–2013. Vaccine. 2014;32(41):5278–84. https://doi.org/10.1016/j.vaccine.2014.07.084 .
doi: 10.1016/j.vaccine.2014.07.084
pubmed: 25092636
pmcid: 4165553
Wang Y, Zhang T, Chen L, et al. Seasonal influenza vaccine effectiveness against medically attended influenza illness among children aged 6–59 months, October 2011-September 2012: A matched test-negative case-control study in Suzhou. China Vaccine. 2016;34(21):2460–5. https://doi.org/10.1016/j.vaccine.2016.03.056 .
doi: 10.1016/j.vaccine.2016.03.056
pubmed: 27016650
pmcid: 8855226
Cheng AC, Kotsimbos T, Kelly HA, et al. Effectiveness of H1N1/09 monovalent and trivalent influenza vaccines against hospitalization with laboratory-confirmed H1N1/09 influenza in Australia: a test-negative case control study. Vaccine. 2011;29(43):7320–5. https://doi.org/10.1016/j.vaccine.2011.07.087 .
doi: 10.1016/j.vaccine.2011.07.087
pubmed: 21810450
Belongia EA, Simpson MD, King JP, et al. Variable influenza vaccine effectiveness by subtype: a systematic review and meta-analysis of test-negative design studies. Lancet Infect Dis. 2016;16(8):942–51. https://doi.org/10.1016/S1473-3099(16)00129-8 .
doi: 10.1016/S1473-3099(16)00129-8
pubmed: 27061888
Feng S, Cowling BJ, Kelly H, Sullivan SG. Estimating Influenza Vaccine Effectiveness with the Test-Negative Design Using Alternative Control Groups: A Systematic Review and Meta-Analysis. Am J Epidemiol. 2017;187(2):389–97. https://doi.org/10.1093/aje/kwx251 .
doi: 10.1093/aje/kwx251
The FREQ Procedure - SAS. https://documentation.sas.com/?cdcId=pgmsascdc&cdcVersion=9.4_3.5&docsetId=procstat&docsetTarget=procstat_freq_examples05.htm&locale=en .
Aragon TJ, Fay MP, Wollschlaeger D, Omidpanah A. R package epitools. Published 2020. https://cran.r-project.org/web/packages/epitools/epitools.pdf .
Rosenberg ES, Dorabawila V, Easton D, et al. Covid-19 vaccine effectiveness in New York State. N Engl J Med. 2022;386:116–27. https://doi.org/10.1056/NEJMoa2116063 .
doi: 10.1056/NEJMoa2116063
pubmed: 34942067
Spinner C, Ding L, David IB, Bernstein DI, et al. Human papillomavirus vaccine effectiveness and herd protection in young women. Pediatrics. 2019;243(2):e20181902.
doi: 10.1542/peds.2018-1902
Yates F. Contingency tables involving small numbers and the χ2 Test. J R Stat Soc. 1934;1(2):217–35. https://doi.org/10.2307/2983604 .
doi: 10.2307/2983604
Haviland MG. Yates’s correction for continuity and the analysis of 2 × 2 contingency tables. Stat Med. 1990;9(4):363–7. https://doi.org/10.1002/sim.4780090403 .
doi: 10.1002/sim.4780090403
pubmed: 2362976
Rückinger S, van der Linden M, Reinert RR, von Kries R. Efficacy of 7-valent pneumococcal conjugate vaccination in Germany: An analysis using the indirect cohort method. Vaccine. 2010;28(31):5012–6. https://doi.org/10.1016/J.VACCINE.2010.05.021 .
doi: 10.1016/J.VACCINE.2010.05.021
pubmed: 20546832
Storer BE, Kim C. Exact properties of some exact test statistics for comparing two binomial proportions. J Am Stat Assoc. 1990;85(409):146–55. https://doi.org/10.1080/01621459.1990.10475318 .
doi: 10.1080/01621459.1990.10475318
Kang SH, Ahn CW. Tests for the homogeneity of two binomial proportions in extremely unbalanced 2 x 2 contingency tables. Stat Med. 2008;27(14):2524–35. https://doi.org/10.1002/sim.3055 . PMID:17847031;PMCID:PMC3921682.
doi: 10.1002/sim.3055
pubmed: 17847031
pmcid: 3921682
Agresti A. An introduction to categorical data analysis. New York: John Wiley and Sons; 1996. https://doi.org/10.1002/0470114754 .
Breslow NE, Day NE. Statistical methods in cancer research. Volume II- - the design and analysis of cohort studies. IARC Sci Publ; 1987.
Fleiss JL, Levin B, Paik M. Statistical Methods for rates and proportions. Third Edition. 1981. https://doi.org/10.1002/0471445428.ch18 .
Casagrande JT, Pike MC, Smith PG. An improved approximate formula for calculating sample sizes for comparing two binomial distributions. Biometrics. 1978;34(3):483–6. https://doi.org/10.2307/2530613 .
doi: 10.2307/2530613
pubmed: 719125
Borgan O, Breslow N, Chatterjee N, Gail MH, Scoot A, Wild CJ. Handbook of Statistical Methods for Case-Control Studies (1st ed.). Chapman and Hall/CRC. https://doi.org/10.1201/9781315154084 .
Agresti A. An introduction to categorical data analysis. 2nd ed. John Wiley and Sons, Inc; 2007.
Emura T, Liao YT. Critical review and comparison of continuity correction methods: The normal approximation to the binomial distribution. Commun Stat Simul Comput. 2017;47(8):2266–85. https://doi.org/10.1080/03610918.2017.1341527 .
doi: 10.1080/03610918.2017.1341527
Dean NE, Halloran ME, Longini IM Jr. Temporal confounding in the test-negative design. Am J Epidemiol. 2020;189(11):1402–7. https://doi.org/10.1093/aje/kwaa084 .
doi: 10.1093/aje/kwaa084
pubmed: 32415834
pmcid: 7604521
Halloran ME, Longini IM Jr, Struchiner CJ. Design and analysis of vaccine studies. Springer; 2010. https://doi.org/10.1007/978-0-387-68636-3 .
Foppa IM, Haber M, Ferdinands JM, Shay DK. The case test-negative design for studies of the effectiveness of influenza vaccine. Vaccine. 2013;31(30):3104–9. https://doi.org/10.1016/J.VACCINE.2013.04.026 .
doi: 10.1016/J.VACCINE.2013.04.026
pubmed: 23624093
Orenstein EW, De Serres G, Haber MJ, et al. Methodologic issues regarding the use of three observational study designs to assess influenza vaccine effectiveness. Int J Epidemiol. 2007;36(3):623–31. https://doi.org/10.1093/ije/dym021 .
doi: 10.1093/ije/dym021
pubmed: 17403908
Shield WS, Heeler R. Analysis of contingency tables with sparse values. J Mark Res. 1979;16(3):382–6.
doi: 10.1177/002224377901600310
Guo SW, Thompson EA. Analysis of sparse contingency tables: Monte Carlo estimation of exact P-values. Department of Statistics: University of Washington; 1989. https://stat.uw.edu/research/preprints/tech-report/analysis-sparse-contingency-tables-monte-carlo-estimation-exact-p .
Baglivo J, Oliver D, Pagano M. Methods for the Analysis of Contingency Tables with Large and Small Cell Counts. J Am Stat Assoc. 1988;83(404):1006–13. https://doi.org/10.1080/01621459.1988.10478692 .
doi: 10.1080/01621459.1988.10478692