A forward search algorithm for detecting extreme study effects in network meta-analysis.
Cook's distance
NMAoutlier
forward search
network meta-analysis
outliers
Journal
Statistics in medicine
ISSN: 1097-0258
Titre abrégé: Stat Med
Pays: England
ID NLM: 8215016
Informations de publication
Date de publication:
10 11 2021
10 11 2021
Historique:
revised:
14
04
2021
received:
23
07
2019
accepted:
28
06
2021
pubmed:
23
7
2021
medline:
30
10
2021
entrez:
22
7
2021
Statut:
ppublish
Résumé
In a quantitative synthesis of studies via meta-analysis, it is possible that some studies provide a markedly different relative treatment effect or have a large impact on the summary estimate and/or heterogeneity. Extreme study effects (outliers) can be detected visually with forest/funnel plots and by using statistical outlying detection methods. A forward search (FS) algorithm is a common outlying diagnostic tool recently extended to meta-analysis. FS starts by fitting the assumed model to a subset of the data which is gradually incremented by adding the remaining studies according to their closeness to the postulated data-generating model. At each step of the algorithm, parameter estimates, measures of fit (residuals, likelihood contributions), and test statistics are being monitored and their sharp changes are used as an indication for outliers. In this article, we extend the FS algorithm to network meta-analysis (NMA). In NMA, visualization of outliers is more challenging due to the multivariate nature of the data and the fact that studies contribute both directly and indirectly to the network estimates. Outliers are expected to contribute not only to heterogeneity but also to inconsistency, compromising the NMA results. The FS algorithm was applied to real and artificial networks of interventions that include outliers. We developed an R package (NMAoutlier) to allow replication and dissemination of the proposed method. We conclude that the FS algorithm is a visual diagnostic tool that helps to identify studies that are a potential source of heterogeneity and inconsistency.
Types de publication
Journal Article
Meta-Analysis
Research Support, Non-U.S. Gov't
Langues
eng
Sous-ensembles de citation
IM
Pagination
5642-5656Informations de copyright
© 2021 The Authors. Statistics in Medicine published by John Wiley & Sons Ltd.
Références
Salanti G. Indirect and mixed-treatment comparison, network, or multiple-treatments meta-analysis: many names, many benefits, many concerns for the next generation evidence synthesis tool. Res Synth Methods. 2012;3(2):80-97.
Higgins JPT, Jackson D, Barrett JK, Lu G, Ades AE, White IR. Consistency and inconsistency in network meta-analysis: concepts and models for multi-arm studies. Res Synth Methods. 2012;3(2):98-110.
König J, Krahn U, Binder H. Visualizing the flow of evidence in network meta-analysis and characterizing mixed treatment comparisons. Stat Med. 2013;32(30):5414-5429.
Mavridis D, Giannatsi M, Cipriani A, Salanti G. A primer on network meta-analysis with emphasis on mental health. Evid Based Ment Health. 2015;18(2):40-46.
Salanti G, Higgins JPT, Ades AE, Ioannidis JPA. Evaluation of networks of randomized trials. Stat Methods Med Res. 2008;17(3):279-301.
Sterne JAC, Egger M, Moher D, eds. Chapter 10. Addressing reporting biases. Cochrane Handbook for Systematic Reviews of Interventions. Version 5.1.0 (updated March 2011). Chichester: John Wiley & Sons; 2011. http://handbook.cochrane.org/chapter_9/9_8_chapter_information.html.
Lee KJ, Thompson SG. Flexible parametric models for random-effects distributions. Stat Med. 2008;27(3):418-434.
Baker R, Jackson D. A new approach to outliers in meta-analysis. Health Care Manag Sci. 2008;11(2):121-131.
Baker R, Jackson D. New models for describing outliers in meta-analysis. Res Synth Methods. 2016;7(3):314-328.
Gumedze FN, Jackson D. A random effects variance shift model for detecting and accommodating outliers in meta-analysis. BMC Med Res Methodol. 2011;11:19.
Beath KJ. A finite mixture method for outlier detection and robustness in meta-analysis. Res Synth Methods. 2014;5(4):285-293.
Lin L, Chu H, Hodges JS. Alternative measures of between-study heterogeneity in meta-analysis: reducing the impact of outlying studies. Biometrics. 2017;73(1):156-166.
Viechtbauer W, Cheung MWL. Outlier and influence diagnostics for meta-analysis. Res Synth Methods. 2010;1(2):112-125.
Viechtbauer W. Conducting meta-analyses in R with the metafor package. J Stat Softw. 2010;36(3):1-48.
R Development Core Team. R: A language and Environment for Statistical Computing Version 3.5.1. Vienna, Austria: R Foundation for Statistical Computing; 2018. http://www.R-project.org.
Chaimani A, Higgins JPT, Mavridis D, Spyridonos P, Salanti G. Graphical tools for network meta-analysis in STATA. PLoS One. 2013;8:10.
Lu G, Ades AE. Assessing evidence inconsistency in mixed treatment comparisons. J Am Stat Assoc. 2006;101(474):447-459.
Zhang J, Fu H, Carlin BP. Detecting outlying trials in network meta-analysis. Stat Med. 2015;34(19):2695-2707.
Zhao H, Hodges JS, Carlin BP. Diagnostics for generalized linear hierarchical models in network meta-analysis. Res Synth Methods. 2017;8(3):333-342.
Noma H, Gosho M, Ishii R, Oba K, Furukawa TA. Outlier detection and influence diagnostics in network meta-analysis. Res Synth Methods. 2020;11(6):891-902.
Shi L, Zuo S, Yu D, Zhou X. Influence diagnostics in meta-regression model. Res Synth Methods. 2017;8(3):343-354.
Atkinson AC. Masking unmasked. Biometrika. 1986;73(3):533-541.
Hadi AS. Identifying multiple outliers in multivariate data. J Royal Stat Soc Ser B. 1992;54:761-771.
Atkinson AC, Riani M. Robust Diagnostic Regression Analysis. New York, NY: Springer Publishing Company; 2000.
Atkinson AC. Fast very robust methods for the detection of multiple outliers. J Am Stat Assoc. 1994;89(428):1329-1339.
Atkinson AC, Riani M, Cerioli A. Exploring Multivariate Data with the Forward Search. New York, NY: Springer Publishing Company; 2004.
Mavridis D, Moustaki I. Detecting outliers in factor analysis using the forward search algorithm. Multivar Behav Res. 2008;43(3):453-475.
Mavridis D, Moustaki I. The forward search algorithm for detecting aberrant response patterns in factor analysis for binary data. J Comput Graph Stat. 2009;18(4):1016-1034.
Mavridis D, Moustaki I, Wall M, Salanti G. Detecting outlying studies in meta-regression models using a forward search algorithm. Res Synth Methods. 2016;8(2):199-211.
Petropoulou M, Schwarzer G, Panos A, Mavridis D. NMAoutlier: detecting outliers in network meta-analysis, R package version 0.1.17; 2021. https://cran.r-project.org/packages/NMAoutlier/NMAoutlier.
Rücker G. Network meta-analysis, electrical networks and graph theory. Res Synth Methods. 2012;3(4):312-324.
Hasselblad V. Meta-analysis of multi-treatment studies. Med Decis Making Int J Soc Med Decis Making. 1998;18(1):37-43.
Rücker G, Krahn U, König J, Efthimiou O, Schwarzer G. netmeta: network meta-analysis using frequentist methods, R package version 1.3-0; 2021. https://CRAN.R-project.org/package=netmeta.
Gupta AK, Paquet M. Network meta-analysis of the outcome 'participant complete clearance' in nonimmunosuppressed participants of eight interventions for actinic keratosis: a follow-up on a Cochrane review. British J Dermatol. 2013;169(2):250-259.
Sciarretta S, Palano F, Tocci G, Baldini R, Volpe M. Antihypertensive treatment and development of heart failure in hypertension: a Bayesian network meta-analysis of studies in patients with hypertension and high cardiovascular risk. Arch Int Med. 2011;171(5):384-394.
Rücker G, Schwarzer G. Reduce dimension or reduce weights? Comparing two approaches to multi-arm studies in network meta-analysis. Stat Med. 2014;33(25):4353-4369.
Schwarzer G, Carpenter JR, Rücker G. Meta-Analysis with R (Use-R!). Cham, Switzerland: Springer International Publishing; 2015.
DerSimonian R, Laird N. Meta-analysis in clinical trials. Control Clin Trials. 1986;7(3):177-188.
Jackson D, White IR, Riley RD. Quantifying the impact of between-study heterogeneity in multivariate meta-analyses. Stat Med. 2012;31(29):3805-3820.
Rousseeuw PJ. Least median of squares regression. J Am Stat Assoc. 1984;79(388):871-880.
Chatterjee S, Hadi AS. Regression Analysis by Example. 5th ed. Hoboken, NJ: Wiley; 2013.
Cook RD, Weisberg S. Residuals and Influence in Regression. New York, NY: Chapman & Hall; 1982.
Krahn U, Binder H, König J. A graphical tool for locating inconsistency in network meta-analyses. BMC Med Res Methodol. 2013;13:35.
White IR, Barrett JK, Jackson D, Higgins JPT. Consistency and inconsistency in network meta-analysis: model estimation using multivariate meta-regression. Res Synth Methods. 2012;3(2):111-125.
Jackson D, Barrett JK, Rice S, White IR, Higgins JPT. A design-by-treatment interaction model for network meta-analysis with random inconsistency effects. Stat Med. 2014;33(21):3639-3654.
Dias S, Welton NJ, Caldwell DM, Ades AE. Checking consistency in mixed treatment comparison meta-analysis. Stat Med. 2010;29(7-8):932-944.
Kontopantelis E, Reeves D. Performance of statistical methods for meta-analysis when true study effects are non-normally distributed: a simulation study. Stat Methods Med Res. 2012;21(4):409-426.
Brockwell SE, Gordon IR. A comparison of statistical methods for meta-analysis. Stat Med. 2001;20(6):825-840.
Filzmoser P. Identification of multivariate outliers: a performance study. Austr J Stat. 2005;34(2):127-138.
Knight NL, Wang J. A comparison of outlier detection procedures and robust estimation methods in GPS positioning. J Navigat. 2009;62(4):699-709.
Hardin J, Rocke DM. Outlier detection in the multiple cluster setting using the minimum covariance determinant estimator. Comput Stat Data Anal. 2004;44(4):625-638.
Johansen S, Nielsen B. Asymptotic analysis of the forward search. Soc Sci Res Netw Electron J. 2013. https://ideas.repec.org/p/nuf/econwp/1302.html.
Nielsen B. ForwardSearch: forward search using asymptotic theory, R package version 1.0; 2014. https://CRAN.R-project.org/package=ForwardSearch.