The process of replication target selection in psychology: what to consider?

consensus replication study selection

Journal

Royal Society open science
ISSN: 2054-5703
Titre abrégé: R Soc Open Sci
Pays: England
ID NLM: 101647528

Informations de publication

Date de publication:
Feb 2023
Historique:
received: 07 04 2021
accepted: 13 01 2023
entrez: 9 2 2023
pubmed: 10 2 2023
medline: 10 2 2023
Statut: epublish

Résumé

Increased execution of replication studies contributes to the effort to restore credibility of empirical research. However, a second generation of problems arises: the number of potential replication targets is at a serious mismatch with available resources. Given limited resources, replication target selection should be well-justified, systematic and transparently communicated. At present the discussion on what to consider when selecting a replication target is limited to theoretical discussion, self-reported justifications and a few formalized suggestions. In this Registered Report, we proposed a study involving the scientific community to create a list of considerations for consultation when selecting a replication target in psychology. We employed a modified Delphi approach. First, we constructed a preliminary list of considerations. Second, we surveyed psychologists who previously selected a replication target with regards to their considerations. Third, we incorporated the results into the preliminary list of considerations and sent the updated list to a group of individuals knowledgeable about concerns regarding replication target selection. Over the course of several rounds, we established consensus regarding what to consider when selecting a replication target. The resulting checklist can be used for transparently communicating the rationale for selecting studies for replication.

Identifiants

pubmed: 36756069
doi: 10.1098/rsos.210586
pii: rsos210586
pmc: PMC9890109
doi:

Banques de données

figshare
['10.6084/m9.figshare.c.6403449']

Types de publication

Journal Article

Langues

eng

Pagination

210586

Informations de copyright

© 2023 The Authors.

Références

Science. 2015 Aug 28;349(6251):aac4716
pubmed: 26315443
Health Technol Assess. 1998;2(3):i-iv, 1-88
pubmed: 9561895
Acad Med. 2016 May;91(5):663-8
pubmed: 26796090
PLoS Med. 2005 Aug;2(8):e124
pubmed: 16060722
Perspect Psychol Sci. 2017 May;12(3):527-542
pubmed: 28475467
Behav Brain Sci. 2018 Jan;41:e157
pubmed: 31064530
BMC Psychol. 2016 Jun 01;4(1):30
pubmed: 27251381
Perspect Psychol Sci. 2012 Nov;7(6):537-42
pubmed: 26168110
Aust N Z J Psychiatry. 2015 Oct;49(10):887-97
pubmed: 26296368
Behav Brain Sci. 2018 Jan;41:e139
pubmed: 31064529
Elife. 2014 Dec 10;3:
pubmed: 25490932
Nat Hum Behav. 2020 Jan;4(1):4-6
pubmed: 31792401
J Exp Psychol Gen. 2016 Dec;145(12):e81-e95
pubmed: 27935731
R Soc Open Sci. 2020 Apr 1;7(4):181351
pubmed: 32431853
J Exp Psychol Gen. 2014 Oct;143(5):1794-805
pubmed: 24933517
Perspect Psychol Sci. 2012 Nov;7(6):657-60
pubmed: 26168127
Perspect Psychol Sci. 2012 Nov;7(6):632-8
pubmed: 26168122
J Adv Nurs. 1994 Jun;19(6):1221-5
pubmed: 7930104
PLoS One. 2020 Jul 22;15(7):e0235905
pubmed: 32697809
Neurotherapeutics. 2017 Jul;14(3):537-545
pubmed: 28653278
PLoS One. 2015 Aug 13;10(8):e0135162
pubmed: 26270647
J Clin Epidemiol. 2019 Apr;108:110-120
pubmed: 30557677
Perspect Psychol Sci. 2011 Mar;6(2):163-71
pubmed: 26162135
R Soc Open Sci. 2021 May 19;8(5):201697
pubmed: 34017596
Psychol Sci. 2011 Nov;22(11):1359-66
pubmed: 22006061
Behav Brain Sci. 2018 Jan;41:e124
pubmed: 31064512
Sports Med. 2023 Jan;53(1):281-291
pubmed: 36066754
Perspect Psychol Sci. 2012 Nov;7(6):528-30
pubmed: 26168108
Psychol Methods. 2021 Dec 20;:
pubmed: 34928679
Behav Brain Sci. 2017 Oct 25;41:e120
pubmed: 29065933
Behav Brain Sci. 2018 Jan;41:e132
pubmed: 31064517
J Pers Soc Psychol. 2017 Aug;113(2):254-261
pubmed: 28714731

Auteurs

Merle-Marie Pittelkow (MM)

Department of Psychometrics and Statistics, Rijksuniversiteit Groningen, Groningen, The Netherlands.

Sarahanne M Field (SM)

Centre of Science and Technology Studies, Leiden University, Leiden, the Netherlands.

Peder M Isager (PM)

Oslo New University College, Oslo, Norway.

Anna E Van't Veer (AE)

Methodology and Statistics Unit, Institute of Psychology, Leiden University, Leiden, the Netherlands.

Thomas Anderson (T)

Department of Psychology, University of Toronto, Toronto, Canada.

Scott N Cole (SN)

School of Education, Language and Psychology, York St John University, York, UK.

Tomáš Dominik (T)

Institute for Interdisciplinary Brain and Behavioral Sciences, Chapman University, Orange, CA, USA.

Roger Giner-Sorolla (R)

School of Psychology, University of Kent, Canterbury, UK.

Sebahat Gok (S)

Program in Cognitive Science, Department of Instructional Systems Technology, Indiana University, Bloomington, IN, USA.

Tom Heyman (T)

Methodology and Statistics Unit, Institute of Psychology, Leiden University, Leiden, the Netherlands.

Marc Jekel (M)

Social Psychology, University of Cologne, Cologne, Germany.

Timothy J Luke (TJ)

Department of Psychology, University of Gothenburg, Gothenburg, Sweden.

David B Mitchell (DB)

WellStar College of Health and Human Services, Kennesaw State University, Kennesaw, GA, USA.

Rik Peels (R)

Philosophy Department and the Faculty of Religion and Theology, Vrije Universiteit, Amsterdam, The Netherlands.

Rosina Pendrous (R)

Centre for Contextual Behavioural Science, School of Psychology, University of Chester, Chester, UK.
Institute of Applied Health Research, University of Birmingham, Birmingham, UK.

Samuel Sarrazin (S)

Maison de santé pluridisciplinaire Pasteur, Chevilly-Larue, France.

Jacob M Schauer (JM)

Department of Preventive Medicine - Division of Biostatistics, Feinberg School of Medicine, Northwestern University, Chicago, IL, USA.

Eva Specker (E)

Department of Cognition, Emotion, and Methods in Psychology, Faculty of Psychology, University of Vienna, Vienna, Austria.

Ulrich S Tran (US)

Department of Cognition, Emotion, and Methods in Psychology, Faculty of Psychology, University of Vienna, Vienna, Austria.

Marek A Vranka (MA)

Faculty of Social Sciences, Charles University, Prague, Czech Republic.

Jelte M Wicherts (JM)

Department of Methodology and Statistics, Tilburg University, Tilburg, The Netherlands.

Naoto Yoshimura (N)

Research Organization of Open Innovation and Collaboration, Ritsumeikan University, Osaka, Japan.
Research Fellow of the Japan Society for the Promotion of Science, Erasmus University Rotterdam, Rotterdam, The Netherlands.

Rolf A Zwaan (RA)

Department of Psychology, Education, and Child Studies, Erasmus University Rotterdam, Rotterdam, The Netherlands.

Don van Ravenzwaaij (D)

Department of Psychometrics and Statistics, Rijksuniversiteit Groningen, Groningen, The Netherlands.

Classifications MeSH