High replicability of newly-discovered social-behavioral findings is achievable.
Failures to replicate evidence of new discoveries have forced scientists to ask whether this unreliability is due to suboptimal implementation of optimal methods or whether presumptively optimal methods are not, in fact, optimal. This paper reports an investigation by four coordinated laboratories of the prospective replicability of 16 novel experimental findings using current optimal practices: high statistical power, preregistration, and complete methodological transparency. In contrast to past systematic replication efforts that reported replication rates averaging 50%, replication attempts here produced the expected effects with significance testing (p<.05) in 86% of attempts, slightly exceeding maximum expected replicability based on observed effect size and sample size. When one lab attempted to replicate an effect discovered by another lab, the effect size in the replications was 97% that of the original study. This high replication rate justifies confidence in rigor enhancing methods and suggests that past failures to replicate may be attributable to departures from optimal procedures.
Back to topCitation
@article{protzko2023,
author = {Protzko, John and Krosnick, Jon and Nelson, Leif and Nosek,
Brian and Axt, Jordan and Berent, Matt and Buttrick, Nicholas and
DeBell, Matthew and Ebersole, Charles R. and Lundmark, Sebastian and
MacInnis, Bo and O’Donnell, Michael and Perfecto, Hannah and
Pustejovsky, James E. and Roeder, Scott and Walleczek, Jan and
Schooler, Jonathan W.},
title = {High Replicability of Newly-Discovered Social-Behavioral
Findings Is Achievable.},
journal = {Nature Human Behavior},
volume = {8},
pages = {311-319},
date = {2023-11-09},
url = {https://doi.org/10.1038/s41562-023-01749-9},
doi = {10.1016/j.jsp.2018.02.003},
langid = {en}
}