The Journal of Finance
Tarun Chordia, Francis Longstaff, Robert Korajczyk, Lubos Pastor, Mahendrarajah Nimalendran, Marti Subrahmanyam, Paolo Pasquariello, Neil Pearson, Ingrid Werner, Pradeep Yadav, Lawrence Glosten, Bidisha Chakrabarty, Terrence Hendershott, Robert Van Ness, Lawrence Eugene Harris, Oleg Bondarenko, Andrew Patton, Dacheng Xiu, Ann Marie Hibbert, Dmitriy Muravyev, Allen Carrion, James Angel, Alex Horenstein, Subramanian Iyer, David Rakowski, Jeffrey Harris, Dermot Murphy, Mikhail Chernov, Shawn Mankad, Davidson Heath, Dominik Maximilian Roesch, Yuehua Tang, Konstantin Sokolov, Jeffrey Black, Brian Roseman, Marcus Painter, Michael Farrell, Edwin Baidoo, Alejandro Lopez Lira, Chukwuma Dim, Woongsun Yoo, Da Huang, Andriy Shkilko
Department of Finance
Abstract
In statistics, samples are drawn from a population in a data?generating process (DGP). Standard errors measure the uncertainty in estimates of population parameters. In science, evidence is generated to test hypotheses in an evidence?generating process (EGP). We claim that EGP variation across researchers adds uncertainty?nonstandard errors (NSEs). We study NSEs by letting 164 teams test the same hypotheses on the same data. NSEs turn out to be sizable, but smaller for more reproducible or higher rated research. Adding peer?review stages reduces NSEs. We further find that this type of uncertainty is underestimated by participants.
