Norges Bank

Working Paper

Non-standard errors

Author:
Albert J. Menkveld et al. Norges Bank co-authors: Saskia Ter Ellen and Hans Christian Wika.
Series:
Working Paper
Number:
13/2021

In statistics, samples are drawn from a population in a datagenerating process (DGP). Standard errors measure the uncertainty in sample estimates of population parameters. In science, evidence is generated to test hypotheses in an evidencegenerating process (EGP). We claim that EGP variation across researchers adds uncertainty: non-standard errors. To study them, we let 164 teams test six hypotheses on the same sample. We find that non-standard errors are sizeable, on par with standard errors. Their size (i) co-varies only weakly with team merits, reproducibility, or peer rating, (ii) declines significantly after peer-feedback, and (iii) is underestimated by participants.

Working Papers inneholder forskningsarbeider og utredninger som vanligvis ikke har fått sin endelige form. Også andre faglige analyser fra økonomer i Norges Bank utgis i serien. Synspunkter og konklusjoner i arbeidene står for forfatternes regning.

Norges Bank Working Papers distribueres også gjennom RepEc og BIS Central Bank Research Hub.

ISSN 1502-8190 (online)

Published 23 November 2021 15:00
Published 23 November 2021 15:00