Non-standard errors
In statistics, samples are drawn from a population in a data-generating process
(DGP). Standard errors measure the uncertainty in sample estimates of population
parameters. In science, evidence is generated to test hypotheses in an
evidence-generating process (EGP). We claim that EGP variation across researchers adds
uncertainty: non-standard errors. To study them, we let 164 teams test six hypotheses
on the same sample. We find that non-standard errors are sizeable, on par with
standard errors. Their size (i) co-varies only weakly with team merits,
reproducibility, or peer rating, (ii) declines significantly after peer-feedback, and
(iii) is underestimated by participants.
Nov 2021
The results of the #fincap project
Paper
Web links
- The paper at SSRN
- Web presentation of the #fincap project