UZH-Logo

Maintenance Infos

Controlling the danger of false discoveries in estimating multiple treatment effects


Wunderli, Dan (2012). Controlling the danger of false discoveries in estimating multiple treatment effects. Working paper series / Department of Economics 60, University of Zurich.

Abstract

I expose the risk of false discoveries in the context of multiple treatment effects. A false discovery is a nonexistent effect that is falsely labeled as statistically significant by its individual t-value. Labeling nonexistent effects as statistically significant has wide-ranging academic and policy-related implications, like costly false conclusions from policy evaluations. I eexamine an empirical labor market model by using state-of-the art multiple testing methods and I provide simulation evidence. By merely using individual t-values at conventional significance levels, the risk of labeling probably nonexistent treatment effects as statistically significant is unacceptably high. Individual t-values even label a number of treatment effects as significant, whereas multiple testing indicates false discoveries in these cases. Tests of a joint null hypothesis such as the well-known F-test control the risk of false discoveries only to a limited extent and do not optimally allow for rejecting individual hypotheses. Multiple testing methods control the risk of false discoveries in general while allowing for individual decisions in the sense of rejecting individual hypotheses.

I expose the risk of false discoveries in the context of multiple treatment effects. A false discovery is a nonexistent effect that is falsely labeled as statistically significant by its individual t-value. Labeling nonexistent effects as statistically significant has wide-ranging academic and policy-related implications, like costly false conclusions from policy evaluations. I eexamine an empirical labor market model by using state-of-the art multiple testing methods and I provide simulation evidence. By merely using individual t-values at conventional significance levels, the risk of labeling probably nonexistent treatment effects as statistically significant is unacceptably high. Individual t-values even label a number of treatment effects as significant, whereas multiple testing indicates false discoveries in these cases. Tests of a joint null hypothesis such as the well-known F-test control the risk of false discoveries only to a limited extent and do not optimally allow for rejecting individual hypotheses. Multiple testing methods control the risk of false discoveries in general while allowing for individual decisions in the sense of rejecting individual hypotheses.

Downloads

1007 downloads since deposited on 01 Feb 2012
7 downloads since 12 months
Detailed statistics

Additional indexing

Item Type:Working Paper
Communities & Collections:03 Faculty of Economics > Department of Economics
Working Paper Series > Department of Economics
Dewey Decimal Classification:330 Economics
JEL Classification:C12, C14, C21, C31, C41, J08, J64
Uncontrolled Keywords:False discoveries, multiple error rates, multiple treatment effects, labor market
Language:English
Date:January 2012
Deposited On:01 Feb 2012 11:40
Last Modified:05 Apr 2016 15:32
Series Name:Working paper series / Department of Economics
Number of Pages:18
ISSN:1664-7041 (P) 1664-705X (E)
Official URL:http://www.econ.uzh.ch/static/wp/econwp060.pdf
Related URLs:http://www.econ.uzh.ch/static/workingpapers.php
Permanent URL: https://doi.org/10.5167/uzh-57865

Download

[img]
Preview
Filetype: PDF
Size: 224kB

TrendTerms

TrendTerms displays relevant terms of the abstract of this publication and related documents on a map. The terms and their relations were extracted from ZORA using word statistics. Their timelines are taken from ZORA as well. The bubble size of a term is proportional to the number of documents where the term occurs. Red, orange, yellow and green colors are used for terms that occur in the current document; red indicates high interlinkedness of a term with other terms, orange, yellow and green decreasing interlinkedness. Blue is used for terms that have a relation with the terms in this document, but occur in other documents.
You can navigate and zoom the map. Mouse-hovering a term displays its timeline, clicking it yields the associated documents.

Author Collaborations