UZH-Logo

Maintenance Infos

Control of Generalized Error Rates in Multiple Testing


Romano, Joseph P; Wolf, Michael (2005). Control of Generalized Error Rates in Multiple Testing. Working paper series / Institute for Empirical Research in Economics No. 245, University of Zurich.

Abstract

Consider the problem of testing s hypotheses simultaneously. The usual approach tondealing with the multiplicity problem is to restrict attention to procedures that controlnthe probability of even one false rejection, the familiar familywise error rate (FWER). Innmany applications, particularly if s is large, one might be willing to tolerate more than onenfalse rejection if the number of such cases is controlled, thereby increasing the ability of thenprocedure to reject false null hypotheses One possibility is to replace control of the FWERnby control of the probability of k or more false rejections, which is called the k-FWER.nWe derive both single-step and stepdown procedures that control the k-FWER in finitensamples or asymptotically, depending on the situation. Lehmann and Romano (2005a)nderive some exact methods for this purpose, which apply whenever p-values are availablenfor individual tests; no assumptions are made on the joint dependence of the p-values. Inncontrast, we construct methods that implicitly take into account the dependence structurenof the individual test statistics in order to further increase the ability to detect false nullnhypotheses. We also consider the false discovery proportion (FDP) defined as the numbernof false rejections divided by the total number of rejections (and defined to be 0 if therenare no rejections). The false discovery rate proposed by Benjamini and Hochberg (1995)ncontrols E(FDP).

Consider the problem of testing s hypotheses simultaneously. The usual approach tondealing with the multiplicity problem is to restrict attention to procedures that controlnthe probability of even one false rejection, the familiar familywise error rate (FWER). Innmany applications, particularly if s is large, one might be willing to tolerate more than onenfalse rejection if the number of such cases is controlled, thereby increasing the ability of thenprocedure to reject false null hypotheses One possibility is to replace control of the FWERnby control of the probability of k or more false rejections, which is called the k-FWER.nWe derive both single-step and stepdown procedures that control the k-FWER in finitensamples or asymptotically, depending on the situation. Lehmann and Romano (2005a)nderive some exact methods for this purpose, which apply whenever p-values are availablenfor individual tests; no assumptions are made on the joint dependence of the p-values. Inncontrast, we construct methods that implicitly take into account the dependence structurenof the individual test statistics in order to further increase the ability to detect false nullnhypotheses. We also consider the false discovery proportion (FDP) defined as the numbernof false rejections divided by the total number of rejections (and defined to be 0 if therenare no rejections). The false discovery rate proposed by Benjamini and Hochberg (1995)ncontrols E(FDP).

Downloads

412 downloads since deposited on 29 Nov 2011
82 downloads since 12 months
Detailed statistics

Additional indexing

Item Type:Working Paper
Communities & Collections:03 Faculty of Economics > Department of Economics
Working Paper Series > Institute for Empirical Research in Economics (former)
Dewey Decimal Classification:330 Economics
Language:English
Date:May 2005
Deposited On:29 Nov 2011 22:32
Last Modified:05 Apr 2016 15:11
Series Name:Working paper series / Institute for Empirical Research in Economics
ISSN:1424-0459
Official URL:http://www.econ.uzh.ch/wp.html
Permanent URL: https://doi.org/10.5167/uzh-52136

Download

[img]
Preview
Filetype: PDF
Size: 1MB

TrendTerms

TrendTerms displays relevant terms of the abstract of this publication and related documents on a map. The terms and their relations were extracted from ZORA using word statistics. Their timelines are taken from ZORA as well. The bubble size of a term is proportional to the number of documents where the term occurs. Red, orange, yellow and green colors are used for terms that occur in the current document; red indicates high interlinkedness of a term with other terms, orange, yellow and green decreasing interlinkedness. Blue is used for terms that have a relation with the terms in this document, but occur in other documents.
You can navigate and zoom the map. Mouse-hovering a term displays its timeline, clicking it yields the associated documents.

Author Collaborations