Header

UZH-Logo

Maintenance Infos

A new goodness of fit test for event forecasting and its application to credit default


Leippold, Markus; Bloechlinger, Andreas (2011). A new goodness of fit test for event forecasting and its application to credit default. Management Science, 57(3):487-505.

Abstract

We develop a new goodness-of-fit test for validating the performance of probability forecasts. Our test statistic is particularly powerful under sparseness and dependence in the observed data. To build our test statistic, we start from a formal definition of calibrated forecasts, which we operationalize by introducing two components. The first component tests the level of the estimated probabilities. The second component validates the shape, measuring the differentiation between high and low robability events. After constructing test statistics for both level and shape, we provide a global goodness-of-fit statistic, which is asymptotically x^2 distributed. In a simulation exercise, we find that our approach is correctly sized and more powerful than alternative statistics. In particular, our shape statistic is significantly more powerful than the Kolmogorov-Smirnov test. Under independence our global test has significantly greater power than the popular Hosmer and Lemeshow's x^2 test. Moreover, even under dependence our global test remains correctly sized and consistent. As a timely and important empirical application of our method, we study the validation of a forecasting model for credit default events.

Abstract

We develop a new goodness-of-fit test for validating the performance of probability forecasts. Our test statistic is particularly powerful under sparseness and dependence in the observed data. To build our test statistic, we start from a formal definition of calibrated forecasts, which we operationalize by introducing two components. The first component tests the level of the estimated probabilities. The second component validates the shape, measuring the differentiation between high and low robability events. After constructing test statistics for both level and shape, we provide a global goodness-of-fit statistic, which is asymptotically x^2 distributed. In a simulation exercise, we find that our approach is correctly sized and more powerful than alternative statistics. In particular, our shape statistic is significantly more powerful than the Kolmogorov-Smirnov test. Under independence our global test has significantly greater power than the popular Hosmer and Lemeshow's x^2 test. Moreover, even under dependence our global test remains correctly sized and consistent. As a timely and important empirical application of our method, we study the validation of a forecasting model for credit default events.

Statistics

Citations

3 citations in Web of Science®
5 citations in Scopus®
Google Scholar™

Altmetrics

Additional indexing

Item Type:Journal Article, refereed, original work
Communities & Collections:03 Faculty of Economics > Department of Banking and Finance
Dewey Decimal Classification:330 Economics
Language:English
Date:March 2011
Deposited On:11 Apr 2013 09:05
Last Modified:05 Apr 2016 16:28
Publisher:Institute for Operations Research and the Management Science
ISSN:0025-1909
Publisher DOI:https://doi.org/10.1287/mnsc.1100.1283
Official URL:http://pubsonline.informs.org/doi/abs/10.1287/mnsc.1100.1283
Other Identification Number:merlin-id:4459

Download

Full text not available from this repository.
View at publisher