Header

UZH-Logo

Maintenance Infos

The success of linear bootstrapping models: Decision domain-, expertise-, and criterion-specific meta-analysis


Kaufmann, Esther; Wittmann, Werner W (2016). The success of linear bootstrapping models: Decision domain-, expertise-, and criterion-specific meta-analysis. PLoS ONE, 11(6):e0157914.

Abstract

The success of bootstrapping or replacing a human judge with a model (e.g., an equation) has been demonstrated in Paul Meehl’s (1954) seminal work and bolstered by the results of several meta-analyses. To date, however, analyses considering different types of meta-analyses as well as the potential dependence of bootstrapping success on the decision domain, the level of expertise of the human judge, and the criterion for what constitutes an accurate decision have been missing from the literature. In this study, we addressed these research gaps by conducting a meta-analysis of lens model studies. We compared the results of a traditional (bare-bones) meta-analysis with findings of a meta-analysis of the success of bootstrap models corrected for various methodological artifacts. In line with previous studies, we found that bootstrapping was more successful than human judgment. Furthermore, bootstrapping was more successful in studies with an objective decision criterion than in studies with subjective or test score criteria. We did not find clear evidence that the success of bootstrapping depended on the decision domain (e.g., education or medicine) or on the judge’s level of expertise (novice or expert). Correction of methodological artifacts increased the estimated success of bootstrapping, suggesting that previous analyses without artifact correction (i.e., traditional meta-analyses) may have underestimated the value of bootstrapping models.

Abstract

The success of bootstrapping or replacing a human judge with a model (e.g., an equation) has been demonstrated in Paul Meehl’s (1954) seminal work and bolstered by the results of several meta-analyses. To date, however, analyses considering different types of meta-analyses as well as the potential dependence of bootstrapping success on the decision domain, the level of expertise of the human judge, and the criterion for what constitutes an accurate decision have been missing from the literature. In this study, we addressed these research gaps by conducting a meta-analysis of lens model studies. We compared the results of a traditional (bare-bones) meta-analysis with findings of a meta-analysis of the success of bootstrap models corrected for various methodological artifacts. In line with previous studies, we found that bootstrapping was more successful than human judgment. Furthermore, bootstrapping was more successful in studies with an objective decision criterion than in studies with subjective or test score criteria. We did not find clear evidence that the success of bootstrapping depended on the decision domain (e.g., education or medicine) or on the judge’s level of expertise (novice or expert). Correction of methodological artifacts increased the estimated success of bootstrapping, suggesting that previous analyses without artifact correction (i.e., traditional meta-analyses) may have underestimated the value of bootstrapping models.

Statistics

Altmetrics

Downloads

10 downloads since deposited on 11 Oct 2016
9 downloads since 12 months
Detailed statistics

Additional indexing

Item Type:Journal Article, refereed, original work
Communities & Collections:06 Faculty of Arts > Institute of Education
Dewey Decimal Classification:370 Education
Language:English
Date:2016
Deposited On:11 Oct 2016 14:28
Last Modified:30 Nov 2016 08:43
Publisher:Public Library of Science (PLoS)
ISSN:1932-6203
Free access at:Publisher DOI. An embargo period may apply.
Publisher DOI:https://doi.org/10.1371/journal.pone.0157914

Download

Download PDF  'The success of linear bootstrapping models: Decision domain-, expertise-, and criterion-specific meta-analysis'.
Preview
Content: Published Version
Language: English
Filetype: PDF
Size: 798kB
View at publisher