Header

UZH-Logo

Maintenance Infos

Multiple Score Comparison: a network meta-analysis approach to comparison and external validation of prognostic scores


Haile, Sarah R; Guerra, Beniamino; Soriano, Joan B; Puhan, Milo A (2017). Multiple Score Comparison: a network meta-analysis approach to comparison and external validation of prognostic scores. BMC Medical Research Methodology, 17(1):172.

Abstract

BACKGROUND Prediction models and prognostic scores have been increasingly popular in both clinical practice and clinical research settings, for example to aid in risk-based decision making or control for confounding. In many medical fields, a large number of prognostic scores are available, but practitioners may find it difficult to choose between them due to lack of external validation as well as lack of comparisons between them. METHODS Borrowing methodology from network meta-analysis, we describe an approach to Multiple Score Comparison meta-analysis (MSC) which permits concurrent external validation and comparisons of prognostic scores using individual patient data (IPD) arising from a large-scale international collaboration. We describe the challenges in adapting network meta-analysis to the MSC setting, for instance the need to explicitly include correlations between the scores on a cohort level, and how to deal with many multi-score studies. We propose first using IPD to make cohort-level aggregate discrimination or calibration scores, comparing all to a common comparator. Then, standard network meta-analysis techniques can be applied, taking care to consider correlation structures in cohorts with multiple scores. Transitivity, consistency and heterogeneity are also examined. RESULTS We provide a clinical application, comparing prognostic scores for 3-year mortality in patients with chronic obstructive pulmonary disease using data from a large-scale collaborative initiative. We focus on the discriminative properties of the prognostic scores. Our results show clear differences in performance, with ADO and eBODE showing higher discrimination with respect to mortality than other considered scores. The assumptions of transitivity and local and global consistency were not violated. Heterogeneity was small. CONCLUSIONS We applied a network meta-analytic methodology to externally validate and concurrently compare the prognostic properties of clinical scores. Our large-scale external validation indicates that the scores with the best discriminative properties to predict 3 year mortality in patients with COPD are ADO and eBODE.

Abstract

BACKGROUND Prediction models and prognostic scores have been increasingly popular in both clinical practice and clinical research settings, for example to aid in risk-based decision making or control for confounding. In many medical fields, a large number of prognostic scores are available, but practitioners may find it difficult to choose between them due to lack of external validation as well as lack of comparisons between them. METHODS Borrowing methodology from network meta-analysis, we describe an approach to Multiple Score Comparison meta-analysis (MSC) which permits concurrent external validation and comparisons of prognostic scores using individual patient data (IPD) arising from a large-scale international collaboration. We describe the challenges in adapting network meta-analysis to the MSC setting, for instance the need to explicitly include correlations between the scores on a cohort level, and how to deal with many multi-score studies. We propose first using IPD to make cohort-level aggregate discrimination or calibration scores, comparing all to a common comparator. Then, standard network meta-analysis techniques can be applied, taking care to consider correlation structures in cohorts with multiple scores. Transitivity, consistency and heterogeneity are also examined. RESULTS We provide a clinical application, comparing prognostic scores for 3-year mortality in patients with chronic obstructive pulmonary disease using data from a large-scale collaborative initiative. We focus on the discriminative properties of the prognostic scores. Our results show clear differences in performance, with ADO and eBODE showing higher discrimination with respect to mortality than other considered scores. The assumptions of transitivity and local and global consistency were not violated. Heterogeneity was small. CONCLUSIONS We applied a network meta-analytic methodology to externally validate and concurrently compare the prognostic properties of clinical scores. Our large-scale external validation indicates that the scores with the best discriminative properties to predict 3 year mortality in patients with COPD are ADO and eBODE.

Statistics

Citations

Dimensions.ai Metrics
2 citations in Web of Science®
1 citation in Scopus®
2 citations in Microsoft Academic
Google Scholar™

Altmetrics

Downloads

6 downloads since deposited on 19 Jan 2018
6 downloads since 12 months
Detailed statistics

Additional indexing

Contributors:3CIA collaboration
Item Type:Journal Article, refereed, original work
Communities & Collections:04 Faculty of Medicine > Epidemiology, Biostatistics and Prevention Institute (EBPI)
Dewey Decimal Classification:610 Medicine & health
Language:English
Date:21 December 2017
Deposited On:19 Jan 2018 10:16
Last Modified:19 Feb 2018 10:23
Publisher:BioMed Central
ISSN:1471-2288
OA Status:Gold
Free access at:PubMed ID. An embargo period may apply.
Publisher DOI:https://doi.org/10.1186/s12874-017-0433-2
Official URL:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5740913/
PubMed ID:29268701

Download

Download PDF  'Multiple Score Comparison: a network meta-analysis approach to comparison and external validation of prognostic scores'.
Preview
Content: Published Version
Filetype: PDF
Size: 955kB
View at publisher
Licence: Creative Commons: Attribution 4.0 International (CC BY 4.0)