Header

UZH-Logo

Maintenance Infos

Impact of the matching algorithm on the treatment effect estimate: A neutral comparison study


Heinz, Priska; Wendel‐Garcia, Pedro David; Held, Ulrike (2024). Impact of the matching algorithm on the treatment effect estimate: A neutral comparison study. Biometrical Journal, 66(1):2100292.

Abstract

Propensity score matching is increasingly being used in the medical literature. Choice of matching algorithms, reporting quality, and estimands are oftentimes not discussed. We evaluated the impact of propensity score matching algorithms, based on a recent clinical dataset, with three commonly used outcomes. The resulting estimands for different strengths of treatment effects were compared in a neutral comparison study and based on a thoroughly designed simulation study. Different algorithms yielded different levels of balance after matching. Along with full matching and genetic matching with replacement, good balance was achieved with nearest neighbor matching with caliper but thereby more than one fifth of the treated units were discarded. Average marginal treatment effect estimates were least biased with genetic or nearest neighbor matching, both with replacement and full matching. Double adjustment yielded conditional treatment effects that were closer to the true values, throughout. The choice of the matching algorithm had an impact on covariate balance after matching as well as treatment effect estimates. In comparison, genetic matching with replacement yielded better covariate balance than all other matching algorithms. A literature review in the British Medical Journal including its subjournals revealed frequent use of propensity score matching; however, the use of different matching algorithms before treatment effect estimation was only reported in one out of 21 studies. Propensity score matching is a methodology for causal treatment effect estimation from observational data; however, the methodological difficulties and low reporting quality in applied medical research need to be addressed.

Abstract

Propensity score matching is increasingly being used in the medical literature. Choice of matching algorithms, reporting quality, and estimands are oftentimes not discussed. We evaluated the impact of propensity score matching algorithms, based on a recent clinical dataset, with three commonly used outcomes. The resulting estimands for different strengths of treatment effects were compared in a neutral comparison study and based on a thoroughly designed simulation study. Different algorithms yielded different levels of balance after matching. Along with full matching and genetic matching with replacement, good balance was achieved with nearest neighbor matching with caliper but thereby more than one fifth of the treated units were discarded. Average marginal treatment effect estimates were least biased with genetic or nearest neighbor matching, both with replacement and full matching. Double adjustment yielded conditional treatment effects that were closer to the true values, throughout. The choice of the matching algorithm had an impact on covariate balance after matching as well as treatment effect estimates. In comparison, genetic matching with replacement yielded better covariate balance than all other matching algorithms. A literature review in the British Medical Journal including its subjournals revealed frequent use of propensity score matching; however, the use of different matching algorithms before treatment effect estimation was only reported in one out of 21 studies. Propensity score matching is a methodology for causal treatment effect estimation from observational data; however, the methodological difficulties and low reporting quality in applied medical research need to be addressed.

Statistics

Citations

Dimensions.ai Metrics
5 citations in Web of Science®
4 citations in Scopus®
Google Scholar™

Altmetrics

Downloads

58 downloads since deposited on 22 Nov 2022
41 downloads since 12 months
Detailed statistics

Additional indexing

Item Type:Journal Article, refereed, original work
Communities & Collections:04 Faculty of Medicine > University Hospital Zurich > Institute of Intensive Care Medicine
04 Faculty of Medicine > Epidemiology, Biostatistics and Prevention Institute (EBPI)
Dewey Decimal Classification:610 Medicine & health
Scopus Subject Areas:Physical Sciences > Statistics and Probability
Social Sciences & Humanities > Statistics, Probability and Uncertainty
Uncontrolled Keywords:Statistics, Probability and Uncertainty, General Medicine, Statistics and Probability, Monte Carlo simulation; causal treatment effect; matching algorithm; neutral comparison; propensity score
Language:English
Date:January 2024
Deposited On:22 Nov 2022 10:01
Last Modified:29 Mar 2024 02:35
Publisher:Wiley-Blackwell Publishing, Inc.
ISSN:0323-3847
OA Status:Hybrid
Publisher DOI:https://doi.org/10.1002/bimj.202100292
PubMed ID:35385172
  • Content: Published Version
  • Language: English
  • Licence: Creative Commons: Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0)