Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-No Derivative Works 4.0 International License.
ORCID
https://orcid.org/0000-0003-1869-7021
Date of Graduation
5-7-2020
Semester of Graduation
Spring
Degree Name
Master of Arts (MA)
Department
Department of Graduate Psychology
Second Advisor
Dena Pastor
Third Advisor
Christine DeMars
Abstract
In the absence of random assignment, researchers must consider the impact of selection bias – pre-existing covariate differences between groups due to differences among those entering into treatment and those otherwise unable to participate. Propensity score matching (PSM) and generalized boosted modeling (GBM) are two quasi-experimental pre-processing methods that strive to reduce the impact of selection bias before analyzing a treatment effect. PSM and GBM both examine a treatment and comparison group and either match or weight members of those groups to create new, balanced groups. The new, balanced groups theoretically can then be used as a proxy for the balanced groups achieved via random assignment. However, in order to successfully employ GBM and PSM, researchers must properly specify the models used to reduce selection bias. Not only do researchers need to account for all covariates related to bias, but they also need to properly specify polynomial terms or interactions. This study investigated scenarios where either a quadratic term or an interaction term contributed to selection bias, and questioned: (1) how incorrectly specified PSM models, correctly specified PSM models, and GBM approaches compare in their ability to create balanced treatment and comparison groups; and (2) how much these methods reduce treatment effect estimation bias. Ultimately, this study found that PSM methods achieved adequate balance, even when misspecified to omit an interaction or quadradic term. In terms of reducing bias, the correctly specified PSM model performed the best, followed by the incorrectly specified PSM model and then the GBM model. All methods had a more accurate treatment effect estimate than the baseline model, which included no pre-processing for selection bias. Recommendations and implications are offered for researchers.
