Creative Commons License

Creative Commons License
This work is licensed under a Creative Commons Attribution-Noncommercial-No Derivative Works 4.0 License.

ORCID

https://orcid.org/0000-0003-1869-7021

Date of Graduation

5-7-2020

Document Type

Thesis

Degree Name

Master of Arts (MA)

Department

Department of Graduate Psychology

Advisor(s)

S. Jeanne Horst

Dena Pastor

Christine DeMars

Abstract

In the absence of random assignment, researchers must consider the impact of selection bias – pre-existing covariate differences between groups due to differences among those entering into treatment and those otherwise unable to participate. Propensity score matching (PSM) and generalized boosted modeling (GBM) are two quasi-experimental pre-processing methods that strive to reduce the impact of selection bias before analyzing a treatment effect. PSM and GBM both examine a treatment and comparison group and either match or weight members of those groups to create new, balanced groups. The new, balanced groups theoretically can then be used as a proxy for the balanced groups achieved via random assignment. However, in order to successfully employ GBM and PSM, researchers must properly specify the models used to reduce selection bias. Not only do researchers need to account for all covariates related to bias, but they also need to properly specify polynomial terms or interactions. This study investigated scenarios where either a quadratic term or an interaction term contributed to selection bias, and questioned: (1) how incorrectly specified PSM models, correctly specified PSM models, and GBM approaches compare in their ability to create balanced treatment and comparison groups; and (2) how much these methods reduce treatment effect estimation bias. Ultimately, this study found that PSM methods achieved adequate balance, even when misspecified to omit an interaction or quadradic term. In terms of reducing bias, the correctly specified PSM model performed the best, followed by the incorrectly specified PSM model and then the GBM model. All methods had a more accurate treatment effect estimate than the baseline model, which included no pre-processing for selection bias. Recommendations and implications are offered for researchers.

Available for download on Thursday, April 15, 2021

Share

COinS