Preferred Name
Heather Dawn Harris
Creative Commons License
This work is licensed under a Creative Commons Attribution-Noncommercial-No Derivative Works 4.0 License.
Date of Graduation
Spring 2015
Document Type
Thesis
Degree Name
Master of Arts (MA)
Department
Department of Graduate Psychology
Advisor(s)
S. Jeanne Horst
Abstract
The applied nature of higher education assessment does not lend itself to rigorous experimental research designs. However, assessment practitioners would like to make claims about the influence of educational programs on student learning outcomes. Propensity score matching (PSM) methods are quasi-experimental techniques that allow researchers to control for known confounding variables. In the context of higher education, PSM techniques allow assessment practitioners to control for confounding variables related to students’ self-selected participation in university programs. Research and recommendations on how to apply PSM techniques are scattered throughout several disciplines. However, additional research is needed to evaluate how well PSM techniques control self-selection bias in the context of educational assessment. To couch PSM techniques within the framework of higher education assessment, the current study first summarized common practices and recommendations from literature across several disciplines, then evaluated the application of common PSM techniques via an applied example of honors program assessment. Specifically, the study applied common PSM techniques to compare students in the honors program with students either not invited into the program or students who decided not to participate in the program. Data analyses four research questions: 1) Do honors students differ from students not in the honors program on motivation variables?; 2) How well do different common PSM techniques create quality comparison groups of students?; 3) How well do different common PSM techniques retain honors students in the comparison of program outcomes?; and 4) Do honors students differ from students not in the honors program on outcomes after PSM techniques are applied? Honors students did not differ from students not in the honors program on motivation variables thought to be related to self-selection into the program. Across matching methods applied in the current study, the quality of propensity score matches was near optimal. Average scores on the outcomes of interest to the honors program did not significantly differ by group. However, decreased sample sizes resulted in a loss of minority student representation in the honors sample and different practically significant results on program outcomes. Recommendations and implications for applied assessment practitioners are offered.
Recommended Citation
Harris, Heather D., "Propensity score matching in higher education assessment" (2015). Masters Theses, 2010-2019. 55.
https://commons.lib.jmu.edu/master201019/55
