Date of Award

Spring 2015

Document Type

Dissertation

Degree Name

Doctor of Philosophy (PhD)

Department

Department of Graduate Psychology

Advisor(s)

Donna L. Sundre

Abstract

Questions regarding the quality of education, both in K-12 systems and higher education, are common. Methods for measuring quality in education have been developed in the past decades, with value-added estimates emerging as one of the most well-known methods. Value-added methods purport to indicate how much students learn over time as a result of their attendance at a particular school. Controversy has surrounded the algorithms used to generate value-added estimates as well as the uses of the estimates to make decisions about school and teacher quality. In higher education, most institutions used cross-sectional rather than longitudinal data to estimate value-added. In addition, much of the data used to generate value-added estimates in higher education were gathered in low-stakes testing sessions. In low-stakes contexts, examinee motivation has been shown to impact test performance. Additionally, recent empirical evidence indicated that the change in test-taking motivation between pre-and post-test was a predictor of change in performance. Because of this, researchers have suggested that test-taking motivation may bias value-added estimates. Further, if interest truly lies in measuring student learning over time, the use of cross-sectional data is problematic, since the pre- and post-test data is gathered from two different groups of students, not the same students at two time points. The current study investigated two overarching questions related to value-added estimation in higher education: 1) are different methods of value-added estimation comparable?; and 2) how does test-taking motivation impact value-added estimates? In this study, first the results from value-added estimates calculated with cross-sectional and longitudinal data were compared. Next, estimates generated from two value-added models were compared: raw difference scores and a longitudinal hierarchical linear model. Finally, estimates were compared when motivation variables were included. Results indicated that at the institution under study, cross-sectional and longitudinal data and analyses yielded similar results and that changes in test-taking motivation between pre- and post-test did impact value-added estimates. Suggestions to combat the effect of motivation on value-added estimates included behavioral as well as statistical interventions.

Share

COinS
 
 

To view the content in your browser, please download Adobe Reader or, alternately,
you may Download the file to your hard drive.

NOTE: The latest versions of Adobe Reader do not support viewing PDF files within Firefox on Mac OS and if you are using a modern (Intel) Mac, there is no official plugin for viewing PDF files within the browser window.