Creative Commons License
This work is licensed under a Creative Commons Attribution-Noncommercial-No Derivative Works 4.0 License.
ORCID
https://orcid.org/0000-0002-9985-6807
Date of Graduation
5-9-2024
Document Type
Dissertation
Degree Name
Doctor of Philosophy (PhD)
Department
Department of Graduate Psychology
Advisor(s)
Christine E. DeMars
Yu Bao
Brian C. Leventhal
Abstract
Inadequate examinee test-taking effort is especially concerning when the results of low-stakes tests are used to inform high-stakes policy or psychometric decisions. Although abundant is the literature on the negative impact of low test-taking effort (e.g., Wise & DeMars, 2005) and ways to address it (e.g., Alahmadi & DeMars, 2024; Lau et al., 2009), integrating it into assessment practices has long been in a rudimentary stage. The findings presented in this dissertation serve as a clarion call for testing professionals to reinforce efforts that bridge research and practice. The current study investigated how rapid guessing (RG)—a form of non-effortful test-taking behavior—biases the estimated item parameters (i.e., the discrimination and difficulty parameters). Previous studies disagreed on the direction of the bias and the contributing factors. Using simulation techniques to mimic data observed in real-world, low-stakes testing contexts, the current study reconciled the inconsistencies by identifying conditions under which item parameters are likely to be overestimated or underestimated, given several factors. Some factors represent item features (viz., the true values of item parameters and the number of RG responses on items) and others represent examinee features (viz., whether rapid guessers exhibit lower proficiency, the variability in RG behavior among rapid guessers, and the pattern of RG responses throughout the test). The current findings accentuate the consequences of ignoring rapid guessing by revealing implications not only in estimating proficiency scores but also in test operations. The dissertation concludes with a necessary discussion on how the presence of rapid guessing may impair operational procedures such as adaptive testing, equating, and standard setting. Solutions for testing professionals and future directions for researchers are offered.
Recommended Citation
Alahmadi, Sarah, "Investigating the direction of item parameter bias in the presence of rapid guessing" (2024). Dissertations, 2020-current. 143.
https://commons.lib.jmu.edu/diss202029/143
Included in
Educational Assessment, Evaluation, and Research Commons, Educational Psychology Commons, Quantitative Psychology Commons