Preferred Name
Kate
Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-No Derivative Works 4.0 International License.
ORCID
https://orcid.org/ 0000-0002-6322-5018
Date of Graduation
5-12-2022
Semester of Graduation
Spring
Degree Name
Master of Arts (MA)
Department
Department of Graduate Psychology
Abstract
Different levels of examinee motivation pose a validity threat to the interpretation of test scores. This problem is heightened in low-stakes, remote testing environments. Though some ways exist to gauge average motivation throughout testing, less ways exist to gauge motivation fluctuations throughout a single test. One of those ways is through response times. Specifically, rapid response behavior occurs when examinees quickly answer an item without reading or engaging with the item. At James Madison University (JMU), students participating in campus-wide Assessment Days typically experienced an in-person, proctored Assessment Day. However, that changed during the pandemic. During the pandemic, examinees participated in Assessment Day remotely and unproctored. This study investigated the changes in the type and quantity of rapid response behavior in examinees from before the pandemic to during the pandemic using latent class analysis. Examinees were categorized by their patterns of solution behavior indices, which capture whether or not an examinee used rapid response behavior on an item. The study indicated three types of patterns based on solution behavior indices: higher engagement (low rapid response behavior), partial engagement (moderate rapid response behavior), and low engagement. Examinee self-reported motivation, performance on the test (total score), and gender were used as validity evidence for the classification of examinees. Finally, administration time point (either pre-pandemic or during the pandemic) was related to class membership. Testing during the pandemic seemed to be characterized by a strong increase in rapid response behavior and thus low motivation compared to testing pre-pandemic.
