This study explored the use of the five invalidity flags plus a new sixth flag based on self-reported effort. Participants were 155 entering first-year university students who were measured during an orientation week and again 18 months later. The instruments were a faculty-developed test of oral communications skills with 40 four-option multiple-choice items and a self-reported measure of test-taking motivation (Student Opinion Survey; Sundre, 1999 adapted from Wolf and Smith, 1995). Results indicated that the Flags explored in this study generalized well to university students. There was a moderate correlation between Response Time Effort and Effort as measured by the Student Opinion Scale, suggesting there was a relationship not captured by the dichotomized flags.
Creative Commons License
This work is licensed under a Creative Commons Attribution-Noncommercial-No Derivative Works 4.0 License.
DeMars, C. E., Wise, S. L., & Smith, L. F. (2009, April). Individual score validity and student effort in higher education assessment. Paper presented at the annual meeting of the National Council on Measurement in Education, San Diego, CA.