Creative Commons License
This work is licensed under a Creative Commons Attribution-Noncommercial-No Derivative Works 4.0 License.
Date of Graduation
Spring 2019
Document Type
Dissertation
Degree Name
Doctor of Philosophy (PhD)
Department
Department of Graduate Psychology
Advisor(s)
Dena A. Pastor
Deborah L. Bandalos
Christine E. DeMars
Monica K. Erbacher
Abstract
One type of context effect is a position effect, which implies parameters of an item are influenced by the position of the item on the test. Researchers often discuss two types of position effects: negative position effects and positive position effects (e.g., Albano, 2013; Debeer & Janssen, 2013). Items exhibiting negative position effects become harder when placed later on the test, whereas items exhibiting positive position effects become easier when placed later on the test. Researchers have primarily examined the underlying causes of position effects through an item or person perspective (e.g., Bulut, 2015; Kingston & Dorans, 1984; Qian, 2014). Researchers who adopted an examinee perspective on position effects exclusively studied the relationships among person variables and position effects. Researchers who adopted an item perspective on position effects exclusively studied the relationships among item variables and position effects. These two perspectives are limiting because they do not encourage researchers to consider the potential interactions among person variables, item variables, and position effects.
In this dissertation, I examined the underlying causes of position effects through an integrated perspective, where I studied the relationships among person variables, item variables, and position effects simultaneously. I conducted a true experiment in which I administered items from two low-stakes assessments in different order to two groups of examinees, examined the presence of position effects, and evaluated the degree to which position effects were moderated by different item (item length, number of response options, mental taxation, and graphic) and person variables (effort, change in effort, and gender). I modeled position effects and their relationships with item and person variables under the generalized linear mixed modeling (GLMM) framework.
On both assessments, I found items exhibited significant negative linear position effects on both assessments, with the magnitude of the position effects varying from item to item. Items became harder when placed later on the assessments but the extent to which they became harder differed slightly across items. Additionally, I found the position effects to be moderated by item difficulty and item length but not number of response options, mental taxation, or graphic. Easier and longer items were more prone to position effects than harder and shorter items; however, items varying in mental taxation, items containing a graphic, and items varying in response options were similarly prone to position effects. More so, I found examinee effort levels, change in effort patterns, and genders did not moderate the relationships among position effects and item features. Based on these findings, testing practitioners should be cautious about administering long or easy items in different order across forms and/or administrations.
Recommended Citation
Ong, Thai Quang, "Are all cognitive items equally prone to position effects? Exploring the relationships among item features and position effects" (2019). Dissertations, 2014-2019. 203.
https://commons.lib.jmu.edu/diss201019/203
