Creative Commons License
This work is licensed under a Creative Commons Attribution-Noncommercial-No Derivative Works 4.0 License.
Date of Graduation
Spring 2015
Document Type
Dissertation
Degree Name
Doctor of Philosophy (PhD)
Department
Department of Graduate Psychology
Advisor(s)
Deborah Bandalos
Abstract
Researchers have studied item serial-order effects on attitudinal instruments by considering how item-total correlations differ based on the item’s placement within a scale (e.g., Hamilton & Shuminsky, 1990). In addition, other researchers have focused on item negative-keying effects on attitudinal instruments (e.g., Marsh, 1996). Researchers consistently have found that negatively-keyed items relate to one another above and beyond their relationship to the construct intended to be measured. However, only one study (i.e., Bandalos & Coleman, 2012) investigated the combined effects of serial-order and negative-keying on attitudinal instruments. Their brief study found some improvements in fit when attitudinal items were presented in a unique, random order to each participant, which is easily implemented using computer survey software.
In this study I replicated and extended these findings by considering three attitudinal scales: Conformity Scale (Goldberg et al., 2006; Jackson, 1994) and two subscales of the Big Five – Conscientiousness and Agreeableness (John & Srivastava, 1999). In addition, I collected and analyzed qualitative data in the form of think-alouds and used these data to inform the quantitative results in an explanatory sequential mixed-methods design (Creswell, 2011). I administered three different groupings of the items on these three instruments to random groups of university students. The items were displayed in either a blocked (i.e., all positively-keyed items followed by all negatively-keyed items), alternating (i.e., items alternated keying every other item beginning with a positively-keyed item), or random (i.e., items presented in a different random order for each participant) order.
When each participant saw a different randomly-ordered version of the attitudinal scale, I found fewer expected measurement error correlations among items of the same keying and in close proximity (i.e., serial order) to one another. Moreover, in this random ordering, the modification indices associated with the suggested measurement error correlations were lower than in the other orderings. Finally, the fit of the model to the data was the best in the random ordering for all except the Agreeableness scale. Practitioners are urged to administer attitudinal scales in a computer-generated random order unique to each participant whenever possible.
Recommended Citation
Gerstner, Jerusha J., "Addressing serial-order and negative-keying effects: A mixed-methods study" (2015). Dissertations, 2014-2019. 16.
https://commons.lib.jmu.edu/diss201019/16
