Creative Commons License
This work is licensed under a Creative Commons Attribution-Noncommercial-No Derivative Works 4.0 License.
Date of Graduation
Spring 5-3-2013
Document Type
Dissertation
Degree Name
Doctor of Philosophy (PhD)
Department
Department of Graduate Psychology
Advisor(s)
Deborah Bandalos
Christine E. DeMars
Sara J. Finney
Abstract
Researchers often collect data on attitudes using “balanced” measurement scales—that is, scales with comparable numbers of positive and negative (i.e., reverse-scored) items. Many previous measurement studies have found the inclusion of negative items to be detrimental to scale reliability and validity. However, these studies have rarely distinguished among negatively-worded items, negatively-keyed items, and items with negative wording and keying. The purpose of the current study was to make those distinctions and investigate why the psychometric properties of balanced scales tend to be worse than those of scales with uniformly positive wording/keying. A mixed-methods approach was employed. In Study 1 (quantitative), item wording and keying were systematically varied in adaptations of two published attitude measures that were administered to a large college student sample. Reliability and dimensionality of the resulting data were examined across the measures in each of four wording/keying configurations. Study 2 (qualitative) incorporated a mix of the same four wording/keying conditions in an adapted measure that was administered individually to a small sample of college students. A think-aloud design was implemented to elicit verbalizations that were subsequently analyzed using a thematic networks approach. Study 1 findings indicated that reliability estimates were generally highest for scales where all items were positively worded/keyed and lowest for scales with balanced keying (or balanced keying and wording). Regarding dimensionality, method variance was more evident when keying was balanced than when keying was consistent. This tended to be the case whether wording was balanced or consistent. Study 2 revealed a number of factors that could contribute to differences in the response patterns elicited by negative and positive items. These factors included the relative difficulty of processing negatively-worded statements, respondent characteristics such as reading skill and frustration tolerance, and idiosyncratic response styles. Among previously posited explanations for the differential functioning of negative and positive items, results from the studies supported some explanations (e.g., method variance; careless responding) more than others (e.g., the substantive explanation). Overall, it appeared that the psychometric consequences of balanced keying are no less substantial than those of balanced wording. An important question raised by the findings is whether the apparent advantage of consistent keying (in terms of reliability and dimensionality) came at the expense of validity, since careless responding and other forms of satisficing may be masked when keying is not balanced.
Recommended Citation
Coleman, Chris M., "Effects of negative keying and wording in attitude measures: A mixed-methods study" (2013). Dissertations, 2014-2019. 73.
https://commons.lib.jmu.edu/diss201019/73
