Date of Award

Spring 2013

Document Type

Dissertation

Degree Name

Doctor of Philosophy (PhD)

Department

Department of Graduate Psychology

Advisor(s)

Deborah Bandalos

Christine E. DeMars

Sara J. Finney

Abstract

Researchers often collect data on attitudes using “balanced” measurement scales—that is, scales with comparable numbers of positive and negative (i.e., reverse-scored) items. Many previous measurement studies have found the inclusion of negative items to be detrimental to scale reliability and validity. However, these studies have rarely distinguished among negatively-worded items, negatively-keyed items, and items with negative wording and keying. The purpose of the current study was to make those distinctions and investigate why the psychometric properties of balanced scales tend to be worse than those of scales with uniformly positive wording/keying. A mixed-methods approach was employed. In Study 1 (quantitative), item wording and keying were systematically varied in adaptations of two published attitude measures that were administered to a large college student sample. Reliability and dimensionality of the resulting data were examined across the measures in each of four wording/keying configurations. Study 2 (qualitative) incorporated a mix of the same four wording/keying conditions in an adapted measure that was administered individually to a small sample of college students. A think-aloud design was implemented to elicit verbalizations that were subsequently analyzed using a thematic networks approach. Study 1 findings indicated that reliability estimates were generally highest for scales where all items were positively worded/keyed and lowest for scales with balanced keying (or balanced keying and wording). Regarding dimensionality, method variance was more evident when keying was balanced than when keying was consistent. This tended to be the case whether wording was balanced or consistent. Study 2 revealed a number of factors that could contribute to differences in the response patterns elicited by negative and positive items. These factors included the relative difficulty of processing negatively-worded statements, respondent characteristics such as reading skill and frustration tolerance, and idiosyncratic response styles. Among previously posited explanations for the differential functioning of negative and positive items, results from the studies supported some explanations (e.g., method variance; careless responding) more than others (e.g., the substantive explanation). Overall, it appeared that the psychometric consequences of balanced keying are no less substantial than those of balanced wording. An important question raised by the findings is whether the apparent advantage of consistent keying (in terms of reliability and dimensionality) came at the expense of validity, since careless responding and other forms of satisficing may be masked when keying is not balanced.

Included in

Psychology Commons

Share

COinS
 
 

To view the content in your browser, please download Adobe Reader or, alternately,
you may Download the file to your hard drive.

NOTE: The latest versions of Adobe Reader do not support viewing PDF files within Firefox on Mac OS and if you are using a modern (Intel) Mac, there is no official plugin for viewing PDF files within the browser window.