The Role of Funnel Items on Response Styles
Faculty Advisor Name
Brian Leventhal
Department
Department of Graduate Psychology
Description
Response styles, the tendency to respond to items systematically, influence the interpretability of scores by introducing construct-irrelevant variance. Baumgartner and Steenkamp (2001) summarize seven distinct response styles identified in the literature that plague interpretations of scores, with midpoint response style (MRS) and extreme response style (ERS) the most-cited (van Vaerenbergh & Thomas, 2013). MRS is an individual’s tendency to select the middle response option and ERS is a tendency to select the endpoints of the response scale. Böckenholt (2017) suggests designing tree-like funnel items to disaggregate response style effects from TOI effects during data collection. In the current study, we investigate whether the funnel items suggested by Böckenholt assist in detecting extreme and midpoint response styles. In the current study, we re-design traditional Likert items into items mimicking the hypothesized decision process respondents use. Using a six-item Intellectual Overconfidence (IO) scale and a five-item Confidence in Communication (CC) scale, we assigned random samples of first-year college students to use either traditional Likert items or funnel items. We used the following procedure to operationalize funnel items. The first asked whether the participant had an opinion. If “yes,” respondents reported whether they agreed or disagreed to the prompt, followed by the next funnel item to report the strength of that opinion. To understand MRS and ERS we investigate the frequency response style indicator (Greenleaf, 1992; Reynolds & Smith, 2010) in which the number of extreme and midpoint responses on each scale are summed. There was a significantly higher mean number of extreme responses with the funnel item as compared to the traditional item on the CC scale, t(4375)=8.20,p<.001, and on the IO scale, t(4375)=8.28,p<.001. On average, there were significantly fewer midpoint responses with the funnel item as compared to the traditional item on the IO scale, t(940.52)=4.55,p<.001, but more midpoint responses on the IO scale, t(4375)=-0.96,p=.336, although this difference was not significant. Moreover, there was a significant relationship between the format of the item and the number of extreme responses on both scales (IO: χ_(df=6)^2=188.65,p<.001; CC: χ_(df=5)^2=229.68,p<.001) as well as between item-format and the number of midpoint responses on the IO scale (χ_(df=6)^2=59.14,p<.001), but not the CC scale (χ_(df=5)^2=10.06,p=.074). Given the detrimental effects of response styles, such as confounding the interpretation of scores (Bolt & Johnson, 2009; Jin & Wang, 2014), we shed light on the difference in efficacy between funnel items and traditional items. Results suggest this design can shed light on the presence of extreme response style and potentially mitigate the effects of midpoint response style.
The Role of Funnel Items on Response Styles
Response styles, the tendency to respond to items systematically, influence the interpretability of scores by introducing construct-irrelevant variance. Baumgartner and Steenkamp (2001) summarize seven distinct response styles identified in the literature that plague interpretations of scores, with midpoint response style (MRS) and extreme response style (ERS) the most-cited (van Vaerenbergh & Thomas, 2013). MRS is an individual’s tendency to select the middle response option and ERS is a tendency to select the endpoints of the response scale. Böckenholt (2017) suggests designing tree-like funnel items to disaggregate response style effects from TOI effects during data collection. In the current study, we investigate whether the funnel items suggested by Böckenholt assist in detecting extreme and midpoint response styles. In the current study, we re-design traditional Likert items into items mimicking the hypothesized decision process respondents use. Using a six-item Intellectual Overconfidence (IO) scale and a five-item Confidence in Communication (CC) scale, we assigned random samples of first-year college students to use either traditional Likert items or funnel items. We used the following procedure to operationalize funnel items. The first asked whether the participant had an opinion. If “yes,” respondents reported whether they agreed or disagreed to the prompt, followed by the next funnel item to report the strength of that opinion. To understand MRS and ERS we investigate the frequency response style indicator (Greenleaf, 1992; Reynolds & Smith, 2010) in which the number of extreme and midpoint responses on each scale are summed. There was a significantly higher mean number of extreme responses with the funnel item as compared to the traditional item on the CC scale, t(4375)=8.20,p<.001, and on the IO scale, t(4375)=8.28,p<.001. On average, there were significantly fewer midpoint responses with the funnel item as compared to the traditional item on the IO scale, t(940.52)=4.55,p<.001, but more midpoint responses on the IO scale, t(4375)=-0.96,p=.336, although this difference was not significant. Moreover, there was a significant relationship between the format of the item and the number of extreme responses on both scales (IO: χ_(df=6)^2=188.65,p<.001; CC: χ_(df=5)^2=229.68,p<.001) as well as between item-format and the number of midpoint responses on the IO scale (χ_(df=6)^2=59.14,p<.001), but not the CC scale (χ_(df=5)^2=10.06,p=.074). Given the detrimental effects of response styles, such as confounding the interpretation of scores (Bolt & Johnson, 2009; Jin & Wang, 2014), we shed light on the difference in efficacy between funnel items and traditional items. Results suggest this design can shed light on the presence of extreme response style and potentially mitigate the effects of midpoint response style.
