The role of response format in reverse-keying method effects : a comparison of bipolar and unipolar Likert-type response scales

Abstract

Reverse-keyed items are widely used in self-report psychological measures to reduce response bias, but they often introduce psychometric complications commonly described as method effects. Previous research has focused primarily on item wording or respondent characteristics to explain these effects, with the role of response format being under-researched. This dissertation investigates the impact of bipolar versus unipolar agreement-based Likert-type response scales on measures’ psychometric properties and participants’ response processes using two studies. Study 1 collected survey data from 620 adults who completed mixed-keyed measures using either a bipolar or unipolar response scale. Contrary to the expectation that unipolar scales might mitigate method effects by simplifying the response task, results indicated that they amplified item-keying effects. Under the unipolar scale condition, regular-keyed items showed flatter distributions, whereas reverse-keyed items were increasingly skewed. The study also identified mismatched item extremity, specifically the practice of softening reverse-keyed items with frequency modifiers, as an overlooked contributor to method variance. Study 2 used cognitive interviewing with 20 adults to compare a unipolar scale group with a matched bipolar scale group and identified validity concerns at the response selection stage. The findings provided a cognitive explanation for patterns observed in the Study 1 quantitative data by revealing differences in response strategies. Although the unipolar response scale reduced directional confusion during response mapping compared to the bipolar scale, it unexpectedly triggered dichotomous thinking. Respondents often reported treating the unipolar continuum as a binary choice and used the option at the low end of the scale (“not at all agree”) to reject negative self- evaluative statements. Together, these findings caution against the use of unipolar agreement scales in mixed-keyed measures. They suggest that response scale polarity does not simply make the response task easier or harder; it may change participants’ response processes and produce distinct validity concerns. This dissertation contributes to the field by highlighting the value of exploring response process data alongside psychometric properties and emphasizing that item characteristics and response scale format should be treated as an interactive system in measurement design.