Publication: User Experience Rating Scales with 7, 11, or 101 Points: Does It Matter?
No Thumbnail Available
Date
2017-02
Authors
Lewis, James R.
Erdinç, Oğuzhan
Journal Title
Journal ISSN
Volume Title
Publisher
Users Experience Professional Assoc, 140 N Bloomingdale Rd, Bloomingdale, Il 60108 USA
Abstract
There is a large body of work on the topic of the optimal number of response options to use in multipoint items. The takeaways from the literature are not completely consistent, most likely due to variation in measurement contexts (e.g., clinical, market research, psychology) and optimization criteria (e.g., reliability, validity, sensitivity, ease-of-use). There is also considerable research literature on visual analog scales (VAS), which are endpoint-anchored lines on which respondents place a mark to provide a rating. Typically, a VAS is a 10-cm line with the marked position converted to a 101-point scale (0-100).
Multipoint rating items are widely employed in user experience (UX) research. The use of the VAS, on the other hand, is relatively rare. It seems possible that the continuous structure of the VAS could offer some measurement advantages. Our objective for this study was to compare psychometric properties of individual items and multi-item questionnaires using 7- and 11-point Likert-type agreement items and the VAS in the context of UX research.
Some characteristics (e.g., means and correlations) of the VAS were different from the Likert-style (7- and 11-point items), so the VAS does not appear to be interchangeable with the Likert-style items. There were no differences in the classical psychometric properties of reliability and concurrent validity. Thus, we did not find any particular measurement advantage associated with the use of 7- point, 11-point, or VAS items. With regard to measurement properties, it doesn't seem to matter (but the literature suggests multipoint items are easier to use).
Description
Keywords
Likert-type item, visual analog scale, VAS, number of response options, user experience, Visual Analog Scales, Psychometric Evaluation, Optimal Number, Response Alternatives, Consumer Research, Categories Shall, Pain Intensity, Likert Scales, Good Enough, Reliability