“Pick Some” is more like a technique that could be applied to the Forced Ranking Scale. When researchers have more than 10, especially more than 20 items for participants to pick, they could randomize the items and ask participants to pick a fixed subset. This type of scale could get very similar results as the Forced Ranking Scale but cost less time. 


University of Michigan - Ann Arbor

Likert and Semantic Differential are two of the most common rating scale systems within survey design and UX Research. Both measure various degrees of opinion, attitudes, perceptions, user preferences. Data collected helps with improving the user experience.

Types of Rating Scales in User Surveys 

Sauro, J. (2018, August 14). MeasuringU: 15 common rating scales explained. MeasuringU. https://measuringu.com/rating-scales/

15 Common Rating Scales Explained

Likert Scale created by social psychologist Rensis Likert in 1932. Purpose: Measure user agreement/opinions. Users are presented with set of statements, where then the user specifies their level of agreement or disagreement with each statement. Finally, after all responses have been analyzed, an overall position for the user is constructed.

Likert Scales in Surveys

Semantic Differential Scales were created in 1957 psychologist Charles Osgood. Contrary to Likert Scales, respondents rate their attitudes on a bipolar adjective scale. 

Semantic Differential Scales in User Surveys 

- Both measure degrees of opinions, but both have differences in terms of their implementation in research.
- More cognitive effort for Semantic Differential Scales 
- More flexibility and greater applications with Likert Scales. 

Likert vs. Semantic Differential Scales 

A Linear Numeric Scale provides a range of numbers for the participant to rate a question or statement. It usually has labels, such as “not at all likely”, “neutral”, and “extremely likely”, so participants could understand the measurement better. 


Linear Numeric Scale

This is a combination of the Linear Numeric Scale and the Likert Scale. The author states that “It’s probably not technically different from a Linear Numeric Scale but I’ve separated it out because they’re so popular for online surveys.” However, participants could be overwhelmed and confused if a matrix contains too many items.

Multiple Rating Matrix

It helps to learn how often a participant performs a certain action, and the number of times could be both specific, such as “every day”, or general, such as “always.” “The frequencies are sequentially ordered and well understood."

Frequency Scales

This type of scale is efficient to understand the priority of certain features or services. For a Linear Numeric Scale, participants might rate all items as “important.” When using a Forced Ranking Scale, all items are presented at once, and participants need to go through each one and rank them instead of evaluating each one individually, so the difference could be more distinctive. “This process is easier with a drag-and-drop interface as in the MUIQ item below.” However, researchers should avoid presenting too many items. The author suggests listing fewer than 10 features, but if a product has more than 10 features, the researchers could ask the participants to only pick some of them, such as “picking the top five.”


Forced Ranking Scale

Pick Some (a.k.a Top Task)

This type of scale helps to understand user preferences. Participants are required to pick one from two options, such as choosing a website or brand they prefer. The options could also be pictures or video instead of text. 

Paired Comparison Scale

Participants are asked to rate what they like and the level of preference on multiple items, and researchers could learn the comparison between them.


Comparative Scale/Comparative Intensity

Researchers could use the Adjective Checklist to learn brand attitude. Please remember to randomize the orders. Microsoft Desirability Toolkit also employed this technique.

Adjective Checklist


Instead of using numeric or adjectives, this scale presents a list of terms for participants to rate the level of intensity of each one. "It’s sort of a cross between the adjective scale and semantic differential scale."


Semantic Distance Scale

Researchers could set a fixed amount, such as 100, and ask participants to distribute the number to different items. To make the task easier, researchers should also show a number indicating the remaining amount. Similar to the Forced Ranking Scale, this is another way to force participants to rank the items based on priority.

Fixed Sum

A Compound Matrix could collect more information for one item, “for example, the importance of features by device type for online banking.” The option fields could be drop-down lists or text boxes.


Compound Matrix

This is similar to a Linear Numeric Scale but without discrete points. Participants are given the freedom to choose any values. However, “the impact of slider scales attributes (labels, showing values, starting position of slider) is still being investigated.”

Visual Analog/Slider Scale

Pictorial/Graphic Scales use graphics instead of text or numerics. The Wong-Baker faces pain scale is an example of the Pictorial/Graphic Scales. Another commonly seen example is the star-rating scale in Google Review and Amazon customer reviews. "Pictorial scales can be particularly helpful when participants might not speak the target language well or even have trouble communicating (hence its widespread use in medical settings)."

Learn Before

Related