- Can produce quantitative and qualitative data: ratings, proportions of answers from questionnaires, and open-ended responses 
- Qualitative data in surveys can be turned into quantitative data 


Surveys and Questionnaires-- Function

The psychological process of responding to a survey item can be modeled as a sequential series of five cognitive processes. First, the respondent engages in question interpretation to comprehend the item's meaning. Next, they perform information retrieval to access relevant memories or beliefs. Third, they use judgment formation to evaluate the retrieved information. Fourth, response formatting occurs as they map their judgment onto the available response options. Finally, response editing allows the participant to adjust their answer before submitting it, often to align with social expectations.

Cognitive Model of Survey Responding

In survey research, context effects are unintended influences on a respondent's answers that arise not from the specific content of the question itself, but from the surrounding context in which the item is presented. These biases can occur when previous questions, response options, or the overall survey format alter a participant's interpretation of an item or the information they retrieve from memory.

Survey Context Effects

Survey formatting is the structural organization of a questionnaire designed to maintain respondent engagement and minimize fatigue. Beyond simply writing clear items, effective formatting involves crafting an introduction to secure informed consent, providing explicit instructions with examples, and strategically sequencing the items. Best practices include placing the most critical questions early in the survey, grouping similar items together, reserving demographic questions for the end, and always concluding with an expression of appreciation to the respondent.

Survey Formatting

- A flexible tool to test people’s attitude and behavior in many contexts: live websites, emails, and usability testing 

Claremont Graduate University

- The methods are very different based on costs, methods, and needs 
- Each method requires a different minimal sample size for statistical significance difference 


Different Types of Quantitative UX Research

World Leaders in Research-Based User Experience. (n.d.). Quantitative user-research methodologies: An overview. Nielsen Norman Group. Retrieved March 5, 2022, from https://www.nngroup.com/articles/quantitative-user-research-methods/ 

Quantitative user-research methodologies: An overview

- Quantitative Usability Testing focuses more on using metrics to measure time to complete a task or success, etc. 
- Sample size: 35 participants or more 

Quantitative Usability Testing

- Analytics-based experiment to test differences in products, which help decide which variation of design is the best 

A/B Testing or Multivariate Testing

Surveys and Questionnaires

- It quantitatively analyzes qualitative data, not a qualitative data collection method

Clustering Qualitative Data

- Quantitatively measures the desirability of products 

Desirability Studies 

- It is used to track users’ eye movements and understand where they look at in the interface, which helps designers and researchers to understand which content to emphasize or deemphasize 

Eye-tracking Testing

Tree testing focuses on evaluating category labels. This type of study is called "tree" testing because it is assessing hierarchical menus. It is often implemented remotely and unmoderated, and primarily used to follow up card sorting. It also identifies if Information Architecture (IA) structure, labels, and placements align with users’ expectations.

Tree Testing 

A Card Sort is used to understand how people think information within a website should be organized and named. It is often used to create or design an information architecture. 

- In a typical card sorting session, participants group the given topics into categories that make sense to them. 
- Users may also make changes to the cards you provided them by: removing a card they find unnecessary, creating a new card, renaming a card, changing the definition of a given card, and/or placing cards they feel belong that to multiple categories. 
- This can be carried out using sticky notes, paper cards, or a card sorting software.


Learn Before

Related

Learn After