- Testing several design elements at once 

Multivariate Testing 

- Many teams fail to run experiments properly and make risky decisions 

A/B Testing or Multivariate Testing--Drawback 

Test two different versions of the same UIs

A/B Testing 

- Analytics-based experiment to test differences in products, which help decide which variation of design is the best 

Claremont Graduate University

- The methods are very different based on costs, methods, and needs 
- Each method requires a different minimal sample size for statistical significance difference 


Different Types of Quantitative UX Research

World Leaders in Research-Based User Experience. (n.d.). Quantitative user-research methodologies: An overview. Nielsen Norman Group. Retrieved March 5, 2022, from https://www.nngroup.com/articles/quantitative-user-research-methods/ 

Quantitative user-research methodologies: An overview

- Quantitative Usability Testing focuses more on using metrics to measure time to complete a task or success, etc. 
- Sample size: 35 participants or more 

Quantitative Usability Testing

A/B Testing or Multivariate Testing

- A flexible tool to test people’s attitude and behavior in many contexts: live websites, emails, and usability testing 

Surveys and Questionnaires

- It quantitatively analyzes qualitative data, not a qualitative data collection method

Clustering Qualitative Data

- Quantitatively measures the desirability of products 

Desirability Studies 

- It is used to track users’ eye movements and understand where they look at in the interface, which helps designers and researchers to understand which content to emphasize or deemphasize 

Eye-tracking Testing

Tree testing focuses on evaluating category labels. This type of study is called "tree" testing because it is assessing hierarchical menus. It is often implemented remotely and unmoderated, and primarily used to follow up card sorting. It also identifies if Information Architecture (IA) structure, labels, and placements align with users’ expectations.

Tree Testing 

A Card Sort is used to understand how people think information within a website should be organized and named. It is often used to create or design an information architecture. 

- In a typical card sorting session, participants group the given topics into categories that make sense to them. 
- Users may also make changes to the cards you provided them by: removing a card they find unnecessary, creating a new card, renaming a card, changing the definition of a given card, and/or placing cards they feel belong that to multiple categories. 
- This can be carried out using sticky notes, paper cards, or a card sorting software.


Learn Before

Related

Learn After