An open card sort is a type of card sorting activity in which participants are given the freedom to create as many topics and categories as they want and name them in a way that makes most sense to them. 

Open Card Sort

A closed card sort is a type of card sorting activity in which participants are to sort the given topics into a pre-defined set of categories. Closed card sorts are typically used when researchers are trying to improve an already existing information architecture.

Closed Card Sort

Some of the methods to analyze data from a card sorting activity includes: simple summary, factor analysis, multidimensional scaling, and path analysis. These methods will help us understand how similar items/topics are with each other in order to determine which items UX researchers should add or remove from a group so that it would be more useful for the users.

Data Analysis and Interpretation for Card Sorting

An information architecture is a combination of labeling content, organizing them into categories, and creating task processes in a way that is easy for the user to understand and accomplish their goal(s).

Information Architecture

A Card Sort is used to understand how people think information within a website should be organized and named. It is often used to create or design an information architecture. 

- In a typical card sorting session, participants group the given topics into categories that make sense to them. 
- Users may also make changes to the cards you provided them by: removing a card they find unnecessary, creating a new card, renaming a card, changing the definition of a given card, and/or placing cards they feel belong that to multiple categories. 
- This can be carried out using sticky notes, paper cards, or a card sorting software.


Ohio State University, Columbus

University of California, Berkeley

Claremont Graduate University

University of Michigan - Ann Arbor

Research methods used by researchers to collect qualitative or quantitative data in studies.

Data collection methods in UX research

- The methods are very different based on costs, methods, and needs 
- Each method requires a different minimal sample size for statistical significance difference 


Different Types of Quantitative UX Research

Baxter, K., Courage, C., & Caine, K. (2015). Understanding Your Users: A Practical Guide to User Research Methods. Elsevier.

Understanding Your Users: A Practical Guide to User Research Methods

User surveys

- Contextual Inquiry

The study will be conducted in the natural environment of the participants. For example, if the participants are students, the environment would be their classroom instead of an office.

Ethnographic field studies in UX research 

Contextual inquiry is a type of ethnographic field study that involves in-depth observation and interviews of small samples of users to gain a deep understanding of work practices and behaviors. Typically, it is conducted during the early discovery stages for a new feature or product. 
Key definitions:
- Context: The research takes place in the users’ natural environment as they conduct their activities the way they normally would. The context could be in their home, office, or somewhere else entirely
- Inquiry: The researcher watches the user as she performs her task and asks for information to understand how and why users do what they do

Contextual inquiry is useful for many domains, but especially well suited for considering users’ interactions with complex systems and in-depth processes as well as the point of view of expert users.

Contextual inquiry

Users are accompanied one-on-one with a researcher in a lab setting, and are told to complete tasks on a product or service.

Usability-Lab Studies

Participants contribute to the study by presenting their ideal experiences with reasons. The necessary design elements and creative materials will be provided by the researchers.

Participatory Design

"Tightly scripted usability studies are performed with several participants, using precise and predetermined measures of performance."

Usability Benchmarking

Researchers conducting usability testings remotely, with tools "such as screen-sharing software and remote control capabilities."

Moderated Remote Usability Studies

Having users install research moderator tool software on their devices and observe their process/thoughts while they complete a task.

Unmoderated Remote Panel Studies

Open and/or close ended information issued by a chosen set of users via a feedback link, button, form, or email.

Customer Feedback

Researchers invite random site visitors to answer questions about their goals of using the website and their feedback. Researchers then observe their subsequent behavior.

True-Intent Studies

Researchers observe user behavior through the recording of screens or pages that users click or see. "It requires the site to be instrumented properly or the application to have telemetry data collection enabled."

Clickstream Analysis

It could be either a quantitative method or a qualitative method. "Participants are offered different visual-design alternatives and are expected to associate each alternative with a set of attributes selected from a closed list."



Desirability Studies

User's Experience Measurements

Qualitative Data

A research method where a researcher in a structured setting asks an individual questions about any given topic for the purpose of collecting data about that topic.  From a UX perspective, interviews are valued insofar as they solicit the real opinions and experiences that users have when interacting with any given site or system, and the information collected can help improve that system. 

User Interviews

Tree testing focuses on evaluating category labels. This type of study is called "tree" testing because it is assessing hierarchical menus. It is often implemented remotely and unmoderated, and primarily used to follow up card sorting. It also identifies if Information Architecture (IA) structure, labels, and placements align with users’ expectations.

Tree Testing 

Card Sorting in UX Research

A qualitative research method where participants log interactions and information related to a specific context over a period of time. Researchers give participants a device to record their lives related to the studied products. 




Diary/Camera Studies in UX Research

Conducted after figuring out your target users. It uses a prototype to explore how a user will use a product. You can use a real user or someone on the design team can act as the user. There will be another colleague that will record any difficulties or concerns raised by the team. A guideline for the walk-through can also be created. This is different from usability testing as it is not used to test hypotheses or part of the rigorous iterative design process.

Walk-throughs in UX Research

A questionnaire is a form designed to collect information from respondents through a series of questions. A variety of question types can be included such as open-ended, multiple choice, or a scale (e.g. On a scale of 1-5, rate your satisfaction with this product).

Questionnaires in UX Research

A focus group is a qualitative data collection method that involves small groups of people participating together in an interview focused on a specific issue or topic. The dynamic interaction among group members can often elicit richer, more detailed information than what might be uncovered in a traditional one-on-one interview. For example, this approach has become a standard technique in business and industry for understanding consumer tastes and preferences. Because these group discussions yield extensive qualitative data, the sessions are typically recorded and transcribed to facilitate thorough later analysis.

Focus Group

- Quantitative Usability Testing focuses more on using metrics to measure time to complete a task or success, etc. 
- Sample size: 35 participants or more 

Quantitative Usability Testing

- Analytics-based experiment to test differences in products, which help decide which variation of design is the best 

A/B Testing or Multivariate Testing

- A flexible tool to test people’s attitude and behavior in many contexts: live websites, emails, and usability testing 

Surveys and Questionnaires

- It quantitatively analyzes qualitative data, not a qualitative data collection method

Clustering Qualitative Data

- Quantitatively measures the desirability of products 

Desirability Studies 

- It is used to track users’ eye movements and understand where they look at in the interface, which helps designers and researchers to understand which content to emphasize or deemphasize 

Learn Before

Related

Learn After