- Avoid experimenter bias
- Encourage participant motivation
- Account for participants individual differences
- Account for differences in tasks and queries
- Control test collection characteristics
- Account for differences in timing response variable
- Compare against a strong baseline

Boston University

Search interfaces are evaluated based on usability:
- Effectiveness
- Efficiency
- Satisfaction

Evaluation of Search Interfaces

Hearst, M. A. (2009). Search user interfaces. Cambridge University Press. [https://searchuserinterfaces.com/book/]

Search User Interfaces

- Standard Information Retrieval Evaluation
- Informal Usability Testing
- Formal Studies and Controlled Experiments
- Longitudinal Studies
- Analyzing Search Engine Server Logs
- Large-Scale Log-Based Usability Testing (Bucket Testing)
- Special Concerns with Evaluating Search Interfaces

Learn Before

Related